Search

Few-Shot Class-Incremental Learning (FSCIL)presents an extension of the Class Incremental Learning (CIL)problem where a model is faced with the problem of data scarcity while addressing the Catastrophic Forgetting (CF)problem. This problem remains an open problem because all recent works are built upon the Convolutional Neural Networks (CNNs)performing sub-optimally compared to the transformer approaches. Our paper presents Robust Transformer Approach (ROBUSTA)built upon the Compact Convolutional Transformer (CCT). The issue of overfitting due to few samples is overcome with the notion of the stochastic classifier, where the classifier's weights are sampled from a distribution with mean and variance vectors, thus increasing the likelihood of correct classifications, and the batch-norm layer to stabilize the training process. The issue of CFis dealt with the idea of delta parameters, small task-specific trainable parameters while keeping the backbone networks frozen. A non-parametric approach is developed to infer the delta parameters for the model's predictions. The prototype rectification approach is applied to avoid biased prototype calculations due to the issue of data scarcity. The advantage of ROBUSTAis demonstrated through a series of experiments in the benchmark problems where it is capable of outperforming prior arts with big margins without any data augmentation protocols. [ABSTRACT FROM AUTHOR]

Adversarial Training has proved to be an effective training paradigm to enforce robustness against adversarial examples in modern neural network architectures. Despite many efforts, explanations of the foundational principles underpinning the effectiveness of Adversarial Training are limited and far from being widely accepted by the Deep Learning community. Moreover, very few research works investigated the limitations of robust Convolutional Neural Networks beyond the well-known accuracy drop on natural images. In this paper, we describe surprising properties of these models, shedding light on mechanisms through which robustness against adversarial attacks is implemented. We also highlight limitations and failure modes that were not discussed in prior works. Through extensive analyses on a wide range of architectures and datasets, we empirically demonstrate that adversarially-trained Convolutional Neural Networks do not exploit efficiently the model capacity and that the simplicity biases induced by Adversarial Training may lead to undesired behaviors. [ABSTRACT FROM AUTHOR]

• First introduced PD control theory for optimizing learning rate in CNN models. • Designed of innovative center selection and elite guidance strategies. • Developed an efficient enhanced battle royale algorithm(EBRO). • Proposed a new EBRO-ICNN model for air quality prediction. PM 2.5 concentration predictions can provide air pollution control, management, and early warning. However, the PM 2.5 data with high-dimensionality, complexity, and dynamics pose a great challenge to achieve optimal prediction results. Convolutional Neural Networks (CNNs) has unique advantages in processing complex data and contributes to state-of-the-art performances. However, designing the architecture and selecting the learning rate for CNN are time-consuming and requires prior knowledge. Evolutionary algorithms, with the advantages of global convergence, ergodicity, robustness and adaptability, are the most commonly used methods to design the optimal framework for CNNs. Therefore, to improve the predictive performance of CNNs, this paper proposes an improved CNN method (EBRO-ICNN) which employs the enhanced battle royale optimization (EBRO) algorithm and proportional-derivative (PD) control. Firstly, the EBRO algorithm with multistrategy collaborative optimization is introduced, and validated by CEC2017 benchmark functions, which demonstrates EBRO strong global exploration capability, fast convergence speed, and low time complexity. Next, PD control is applied to adjust the learning rate of CNN (ICNN) dynamically, which improves the efficiency and stability of the network training process. At the same time, the ICNN model is optimized using the EBRO algorithm, which can reduce the human interference, generate the optimal framework automatically and enhance the prediction accuracy effectively. Finally, a private air quality dataset and two public datasets are utilized to evaluate the performance of the EBRO-ICNN model, considering 3 error evaluation metrics and 7 prediction comparison models. The experimental results demonstrate that the EBRO-ICNN model exhibits great accuracy and stability. [ABSTRACT FROM AUTHOR]

• Equity value and debt maturity make this portfolio more complete by using KMV model. • The objectives of the proposed portfolio include order moments and default distance. • Improved NSGA-III with chaos map is developed to solve five-objective optimization. • Static penalty is transformed the constrained optimization to unconstrained problem. • CNN-BiLSTM is firstly used to predict trend when selecting stocks. Portfolio management is a critical aspect of investment strategies, with the goal to balance the low-risk and high-return investments. Despite this, existing portfolios frequently overlook the integration of stock selection outcomes and underutilize data from listed companies, leading to suboptimal portfolio performance. Addressing these shortcomings, this paper introduces a hybrid system involving stock selection and portfolio optimization. In stock selection, the system employs a combination of convolutional neural network and bi-directional recurrent neural network to predict stock trends. This approach enables the identification of stocks likely to appreciate in value, setting the stage for their inclusion in the subsequent optimization process. For portfolio optimization, the study formulates a five-objective optimization problem that incorporates mean, variance, skewness, kurtosis, and distance-to-default as key considerations. To solve the many-objective constrained optimization problem, an advanced strategy employing a static penalty function and an improved Non-dominated Sorting Genetic Algorithm III (NSGA-III) based on tent chaotic mapping is utilized. The efficacy of the proposed hybrid system is rigorously tested through three sets of ablation experiments alongside two discussions focused on its robustness and computational efficiency. The findings from these investigations reveal that the hybrid system outperforms traditional approaches, reducing risks and improving returns for investors. [ABSTRACT FROM AUTHOR]

Palmprint recognition techniques have been widely applied in security authentication. As a typical image processing method, convolutional neural network (CNN) has been applied to extract features from palmprint images. However, most existing CNN methods cannot meet the demand in recognition speed due to their high computational complexity. This paper proposes a simple yet effective network (SYEnet) for palmprint recognition, which is a lightweight neural network. SYEnet is composed of four learnable feature extraction networks, which can extract discriminative information from 9 patches. In addition, a dedicated network for palmprint ROI extraction (PREnet) is designed. PREnet extracts ROI from the original image, generating ROI images of consistent dimensions, which can be adapted to complex environments without limitations of lighting, gesture, image quality, and resolution. Moreover, a new contactless palmprint database is collected to validate our method. The database contains 12,800 images from 400 palms and provides data supplementation for current research on palmprint recognition. The proposed SYEnet is evaluated on eight real-world palmprint databases, in comparison to other related methods. Experimental results prove that our method is more efficient and has higher recognition accuracy. [ABSTRACT FROM AUTHOR]

Temporal Knowledge Graphs (TKGs) model time-dependent facts as relations between entities at specific timestamps, making them well-suited for real-world scenarios. However, TKGs are susceptible to incompleteness, necessitating Temporal Knowledge Graph Completion (TKGC) to predict missing facts. Prior methods often struggle to effectively handle two critical properties of TKGs, time-variability and time-stability, simultaneously, which hinders their performance. In this paper, we propose Space Adaptation Network (SANe), a novel approach for TKGC. SANe adapts facts at different timestamps to distinct latent spaces, effectively addressing time-variability. Our model introduces Parameter Generation Network to produce separate neural networks for each snapshot, which are then encoded into different latent spaces. A dynamic convolutional neural network processes entities and relations, utilizing different learned parameters generated by parameter generation network with respect to timestamps. By handling different temporal snapshots separately, TKGC is transformed into static KGC, enabling the modeling of time-variability. Dynamic convolutional neural network efficiently learns collective knowledge over large periods and supplements more specific knowledge gradually in smaller periods, facilitating time-stability. To strike a balance between learning time-variability and time-stability, we introduce a time-aware parameter generator to produce parameters hierarchically based on year, month, and day timestamps. Long-term knowledge is effectively shared across adjacent snapshots within the same year or month, while short-term knowledge within a day is preserved in specific parameters. However, in unbalanced TKGs, where many facts occur in small intervals, the large number of parameters generated by time-aware parameter generator may remain underutilized. To address this, we propose Adaptive Parameter Generation with a partition tree, ensuring parameter load balancing while maintaining time-stability. We conduct extensive experiments on five benchmark datasets, demonstrating the superiority of SANe over existing methods for TKGC, achieving state-of-the-art performance. Our contributions include pioneering TKGC from the perspective of space adaptation, achieving a balance between time-variability and time-stability through latent space overlap constraints, and substantiating the effectiveness of our model through comprehensive experiments on rich temporal datasets. [ABSTRACT FROM AUTHOR]

The 12-lead electrocardiogram (ECG) is a reliable diagnostic tool for detecting and treating severe cardiovascular conditions like arrhythmia and heart attack. Deep neural networks (DNNs) have achieved higher accuracy in recent years than traditional ECG signal classification task methods. Convolutional neural network (CNN) and Transformer are the two mainstream architectures of DNN, respectively good at extracting local and global features from input data. This paper proposes the multi-scale convolutional Transformer network (MCTnet), an efficient combination of Transformer encoder and CNN for ECG signal classification. MCTnet utilizes the advantages of CNN and self-attention mechanisms to capture potential features in ECG signal accurately. The dual-branch Transformer encoder extracts different-scale feature representations, enabling the capture of both local and global information. Additionally, an information bottleneck method eliminates redundant information and enhances task-relevant information in the learned representations. To evaluate the performance of MCTnet, comprehensive experiments are conducted on three commonly used ECG datasets. The results demonstrate that MCTnet outperforms current deep learning-based models, highlighting its effectiveness in ECG signal classification. It also shows that the performance of the model can be effectively improved by utilizing multi-scale representation learning and information bottleneck. [ABSTRACT FROM AUTHOR]

Providing interpretability of deep-learning models to non-experts, while fundamental for a responsible real-world usage, is challenging. Attribution maps from xAI techniques, such as Integrated Gradients, are a typical example of a visualization technique containing a high level of information, but with difficult interpretation. In this paper, we propose two methods, Maximum Activation Groups Extraction (MAGE) and Multiscale Interpretable Visualization (Ms-IV), to explain the model's decision, enhancing global interpretability. MAGE finds, for a given CNN, combinations of features which, globally, form a semantic meaning, that we call concepts. We group these similar feature patterns by clustering in "concepts", that we visualize through Ms-IV. This last method is inspired by Occlusion and Sensitivity analysis (incorporating causality) and uses a novel metric, called Class-aware Order Correlation (CAOC), to globally evaluate the most important image regions according to the model's decision space. We compare our approach to xAI methods such as LIME and Integrated Gradients. Experimental results evince the Ms-IV higher localization and faithfulness values. Finally, qualitative evaluation of combined MAGE and Ms-IV demonstrates humans' ability to agree, based on the visualization, with the decision of clusters' concepts; and, to detect, among a given set of networks, the existence of bias. • We introduce Maximum Activation Groups Extraction (MAGE) to represent feature maps. • The representations are based on activation patterns localization in multiple images. • We introduce Class-aware Order Correlation (CAOC), to determine occlusion impacts. • We introduce Multiscale Interpretable Visualization (Ms-IV) highlighting image parts. • CAOC , used with Ms-IV, considers the relation of dataset images according to the model. [ABSTRACT FROM AUTHOR]

Filter pruning has been successfully applied in convolutional neural networks (CNN) to reduce model redundancy and speed up inference. Current pruning algorithms require setting a target value for a pruned model, which is not conducive to the rapid implementation in practical application. Firstly, for a target model, it may be necessary to pay attention to multiple performances at the same time, but it is hard to keep the best trade-off among indicators. Secondly, it is time-consuming to find a suitable model by pruning in a trial-and-error way. Therefore, this paper attempts to combine the most concerning indicators into a search target, rather than producing a model that only contains some pre-defined constraint. Specifically, the multi-object differential evolution algorithm is introduced into the search process, which is used to obtain a set of sparse structures that have taken all useful indicators into account. Then select proper sub-network to retrain from search results according to actual requirements. Experiments on multiple advanced CNN networks show that our algorithm can compress different indicators while keeping comparable or better performance than prior works. On ImageNet, pruning based on ResNet50 reduces FLOPs by 60.4%, while the accuracy of top-1 and top-5 only loses 1.09% and 0.57%, respectively. [ABSTRACT FROM AUTHOR]

Network pruning has been a hot topic in recent years, and many popular pruning methods rely on network design expertise. However, the pruning process usually involves manual intervention and can be difficult for users who lack prior knowledge. Automatic pruning using evolutionary algorithms shows great promise, but it must address the challenge of performing time-consuming model evaluations and searching through a large solution space. Dataset distillation is a technique that compresses the original dataset to decrease the cost of fine-tuning models. In this paper, we explore the potential of using the distilled dataset to exhibit a similar role as the real dataset in network pruning, and proposed the evolutionary pruning framework using distilled dataset. Specifically, the network pruning pipeline is carried out on the distilled dataset to significantly reduce the model evaluation cost, and the number of filters in the convolutional layer is directly coded to narrow the search space. In addition, a tailored evolutionary algorithm is proposed that takes the form of constrained optimization to search the most suitable pruned network. The experiments conducted on VGG16, VGG19, ResNet56, and ResNet110 demonstrate that the proposed method reduces at least 41.56% of the flops and achieves competitive results with little compromising accuracy. • Distillation dataset is first applied to network pruning. • A constrained evolutionary algorithm with double-balanced multi-branch is proposed. • Deep convolutional neural network pruning is performed using the constrained evolutionary algorithm. [ABSTRACT FROM AUTHOR]