Search

Extensive research has been conducted on analyzing human movements, driven by its diverse practical applications such as human–robot interaction, human learning, and clinical diagnosis. However, the current state-of-the-art still encounters scientific challenges when it comes to modeling human movements. There are two key aspects that need to be addressed. Firstly, new models should consider the stochastic nature of human movement and the physical structure of the human body to accurately predict the patterns in full-body motion descriptors over time. Secondly, while deep learning algorithms have been utilized, they lack explainability in terms of predicting body posture sequences, making it essential to improve their comprehensible representation of human movement. This paper aims to tackle these challenges by presenting three innovative methods for creating explainable representations of human movement. The study formulates human body movement as a state-space model based on the Gesture Operational Model (GOM). Model parameters are estimated through either one-shot training employing Kalman Filters or data-intensive training utilizing artificial neural networks. The trained models are utilized for analyzing the dexterity of expert professionals in full-body movements, enabling the identification of dynamic associations between body joints and gesture recognition. Additionally, these models are employed to generate artificial professional movements. • Three innovative methods for explainable human movement models. • Body dexterity analysis: Models reveal how artisans and operators perform tasks. • Minimal motion descriptors to enhance task recognition. • Artificial generation of full-body professional movements. • Extensive experimentation with real-world datasets. [ABSTRACT FROM AUTHOR]

Active learning alleviates labeling costs by selecting and labeling the most informative examples from an unlabeled pool. However, most existing active learning approaches estimate informativeness with uncalibrated confidence, resulting in unreliable informativeness estimation. These approaches generally ignored two significant issues caused by uncalibrated confidence methods. Firstly, the average uncalibrated confidence generated by modern neural networks is usually higher than the accuracy. Secondly, examples located near the decision boundaries are unstable during prediction when the target model updates parameters in the last several epochs, even throughout the training process. This phenomenon, caused by the forgetting characteristic of neural networks, has a significant impact on some specific models that estimate the informativeness by predicted probability vectors or pseudo labels. To address these issues, in this paper, we propose a novel active learning approach to reliably estimate informativeness with calibrated confidence. Specifically, we integrate the intermediate predictions for each unlabeled example, generated by the target model during the training process, to generate calibrated confidence. The calibrated confidence can capture a tendentious label from an indecisive subset of the class space. We show that the calibrated confidence with tendentiousness can maintain the ability of correct predictions. The empirical results demonstrate that our approach outperforms the state-of-the-art active learning methods on image classification tasks. • The active learning algorithm selects unstable examples by calibrated confidence.. • The confidence calibration method only requires training a single model. • Extensive experiments show the competitive performance of the proposed approach. [ABSTRACT FROM AUTHOR]

• A dynamic SPECT kidney image database was established by manually labeling clinical renal dynamic images. • A multi-scale spatial perception module was proposed to enhance the semantic information of high-level feature maps. • Decoding modules with attention connection were proposed to refine the details gradually. • RASNet achieved the optimal performance in most of the indicators in automatic kidney segmentation. For the treatment of renal disease, the application of radioactive equipment has become one of the important methods. Accurate segmentation of renal contour plays an important role in clinical diagnosis. However, manual renal contour drawing is not only inefficient but also prone to inaccurate outlining results due to different manual proficiency and fatigue caused by long-term work. There is little research on automatic renal segmentation with renal dynamic imaging. To address this issue, an improved model based on a deep neural network called Renal Automatic Segmentation Network (RASNet) is proposed, to aid in the automatic segmentation of renal contours. Besides, a multi-scale spatial perception module and a decoding module with attention connection are introduced to enrich the semantic information and further improve the accuracy of network segmentation. Extensive experiments were conducted on a renal dynamic medical image database established in this paper. Analysis results show the superiority of the proposed RASNet to several existing segmentation frameworks. [ABSTRACT FROM AUTHOR]

Deep Neural Networks (DNNs) are vulnerable to invisible perturbations on the images generated by adversarial attacks, which raises researches on the adversarial robustness of DNNs. A series of methods represented by the adversarial training and its variants have proven as one of the most effective techniques in enhancing the DNN robustness. Generally, adversarial training focuses on enriching the training data by involving perturbed data. Such data augmentation effect of the involved perturbed data in adversarial training does not contribute to the robustness of DNN itself and usually suffers from clean accuracy drop. Towards the robustness of DNN itself, we in this paper propose a novel defense that aims at augmenting the model in order to learn features that are adaptive to diverse inputs, including adversarial examples. More specifically, to augment the model, multiple paths are embedded into the network, and an orthogonality constraint is imposed on these paths to guarantee the diversity among them. A margin-maximization loss is then designed to further boost such DIversity via Orthogonality (DIO). In this way, the proposed DIO augments the model and enhances the robustness of DNN itself as the learned features can be corrected by these mutually-orthogonal paths. Extensive empirical results on various data sets, structures and attacks verify the stronger adversarial robustness of the proposed DIO utilizing model augmentation. Besides, DIO can also be flexibly combined with different data augmentation techniques (e.g. , TRADES and DDPM), further promoting robustness gains. • A novel adversarial defense exploring model properties to improve DNNs' robustness • Multiple paths augment DNNs for diverse features adaptive to adversarial inputs • An orthogonality loss and a margin-maximization loss contribute to DNNs' diversity • The proposed method outperforms the non-data-augmented adversarial defenses • Compatibility with the data-augmented techniques, leading to improved robustness [ABSTRACT FROM AUTHOR]

Brain tumors are threatening the life and health of people in the world. Automatic brain tumor segmentation using multiple MR images is challenging in medical image analysis. It is known that accurate segmentation relies on effective feature learning. Existing methods address the multi-modal MR brain tumor segmentation by explicitly learning a shared feature representation. However, these methods fail to capture the relationship between MR modalities and the feature correlation between different target tumor regions. In this paper, I propose a multi-modal brain tumor segmentation network via disentangled representation learning and region-aware contrastive learning. Specifically, a feature fusion module is first designed to learn the valuable multi-modal feature representation. Subsequently, a novel disentangled representation learning is proposed to decouple the fused feature representation into multiple factors corresponding to the target tumor regions. Furthermore, contrastive learning is presented to help the network extract tumor region-related feature representations. Finally, the segmentation results are obtained using the segmentation decoders. Quantitative and qualitative experiments conducted on the public datasets, BraTS 2018 and BraTS 2019, justify the importance of the proposed strategies, and the proposed approach can achieve better performance than other state-of-the-art approaches. In addition, the proposed strategies can be extended to other deep neural networks. • A novel multi-modal brain tumor segmentation network is proposed. • A multi-modal fusion module is introduced to fuse multi-modalities. • Disentangled representation learning is proposed to learn tumor-related features. • Region-aware contrastive learning is designed to learn useful feature representations. • Extensive experiments demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

• We analyze the limitations of directly applying the high-order multi-hop neighbors to graph neural networks, including computational inefficiency and limited representation ability of the multi-hop neighbor. • We propose a Markov Clustering Regularized Graph Neural Network (MCMGN) to overcome the limitations. By introducing iterative approximation and Regularized Markov Clustering strategies, the proposed model improves the performance of the multi-hop graph neural network for graph-level representation learning. • Extensive experimental results have verified the effectiveness of the proposed Markov Clustering Regularized Graph Neural Network for graph classification on eight graph benchmark datasets. • In this paper, we develop a Markov Clustering Regularized Multi-hop Graph Neural Network (MCMGN) to overcome the two limitations of higher-order multi-hop graph neural network, i.e., computational inefficiency and limited representation ability of multi-hop neighbor. We analyze the reasons of these two limitations in detail and give the corresponding solutions, i.e., iteration approximation and Markov Clustering regularized approaches, to achieve a excellent performance for graph-level representation learning. To evaluate the effectiveness of the proposed model, we carry out a series of experiments including the comparison with existing models, prune threshold comparison, impact of the number of iteration, impact of the normalization of node feature matrix, and computation cost comparison. Extensive experimental results strongly demonstrate the superiority of MCMGN in predictive performance for graph-level representation learning. [Display omitted] Graph Neural Networks (GNNs) have shown great potential for graph data analysis. In this paper, we focus on multi-hop graph neural networks and aim to extend existing models to a high-order multi-hop form for graph-level representation learning. However, such a directly extending method suffers from two limitations, i.e., computational inefficiency and limited representation ability of the multi-hop neighbor. For the former limitation, we utilize an iteration approach to approximate the power of a complex adjacency matrix to achieve linear computational complexity. For the latter limitation, we introduce the Regularized Markov Clustering (R-MCL) to regularize the flow matrix, i.e., the adjacency matrix, in each iteration step. With these two strategies, we construct Markov Clustering Regularized Multi-hop Graph Neural Network (MCMGN) for graph-level representation learning tasks. Specifically, MCMGN consists of a multi-hop message passing phase and a readout phase, where the multi-hop message passing phase aims to learn multi-hop node embedding, and then the readout phase aggregates multi-hop node representations to generate graph embedding for graph-level representation learning tasks. Extensive experiments on eight graph benchmark datasets strongly demonstrate the effectiveness of Markov Clustering Regularized Multi-hop Graph Neural Network, leading to superior performance on graph classification. [ABSTRACT FROM AUTHOR]

• This paper jointly exploits strong and weak augmentations for unsupervised image clustering. • Two levels of contrastive learning are conducted on three streams of weak/strong augmented views. • A novel end-to-end deep clustering approach termed SACC is proposed. • Extensive experimental results confirm the superiority of SACC over the state-of-the-art. Deep clustering has attracted increasing attention in recent years due to its capability of joint representation learning and clustering via deep neural networks. In its latest developments, the contrastive learning has emerged as an effective technique to substantially enhance the deep clustering performance. However, the existing contrastive learning based deep clustering algorithms mostly focus on some carefully-designed augmentations (often with limited transformations to preserve the structure), referred to as weak augmentations, but cannot go beyond the weak augmentations to explore the more opportunities in stronger augmentations (with more aggressive transformations or even severe distortions). In this paper, we present an end-to-end deep clustering approach termed S trongly A ugmented C ontrastive C lustering (SACC), which extends the conventional two-augmentation-view paradigm to multiple views and jointly leverages strong and weak augmentations for strengthened deep clustering. Particularly, we utilize a backbone network with triply-shared weights, where a strongly augmented view and two weakly augmented views are incorporated. Based on the representations produced by the backbone, the weak-weak view pair and the strong-weak view pairs are simultaneously exploited for the instance-level contrastive learning (via an instance projector) and the cluster-level contrastive learning (via a cluster projector), which, together with the backbone, can be jointly optimized in a purely unsupervised manner. Experimental results on five challenging image datasets have shown the superiority of our SACC approach over the state-of-the-art. The code is available at https://github.com/dengxiaozhi/SACC. [ABSTRACT FROM AUTHOR]

• This paper systematically analyzes the robustness of elimination-based defense under closed-set and open-set attacks. Experimental results show that the residual perturbations can exhibit attacking effects as strong as full perturbations. • This paper reveals that it is essential to improve the robustness by breaking the correlation between perturbations and legitimate information, together with compressing the attachment space of perturbations. • This paper proposes an efficient defense strategy, Frequency-Adaptive Compression and rEconstruction (FACE), to mitigate attacking effects. • Extensive experimental results demonstrate that the proposed method achieves a 27.9% improvement in robust accuracy on ImageNet over elimination-based defense methods, and reduces TASR and FPR under closed-set and open-set attacks. The increasing use of deep neural networks exposes themselves to adversarial attacks in the real world drawn from closed-set and open-set, which poses great threats to their application in safety-critical systems. Since adversarial attacks tend to mislead an original model by adding small perturbations into clean images, an intuitive idea of defensing adversarial attacks is eliminating perturbations as much as possible to mitigate attacking effects. However, such elimination-based strategies unfortunately fail to achieve satisfactory robustness. Aiming to investigate the intrinsic reasons for this phenomenon, systematic experiments are carried out in this paper to indicate that even a 20% residual perturbation can still preserve and exhibit attacking effects as strong as a full one. Our study also indicates that there are strong correlations between perturbations and legitimate images. Thus, breaking the correlation across multiple bands is more effective in mitigating attacking effects. Based on these findings, this paper proposes an efficient defense strategy called "Frequency-Adaptive Compression and rEconstruction (FACE)" to improve the robustness of the model to adversarial attacks. Specifically, low-frequency bands containing semantic information are compressed by a down-sampling operation, while the channel width of high-frequency bands is squeezed and further compressed by adding noise before the Tanh activating function. Meanwhile, attachment spaces of perturbations are also squeezed to the extent as much as possible. Finally, a clean output is obtained by upsampling together with expanded reconstruction. Experiments are extensively conducted on widely used datasets to demonstrate the effectiveness of the proposed method. For closed-set attacks, FACE outperforms the STOA elimination-based methods on ImageNet, achieving a 27.9% improvement. For the MNIST open-set attacks, it not only reduces the success rate of targeted attack by a large margin (from 100% to 24.7%), but also mitigates attacking effects with an FPR-95 value of 0.3. [ABSTRACT FROM AUTHOR]

• We introduce a general framework to incorporate attribute selection into GNNs to enhance the compacity and robustness of GNNs. • We develop two kinds of GNNs, i.e., AsGCN and AsGAT for attributed graph data representation and learning. • We derive a simple and effective algorithm to optimize the proposed AsGCN and AsGAT. Graph Neural Networks (GNNs) have been widely used for graph learning tasks. The main aspect of GNN's layer-wise message passing is conducting attribute/feature propagation on graph. Most existing GNNs generally conduct feature propagation across all feature dimensions. However, in many real applications, attributes usually contain irrelevant and redundant noise. In this case, attribute/feature selection is desired to extract meaningful features and eliminate noisy ones for GNN's layer-wise propagation. Based on this observation, in this paper, we combine ℓ 2 , 1 / ℓ 1 -norm regularized attribute selection and GNNs together and propose a novel Attribute selection guided GNNs (AsGNNs) for graph data representation. AsGNNs aim to adaptively select some desired meaningful features/attributes that best serve GNNs. Moreover, an effective optimization framework has also been derived to train the proposed AsGNNs. The proposed AsGNNs provide a general framework which can incorporate any GNNs to conduct feature selection for layer-wise propagation. In this paper, we implement AsGNNs on both graph convolutional network (GCN) and graph attention network (GAT) and develop AsGCN and AsGAT for graph learning. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed AsGNNs (AsGCN, AsGAT) on semi-supervised learning tasks. [ABSTRACT FROM AUTHOR]

Although deep neural networks (DNNs) have performed well on many perceptual tasks, they are vulnerable to adversarial examples that are generated by adding slight but maliciously crafted perturbations to benign images. Adversarial detection is an important technique for identifying adversarial examples before they are entered into target DNNs. Previous studies that were performed to detect adversarial examples either targeted specific attacks or required expensive computation. Designing a lightweight unsupervised detector is still a challenging problem. In this paper, we propose an A uto E ncoder-based A dversarial E xamples (AEAE) detector that can guard DNN models by detecting adversarial examples with low computation in an unsupervised manner. The AEAE includes only a shallow autoencoder that performs two roles. First, a well-trained autoencoder has learned the manifold of benign examples. This autoencoder can produce a large reconstruction error for adversarial images with large perturbations, so we can detect significantly perturbed adversarial examples based on the reconstruction error. Second, the autoencoder can filter out small noises and change the DNN's prediction on adversarial examples with small perturbations. It helps to detect slightly perturbed adversarial examples based on the prediction distance. To cover these two cases, we utilize the reconstruction error and prediction distance from benign images to construct a two-tuple feature set and train an adversarial detector using the isolation forest algorithm. We show empirically that AEAE is an unsupervised and inexpensive detector against most state-of-the-art attacks. Through the detection in these two cases, there is nowhere to hide adversarial examples. • We observe that adversarial detection is sensitive to the perturbation level. • We train a shallow autoencoder to find two key features from adversarial examples. • We propose a lightweight and unsupervised adversarial detector. [ABSTRACT FROM AUTHOR]

• A deep capsule network model named DCapsNet is proposed to automatically extract the activity or gait features from sensors and classify them. • The proposed model combines a set of convolutional layers and capsule network. • The set of convolutional layers are suitable for processing temporal sequences and providing scalar output features. • These features are then passed through capsule network which is trained by a dynamic routing algorithm to provide probability vectors of the human activity/gait. • The proposed model achieves better performance than the state-of-the-art models for recognizing human activity and gait. Recently, deep neural networks are used to recognize human activity/gait through mobile sensors which have attracted a great attention. Although the existing deep neural networks that perform automatic feature extraction have achieved desirable performance, their classification accuracy needs to be improved. In this paper, a deep neural network that combines a set of convolutional layers and capsule network is proposed. The proposed architecture named DCapsNet is suited to automatically extract the activity or gait features through built in sensors and classify them. The convolutional layers of the DCapsNet are more suitable for processing temporal sequences and provide scalar outputs but not the equivariance. The capsule network (CapsNet) is then trained by a dynamic routing algorithm to capture the equivariance having a magnitude and orientation, which increases the efficiency of the model classification. The performance of the proposed model is evaluated on four public datasets: two HAR datasets (UCI-HAR and WISDM) and two gait datasets (WhuGAIT). The recognition accuracy of the proposed model for the UCI-HAR and WISDM datasets are 97.92 % and 99.30 %, respectively, and for the WhuGAIT Dataset #1 and Dataset #2 are 94.75 % and 97.16 %, respectively. Experimental results show that the proposed model achieves the highest recognition accuracy over the reported results of the state-of-the-art models. [ABSTRACT FROM AUTHOR]

Deep Neural Networks have received considerable attention in recent years. As the complexity of network architecture increases in relation to the task complexity, it becomes harder to manually craft an optimal neural network architecture and train it to convergence. As such, Neural Architecture Search (NAS) is becoming far more prevalent within computer vision research, especially when the construction of efficient, smaller network architectures is becoming an increasingly important area of research, for which NAS is well suited. However, despite their promise, contemporary and end-to-end NAS pipeline require vast computational training resources. In this paper, we present a comprehensive overview of contemporary NAS approaches with respect to image classification, object detection, and image segmentation. We adopt consistent terminology to overcome contradictions common within existing NAS literature. Furthermore, we identify and compare current performance limitations in addition to highlighting directions for future NAS research. [ABSTRACT FROM AUTHOR]

Offline-trained Siamese networks have realized very promising tracking precision and efficiency. However, the performance is still limited by the drawbacks in online update. Traditional strategies cannot tackle the irregular variations of object and the sampling noise, so it is quite risky to adopt them to update Siamese trackers. In this paper, we present a two-stage one-shot learner by exploring the learning scheme of Siamese network, which reveals there are two key issues during online update, i.e., feature fusion and feature comparison. Based on this finding, we propose an updatable Siamese tracker by introducing two independent transformers (SiamTOL). Concretely, a Cross-aware transformer is designed to combine the features of the initial and the dynamic templates, while a Decoder-favored transformer is exploited to compare the fusing template and the search region. By combining these transformers, our tracker is able to adequately model the feature dependencies between multi-frame object samples. Extensive experimental results on several popular benchmarks well manifest that the proposed approach achieves the leading performance, and outperforms other state-of-the-art trackers. • A two-stage one-shot learner is proposed to learn multi-time historic object features. • An updatable Siamese tracker is presented based on the two-stage one-shot learner. • Massive experimental results manifest the superiority of the proposed tracker. [ABSTRACT FROM AUTHOR]

• The improved word embedding model helps to facilitate adequate-ly representation learning in pre-training. • The improved dropout method can reduce the number of iterations needed for training, accelerate the training of deep neural networks. • We expanded the original non-functional requirements dataset (PROMISE dataset) and designed a new dataset called SOFTWARE NFR. • This model can be applied in the automation of software engineering. Non-functional requirements are property that software products must have in order to meet the user's business requirements, and are additional constraints on the quality and characteristics of software systems. They are generally written by software designers and documented in various parts of requirements documentation. When developing systems, developers need to classify non-functional requirements from requirements documents, and classifying these non-functional requirements requires professional skills, experience, and domain knowledge, which is challenging and time-consuming for developers. It would be beneficial to implement automatic classification of non-functional requirements from requirements documents, which could reduce the manual, time, and mental fatigue involved in identifying specific non-functional requirements from a large number of requirements. In this paper, a deep neural network model called NFRNet is designed to automatically classify non-functional requirements from software requirement documents. The network consists of two parts. One is an improved BERT word embedding model based on N-gram masking for learning context representation of the requirement descriptions, and the other is a Bi-LSTM classification network for capture context information of the requirement descriptions. We use a Softmax classifier in the end to classify the requirement descriptions. At the same time, in order to accelerate the training and improve the generalization ability of the model, the network uses multi-sample dropout regularization technology. This new regularization technology can reduce the number of iterations needed for training, accelerate the training of deep neural networks, and the networks trained achieved lower error rates. In addition, we expanded the original non-functional requirements dataset (PROMISE dataset) and designed a new dataset called SOFTWARE NFR. The new dataset far exceeds the original dataset in terms of the number of requirement description sentences and the number of non-functional requirements categories. It can be taken as a new testbed for non-functional requirements classification. Through cross-validation on the new dataset, the experimental results show that the network designed in this paper is significantly better than the other 17 classification methods in terms of Precision, Recall, and F1-score. At the same time, for the training set and the validation set, using the multi-sample dropout regularization technology can accelerate the training speed, reduce the number of iterations, and achieve lower error rates and loss. [ABSTRACT FROM AUTHOR]

Graph convolution is a fundamental building block for many deep neural networks on graph-structured data. In this paper, we introduce a simple, yet very effective graph convolutional network with skip connections for semi-supervised anomaly detection. The proposed layerwise propagation rule of our model is theoretically motivated by the concept of implicit fairing in geometry processing, and comprises a graph convolution module for aggregating information from immediate node neighbors and a skip connection module for combining layer-wise neighborhood representations. This propagation rule is derived from the iterative solution of the implicit fairing equation via the Jacobi method. In addition to capturing information from distant graph nodes through skip connections between the network's layers, our approach exploits both the graph structure and node features for learning discriminative node representations. These skip connections are integrated by design in our proposed network architecture. The effectiveness of our model is demonstrated through extensive experiments on five benchmark datasets, achieving better or comparable anomaly detection results against strong baseline methods. We also demonstrate through an ablation study that skip connection helps improve the model performance. • We propose a multi-layer graph neural network for semi-supervised anomaly detection. • We introduce a learnable skip-connection module to improve the node representations. • We analyze the complexity of our model and train it using a regularized loss function. • We demonstrate through experiments that our model achieves superior performance. [ABSTRACT FROM AUTHOR]

Spiking neural networks are efficient computation models for low-power environments. Spike-based BP algorithms and ANN-to-SNN (ANN2SNN) conversions are successful techniques for SNN training. Nevertheless, the spike-base BP training is slow and requires large memory costs, while ANN2SNN needs many inference steps to obtain good performance. In this paper, we propose an A ctivation C onsistency C oupled A NN- S NN (A C 2 AS) framework to train the SNN in a fast and memory-efficient way. The A C 2 AS consists of two components: (a) a weight-shared architecture between ANN and SNN and (b) spiking mapping units. Firstly, the architecture trains the weight-shared parameters on the ANN branch, resulting in fast training and low memory costs for SNN. Secondly, the spiking mapping units are designed to ensure that the activation values of the ANN are the spiking features. As a result, the activation consistency is guaranteed, and the classification error of the SNN can be optimized by training the ANN branch. Besides, we design an adaptive threshold adjustment (ATA) algorithm to decrease the firing of noisy spikes. Experiment results show that our A C 2 AS-based models perform well on the benchmark datasets (CIFAR10, CIFAR100, and Tiny-ImageNet). Moreover, the A C 2 AS achieves comparable accuracy under 0. 625 × time steps, 0. 377 × training time, 0. 27 × GPU memory costs, and 0. 33 × spike activities of the Spike-based BP model. The code is available at https://github.com/TJXTT/AC2ASNN. • Proposed the A C 2 AS framework for fast and memory efficient SNN training. • Designed an adaptive threshold adjustment algorithm and two spiking mapping units to ensure that the SNN can be equivalently represented by the ANN under the A C 2 AS framework. • The SNNs trained by A C 2 AS achieve considerable performance, sparse spike activity, and fast inference. [ABSTRACT FROM AUTHOR]

Modern artificial intelligence systems rely heavily on deep learning techniques. However, deep neural networks are easily disturbed by adversarial objects. Adversarial examples are beneficial to improve the robustness of the 3D neural network model and enhance the stability of the artificial intelligence system. At present, most 3D adversarial attack methods perturb the entire point cloud to generate adversarial examples, which results in high perturbation costs and low operability in the physical world. In this paper, we propose an adaptive local adversarial attack method (AL-Adv) on 3D point clouds to generate adversarial point clouds. First, we analyze the vulnerability of the 3D network model and extract the salient regions of the input point cloud, namely the vulnerable regions. Second, we propose an adaptive gradient attack algorithm that targets salient regions. The proposed attack algorithm adaptively assigns different disturbances in different directions of the three-dimensional coordinates of the point cloud. Experimental results show that our proposed AL-Adv method achieves a higher attack success rate than the global attack method. Specifically, the adversarial examples generated by AL-Adv demonstrate good imperceptibility and small generation costs. • The proposed AL-Adv method focuses on local regions of point clouds. • We design a novel adaptive gradient attack algorithm for local regions. • The proposed method AL-Adv achieves a higher attack success rate. • The proposed AL-Adv method generates adversarial examples with good imperceptibility. • The proposed AL-Adv method outperforms the global attack method. [ABSTRACT FROM AUTHOR]

• Improving the results of Siamese networks using CP-CVV. • Improving One-Shot-Learning results by class partitioning. • Improving the results of CIFAR-FS with CP-CVV. • Tuning CP-CVV hyper-parameters for optimal results. • CP-CVV as an alternative to traditional ensemble models. One Shot Learning includes all those techniques that make it possible to classify images using a single image per category. One of its possible applications is the identification of food products. For a grocery store, it is interesting to record a single image of each product and be able to recognise it again from other images, such as photos taken by customers. Within deep learning, Siamese neural networks are able to verify whether two images belong to the same category or not. In this paper, a new Siamese network training technique, called CP-CVV, is presented. It uses the combination of different models trained with different classes. The separation of validation classes has been done in such a way that each of the combined models is different in order to avoid overfitting with respect to the validation. Unlike normal training, the test images belong to classes that have not previously been used in training, allowing the model to work on new categories, of which only one image exists. Different backbones have been evaluated in the Siamese composition, but also the integration of multiple models with different backbones. The results show that the model improves on previous works and allows the classification problem to be solved, an additional step towards the use of Siamese networks. To the best of our knowledge, there is no existing work that has proposed integrating Siamese neural networks using a class-based validation set separation technique so as to be better at generalising for unknown classes. Additionally, we have applied Cross-Validation-Voting with ConvNeXt to improve the existing classification results of a well-known Grocery Store Dataset. [ABSTRACT FROM AUTHOR]

• We present a comprehensive review and new perspectives for data-driven single image deraining (SID); • We re-examine the three factors (i.e., data, rain model and network architecture) for SID problem; • We analyze the factors by proposing new and more reasonable criteria (i.e., general vs. specific, synthetical vs. mathematical, black-box vs. white-box); • We also study the relationship of the three factors from new perspectives of data, and reveal two different solving paradigms (explicit vs. implicit) for the SID task; • We profoundly studied one of the three factors, i.e., data, and measured the performance of current methods on different datasets by extensive experiments to reveal the effectiveness of SID data. S ingle I mage D eraining (SID) aims at recovering the rain-free background from an image degraded by rain streaks. For the powerful fitting ability of deep neural networks and massive training data, data-driven deep SID methods have obtained significant improvement over traditional model/prior-based ones. Current studies usually focus on improving the deraining performance by proposing different categories of deraining networks, while neglecting the interpretation of the solving process. As a result, the generalization ability may still be limited in real-world scenarios, and the deraining results also cannot effectively improve the performance of subsequent high-level tasks (e.g., object detection). To explore these issues, we in this paper re-examine the three important factors (i.e., data, rain model and network architecture) for the SID problem, and specifically analyze them by proposing new and more reasonable criteria (i.e., general vs. specific, synthetical vs. mathematical, black-box vs. white-box). We also study the relationship of the three factors from a new perspective of data, and reveal two different solving paradigms (explicit vs. implicit) for the SID task. We further discuss the current mainstream data-driven SID methods from five aspects, i.e., training strategy, network pipeline, domain knowledge, data preprocessing, and objective function, and some useful conclusions are summarized by statistics. Besides, we profoundly studied one of the three factors, i.e., data, and measured the performance of current methods on different datasets through extensive experiments to reveal the effectiveness of SID data. Finally, with the comprehensive review and in-depth analysis, we draw some valuable conclusions and suggestions for future research. [ABSTRACT FROM AUTHOR]

• A theoretical framework for domain generalization (DG) from the perspective of information bottleneck (IB) principle is established. • Based on the framework, a feasible solution by class-wise instance discrimination (CID) combined with maximizing feature entropy (MFE) is proposed to learn the desired representation for DG. • The proposed method achieves excellent performance on multiple datasets without knowing domain labels. • The proposed regularization rule (MFE) improves other invariance-based DG methods consistently. Although deep neural networks have achieved superior performance on many classical tasks, they deteriorate in real applications due to the unpredictable distribution shift. Domain generalization (DG) focuses on improving the generalization ability of the predictive model in unseen domains by training on multiple available source domains. All these domains share the same categories but commonly obey different distributions. In this paper, we establish a new theoretical framework for domain generalization from the perspective of the information bottleneck (IB) principle, which links representation learning in DG with domain-invariant representation learning and maximizing feature entropy (MFE). Based on the theoretical framework, we provide a feasible solution by class-wise instance discrimination combined with inter-dimension decorrelation and intra-dimension uniformity to learn the desired representation for domain generalization, which achieves excellent performance on multiple datasets without knowing domain labels. Extensive experiments show that the proposed regularization rule (MFE) can improve invariance-based DG methods consistently. Moreover, as an extreme case of domain generalization, we also show that MFE is promising to improve adversarial robustness. [ABSTRACT FROM AUTHOR]

• The IVP framework computes probability intervals by categorizing labeled data. • The use of embedding representations allow application of IVP on high-dimensional data. • Collection and use of unlabeled data improves accuracy, calibration and efficiency. • Pseudo-labeling of new unlabeled data has minimal performance overhead in runtime. Learning enabled components are frequently used by autonomous systems and it is common for deep neural networks to be integrated in such systems for their ability to learn complex, non-linear data patterns and make accurate predictions in dynamic environments. However, their large number of parameters and their use as black boxes introduce risks as the confidence in each prediction is unknown and output values like softmax scores are not usually well-calibrated. Different frameworks have been proposed to compute accurate confidence measures along with the predictions but at the same time introduce a number of limitations like execution time overhead or inability to be used with high-dimensional data. In this paper, we use the Inductive Venn Predictors framework for computing probability intervals regarding the correctness of each prediction in real-time. We propose taxonomies based on distance metric learning to compute informative probability intervals in applications involving high-dimensional inputs. By assigning pseudo-labels to unlabeled input data during system deployment we further improve the efficiency of the computed probability intervals. Empirical evaluation on image classification and botnet attacks detection in Internet-of-Things (IoT) applications demonstrates improved accuracy and calibration. The proposed method is computationally efficient, and therefore, can be used in real-time. The code is available at https://github.com/dboursinos/Efficient-Probability-Intervals-Classification-Inductive-Venn-Predictors. [ABSTRACT FROM AUTHOR]

• We propose a new SN P system. Three new types of rules with related evolution of spikes on the hybrid structure of neural-like P systems are designed. • We designed two dynamic regulatory factors on new rules to help guide the optimization of the HN P systems automatically. • A segmentation model for BMs is developed based on the HN P system that integrates the high accuracy of functions in deep learning models in semantic segmentation into P systems with good parallelism and robustness. Neural-like P systems are membrane computing models inspired by natural computing. Spiking neurral (SN) P systems, a kind of neural-like P systems, are viewed as third-generation neural network models. Although real neurons have complex structures, classical SN P systems simplify the structures and corresponding mechanisms to stationary two-dimensional graphs and lack related evolution mechanisms on spikes and channels, which limits the real applications of these models. In this paper, we propose a new hybrid SN P system with evolutionary channels (HN P systems), including three new types of rules for dynamically generating or removing one-one and one-many/many-one channels with related evolutions of spikes on the hybrid neuron structures. Two dynamic regulatory factors are also presented on rules to help guide the optimization of the HN P systems automatically. Based on the new P system, a multiple brain metastases (BMs) segmentation model is developed. The experimental results indicate that the proposed models outperform the state-of-the-art methods on the BMs, which have large variations in sizes, positions and shapes, and low contrast with their surroundings. Performances on the head and neck segmentation dataset also verifies the effectiveness of the HN P system. [ABSTRACT FROM AUTHOR]

• We propose a joint training framework where the knowledge of ANN is consistently transferred to the SNN during training. • We factorize the parameters of both ANN and SNN and restrict them to have the same singular vectors while learning different singular values. • Extensive experiments are conducted on vision benchmarks, demonstrating that our method reaches SOTA. Emerged as a biology-inspired method, Spiking Neural Networks (SNNs) mimic the spiking nature of brain neurons and have received lots of research attention. SNNs deal with binary spikes as their activation and therefore derive extreme energy efficiency on hardware. However, it also leads to an intrinsic obstacle that training SNNs from scratch requires a re-definition of the firing function for computing gradient. Artificial Neural Networks (ANNs), however, are fully differentiable to be trained with gradient descent. In this paper, we propose a joint training framework of ANN and SNN, in which the ANN can guide the SNN's optimization. This joint framework contains two parts: First, the knowledge inside ANN is distilled to SNN by using multiple branches from the networks. Second, we restrict the parameters of ANN and SNN, where they share partial parameters and learn different singular weights. Extensive experiments over several widely used network structures show that our method consistently outperforms many other state-of-the-art training methods. For example, on the CIFAR100 classification task, the spiking ResNet-18 model trained by our method can reach to 77.39 % top-1 accuracy with only 4 time steps. [ABSTRACT FROM AUTHOR]

• Long-tailed distribution leads to decision boundary bias in deep neural networks. • The classification performance declines under the bias caused by data imbalance. • A novel margin-aware augmentation method rectifies the biased boundary. • Large improvements on long-tailed classification benchmark with a better calibrated classifier. The long-tailed data distribution is prevalent in real world and it poses great challenge on deep neural network training. In this paper, we propose Margin-aware Rectified Augmentation (MRA) to tackle this problem. Specifically, the MRA consists of two parts. From the data perspective, we analyze that data imbalance will cause the decision boundary be biased, and we propose a novel Margin-aware Rectified mixup (MR-mixup) that adaptively rectifies the biased decision boundary. Furthermore, from the model perspective, we analyze that the imbalance will also lead to consistent 'gradient suppression' on minority class logits. Then we propose Reweighted Mutual Learning (RML) that provides extra 'soft target' as supervision signal and augments the 'encouraging gradients' on the minority classes. We conduct extensive experiments on benchmark datasets CIFAR-LT, ImageNet-LT and iNaturalist18. The results demonstrate that the proposed MRA not only achieves state-of-the-art performance, but also yields a better-calibrated prediction. [ABSTRACT FROM AUTHOR]

• A Perturbation Generative Network (PGN) is proposed to generate Adversarial Examples (AEs) under black-box scenarios. The discriminator is employed to enhance the fidelity. The generated adversarial noises are further compressed by a designed compression strategy. • A Black-box Reversible Adversarial Example (B-RAE) scheme is proposed to protect shared images, which not only generates adversarial examples efficiently, but also balances the visual quality and attack ability of adversarial examples more flexibly. • The PGN can generate adversarial examples with high robustness and transfer attack ability. The ensemble strategy is applied to strengthen the attack ability to different models. The robustness and the black-box attack ability of B-RAE provide a promising solution for practical applications. Shared images on the Internet are easily collected, classified, and analyzed by unauthorized commercial companies through Deep Neural Networks (DNNs). The illegal use of these data damages the rights and interests of authorized companies and individuals. How to ensure that network-shared data is legally used by authorized users and not used by unauthorized DNNs has become an urgent problem. Reversible Adversarial Example (RAE) provides an effective solution, which can mislead the classification of unauthorized DNNs and does not affect the authorized users. The existing RAE schemes assumed that we could know the parameters of the target model and thus generate reversible adversarial examples. However, model parameters are often protected to avoid leakage, increasing the difficulty of generating accurate RAEs. In this paper, we first propose a Black-box Reversible Adversarial Example (B-RAE) scheme to generate robust reversible adversarial examples. We aim to protect image privacy while maintaining data usability in real scenarios. Experimental results and analysis have demonstrated that the proposed B-RAE is more effective and robust compared with the existing schemes. [ABSTRACT FROM AUTHOR]

Batch normalization (BN) is widely used in modern deep neural networks, which has been shown to represent the domain-related knowledge, and thus is ineffective for cross-domain tasks like unsupervised domain adaptation (UDA). Existing BN variant methods aggregate source and target domain knowledge in the same channel in normalization module. However, the misalignment between the features of corresponding channels across domains often leads to a sub-optimal transferability. In this paper, we exploit the cross-domain relation and propose a novel normalization method, Reciprocal Normalization (RN). Specifically, RN first presents a Reciprocal Compensation (RC) module to acquire the compensatory for each channel in both domains based on the cross-domain channel-wise correlation. Then RN develops a Reciprocal Aggregation (RA) module to adaptively aggregate the feature with its cross-domain compensatory components. As an alternative to BN, RN is more suitable for UDA problems and can be easily integrated into popular domain adaptation methods. Experiments show that the proposed RN outperforms existing normalization counterparts by a large margin and helps state-of-the-art adaptation approaches achieve better results. The source code is available on https://github.com/Openning07/reciprocal-normalization-for-DA. [ABSTRACT FROM AUTHOR]

• We propose a unified two-stage framework for automatic model compression. • We propose Dynamic BN to predict the performance of each compression policy. • We propose a solving algorithm to search for the compression ratio allocation. Deep Neural Networks have become ubiquitous in various domains. Meanwhile, the problems of massive storage and computation costs have hindered the deployment of these models to real-world applications. This paper proposes a novel and unified two-stage framework for automatic model compression. To determine the compression ratio of each layer, we improve the optimization from two aspects. First, to predict the performance of each compression policy, we propose Dynamic BN, which improves the correlation significantly with little computation overhead. Second, to search for the compression ratio allocation, we propose an efficient and hyperparameter-free solving algorithm based on the proposed Hessian matrix approximation and Knapsack problem reformulation. Moreover, comprehensive experiments and analyses are conducted on the CIFAR-100&ImageNet datasets and various network architectures to demonstrate its performance advantages over existing model compression methods under the quantization-only, pruning-only, and pruning-quantization settings. [ABSTRACT FROM AUTHOR]

• A directed convolutional neural network model is proposed • Partial dilation-Global erosion generates resistance against information loss • Several recent approaches for the segmentation of graphemes/characters are discussed • The over-traces issue is addressed for the first time in the literature • MMOs can provide real solutions if a convenient strategy is adopted Due to the nature of Arabic handwriting, segmenting words into characters/graphemes is the most difficult and critical task of the recognition system. The present paper proposes an approach to segment handwritten Arabic words into graphemes based on a directed Convolutional Neural Network (CNN) and Mathematical Morphology Operations (MMO). Arabic script is cursive, which means that almost all graphemes are connected via horizontal links; therefore, a technique to remove links will facilitate the segmentation of graphemes. In general, an MMO such as erosion seems suitable for getting the job done, but since Arabic handwriting is difficult, MMOs cause information loss and suffer from many issues such as diacritics and over-traces, which lead to over/under/bad segmentations. To overcome limitations, the present paper addresses these issues in the following order: the over-traces issue is addressed for the first time in the literature; a robust algorithm for diacritics extraction is provided; and finally, the main segmentation algorithm adopts a strategy based on a Partial Dilation (PD)-Global Erosion (GE) technique to combat the information loss issue. The PD phase amplifies important regions, while GE eliminates links between graphemes. The complementarity between PD and GE facilitates the extraction of graphemes and creates resistance against information loss. To properly tackle these difficult problems, this article exploits the robustness of CNNs, so a new directed CNN model is suggested. The idea is to draw the model's attention to certain targeted features, which are selected according to the nature of the problem addressed. The proposed directed CNN is used in all phases of the segmentation process. The experimental results are very encouraging and show that the proposed directed CNN model outperformed basic CNN in many experiments. The results also reveal that the followed strategy improved the ability of MMOs to perform segmentation and to compete with other approaches in this research area. [ABSTRACT FROM AUTHOR]

• We re-investigate the pipeline of fine-grained visual categorization (FGVC) techniques from the view of human visual recognition system, and propose a novel Attention-Shift based Deep Neural Network (AS-DNN) for automatic parts locating and semantic correlation learning. • We propose an end-to-end trainable sub-network structure C sft to simulate the attention-shift process. C sft locates the discriminative regions automatically and encodes and decodes the semantic relations among diverse discriminative parts iteratively. • Comprehensive experiments show that AS-DNN achieves state-of-the-art performances in three widely used challenging datasets. Moreover, the visualization of located discriminative parts proves the robustness of AS-DNN in complex backgrounds and postures. Fine-grained visual categorization (FGVC) has attracted extensive attention in recent years. The general pipeline of current FGVC techniques is to 1) locate the discriminative regions; 2) extract features from each region independently; and 3) feed the integrated features to a classifier. In this paper, we re-investigate the pipeline from the view of human visual recognition mechanisms. The perceiving of discriminative regions is a temporal processing by the human visual system (HVS) via the attention-shift mechanism. However, the existing independent feature extracting and one-pass feeding strategy ignore the inherent semantic relationships among discriminative regions, and thus is improper to model the attention-shift process properly. Therefore, in this paper, we propose a novel end-to-end FGVC network structure named Attention-Shift based Deep Neural Network (AS-DNN) to locate the discriminative regions automatically and encode the semantic correlations iteratively. AS-DNN consists of two channels: 1) the global perception channel C glb and 2) the attention-shift channel C sft , simulating the global perception and the attention-shift mechanism, respectively. Experimental results show that AS-DNN achieves state-of-the-art performances by outperforming both the CNN-based weakly or strongly-supervised FGVC algorithms on several widely-used fine-grained datasets, and the visualization of attention regions exhibit that the proposed method can locate the discriminative regions robustly in complex backgrounds and postures. [ABSTRACT FROM AUTHOR]

• In this paper, we propose a variation of deep residual convolutional neural networks with robustness and efficiency in both learning and testing. • More importantly, we propose multiple strategies for model adaptation to the internal contents of the lowresolution input image and analyze their strong points and weaknesses. • Our adaptation especially favors images with repetitive structures or high resolutions. • We hope to arouse the interests of communities in focusing on internal priors, which are limited but have been proved effective and highly relevant. In recent years, impressive advances have been made in single-image super-resolution. Deep learning is behind much of this success. Deep(er) architecture design and external prior modeling are the key ingredients. The internal contents of the low-resolution input image are neglected with deep modeling, despite earlier works that show the power of using such internal priors. In this paper, we propose a variation of deep residual convolutional neural networks, which has been carefully designed for robustness and efficiency in both learning and testing. Moreover, we propose multiple strategies for model adaptation to the internal contents of the low-resolution input image and analyze their strong points and weaknesses. By trading runtime and using internal priors, we achieve improvements from 0.1 to 0.3 dB PSNR over the reported results on standard datasets. Our adaptation especially favors images with repetitive structures or high resolutions. It indicates a more practical usage when our adaption approach applies to sequences or videos in which adjacent frames are strongly correlated in their contents. Moreover, the approach can be combined with other simple techniques, such as back-projection and enhanced prediction, to realize further improvements. [ABSTRACT FROM AUTHOR]

• We reveal that the difficulty of clean-label backdoor attacks is mostly due to the antagonistic effects of 'robust features' and verify that DNNs have different learning abilities for different samples. • We revisit the paradigm of existing clean-label backdoor attacks and propose a new complementary paradigm by considering the different learning difficulties of samples. • We empirically verify the effectiveness and the poisoning transferability of our method on benchmark datasets and discuss its intrinsic mechanism. Recent studies demonstrated that deep neural networks (DNNs) are vulnerable to backdoor attacks. The attacked model behaves normally on benign samples, while its predictions are misled whenever adversary-specified trigger patterns appear. Currently, clean-label backdoor attacks are usually regarded as the most stealthy methods in which adversaries can only poison samples from the target class without modifying their labels. However, these attacks can hardly succeed. In this paper, we reveal that the difficulty of clean-label attacks mainly lies in the antagonistic effects of 'robust features' related to the target class contained in poisoned samples. Specifically, robust features tend to be easily learned by victim models and thus undermine the learning of trigger patterns. Based on these understandings, we propose a simple yet effective plug-in method to enhance clean-label backdoor attacks by poisoning 'hard' instead of random samples. We adopt three classical difficulty metrics as examples to implement our method. We demonstrate that our method can consistently improve vanilla attacks, based on extensive experiments on benchmark datasets. [ABSTRACT FROM AUTHOR]

• The skewness and asymmetry in the relationship between channelwise responses and their importance. • An interpretable modeling on channelwise attention. • Consistent performance increase on various tasks and various backbones. • Visualization shows learned attention. Channelwise attention mechanisms have recently been demonstrated to boost the performance of deep convolutional neural networks (CNNs). The hypothesis on the negative correlation between channelwise responses and their importance levels has been verified. Therefore, according to the shifted means and unbalanced responses that are observed in attention distribution, two empirical hypotheses are proposed in this paper. Then, as an interpretable attention module, bilateral asymmetric skewed Gaussian attention (bi-SGA) is utilized to combine skewed and asymmetric properties into the Gaussian context transformer (GCT), which is the state-of-the-art. Finally, extensive experiments on the different benchmarks validate the rationality of the hypotheses. The proposed bi-SGA improves the overall performance of GCT attention mechanisms with only two extra parameters to be learned. Moreover, our attention mechanism also provides a perspective on analyzing the channelwise importance levels in deep neural networks in an interpretable and logical manner. [ABSTRACT FROM AUTHOR]

• CNN based detectors are vulnerable to adversarial pan-sharpening images. • Pan-sharpening technology is used to generate adversarial remote sensing images. • We propose a pan-sharpening network for attacking object detector. Pan-sharpening, as one of the most commonly used techniques in remote sensing systems, aims to fuse texture-rich PAN images and multi-spectral MS images to obtain texture-rich MS images. With the development of deep learning, CNN based Pan-sharpening methods have received more and more attention in recent years. Since Pan-sharpening technique can integrate the complementary information of Pan and MS images, researchers usually apply object detectors on these pan-sharpened images to achieve reliable detection results. However, recent studies have shown that Deep Learning-based object detection methods are vulnerable to adversarial examples, i.e., adding imperceptible noise to clean images can fool well-trained deep neural networks. It is interesting to combine the pan-sharpening technique with adversarial examples to attack object detectors in remote sensing. In this paper, we propose a framework to generate adversarial pan-sharpened images. Specifically, we propose a two-stream network to generate the pan-sharpened images, and then utilize the shape loss and label loss to perform the attack task. To guarantee the quality of pan-sharpened images, a perceptual loss is utilized to balance spectral preservation and attacking performance. Experimental results demonstrate that the proposed method can generate effective adversarial pan-sharpened images that maintain a high success rate for white-box attacks and achieve transferability for black-box attacks. [ABSTRACT FROM AUTHOR]

• A novel deep CNN architecture is proposed for jointly learning the expert consensus label and the annotator's label. The proposed architecture (Fig. 1) consists of two coupled CNNs where one estimates the expert consensus label probabilities and the other models the characteristics of individual annotators (e.g., tendency to over-segmentation, mix-up between different classes, etc) by estimating the pixel-wise confusion matrices (CMs) on a per image basis. Unlike STAPLE [25] and its variants, our method models, and disentangles with deep neural networks, the complex mappings from the input images to the annotator behaviours and to the expert consensus label. • The parameters of our CNNs are "global variables" that are optimised across different image samples; this enables the model to disentangle robustly the annotators' mistakes and the expert consensus label based on correlations between similar image samples, even when the number of available annotations is small per image (e.g., a single annotation per image). In contrast, this would not be possible with STAPLE [25] and its variants [5,8] where the annotators' parameters are estimated on every target image separately. • This paper extends the preliminary version of our method presented at the NeurIPS Thirty-fourth Annual Conference on Neural Information Processing Systems [30] , by extensively evaluating our model on a new created real-world multiple sclerosis lesion dataset (QSMSC at UCL: Queen Square Multiple Sclerosis Center at UCL, UK). This dataset is generated with manual segmentations from 4 different annotators (3 radiologists with different level skills and 1 expert to generate the expert consensus label). Additionally, we presented a comprehensive discussion about our model's potential applications (e.g., estimate annotator's quality and annotation's quality), the future works we are going to explore, and the potential limitations of our model. [Display omitted] Supervised machine learning methods have been widely developed for segmentation tasks in recent years. However, the quality of labels has high impact on the predictive performance of these algorithms. This issue is particularly acute in the medical image domain, where both the cost of annotation and the inter-observer variability are high. Different human experts contribute estimates of the "actual" segmentation labels in a typical label acquisition process, influenced by their personal biases and competency levels. The performance of automatic segmentation algorithms is limited when these noisy labels are used as the expert consensus label. In this work, we use two coupled CNNs to jointly learn, from purely noisy observations alone, the reliability of individual annotators and the expert consensus label distributions. The separation of the two is achieved by maximally describing the annotator's "unreliable behavior" (we call it "maximally unreliable") while achieving high fidelity with the noisy training data. We first create a toy segmentation dataset using MNIST and investigate the properties of the proposed algorithm. We then use three public medical imaging segmentation datasets to demonstrate our method's efficacy, including both simulated (where necessary) and real-world annotations: 1) ISBI2015 (multiple-sclerosis lesions); 2) BraTS (brain tumors); 3) LIDC-IDRI (lung abnormalities). Finally, we create a real-world multiple sclerosis lesion dataset (QSMSC at UCL: Queen Square Multiple Sclerosis Center at UCL, UK) with manual segmentations from 4 different annotators (3 radiologists with different level skills and 1 expert to generate the expert consensus label). In all datasets, our method consistently outperforms competing methods and relevant baselines, especially when the number of annotations is small and the amount of disagreement is large. The studies also reveal that the system is capable of capturing the complicated spatial characteristics of annotators' mistakes. [ABSTRACT FROM AUTHOR]

• A detailed review of latest deep metric learning methods for few-shot image classification. • More than 60 methods published from 2018 to 2022 are surveyed. • Methods are divided according to three stages of deep metric learning. • Challenges and future directions of few-shot image classification are discussed. Few-shot image classification is a challenging problem that aims to achieve the human level of recognition based only on a small number of training images. One main solution to few-shot image classification is deep metric learning. These methods, by classifying unseen samples according to their distances to few seen samples in an embedding space learned by powerful deep neural networks, can avoid overfitting to few training images in few-shot image classification and have achieved the state-of-the-art performance. In this paper, we provide an up-to-date review of deep metric learning methods for few-shot image classification from 2018 to 2022 and categorize them into three groups according to three stages of metric learning, namely learning feature embeddings, learning class representations, and learning distance measures. Under this taxonomy, we identify the trends of transitioning from learning task-agnostic features to task-specific features, from simple computation of prototypes to computing task-dependent prototypes or learning prototypes, from using analytical distance or similarity measures to learning similarities through convolutional or graph neural networks. Finally, we discuss the current challenges and future directions of few-shot deep metric learning from the perspectives of effectiveness, optimization and applicability, and summarize their applications to real-world computer vision tasks. [ABSTRACT FROM AUTHOR]

• Establish a comprehensive framework for model robustness, containing 23 different metrics. • Provide an open-sourced platform to support easy-to-use robustness evaluation and continuous integration. • Conduct large-scale experiments and provide preliminary suggestions to the evaluation of model robustness. Deep neural networks (DNNs) have achieved remarkable performance across a wide range of applications, while they are vulnerable to adversarial examples, which motivates the evaluation and benchmark of model robustness. However, current evaluations usually use simple metrics to study the performance of defenses, which are far from understanding the limitation and weaknesses of these defense methods. Thus, most proposed defenses are quickly shown to be attacked successfully, which results in the "arm race" phenomenon between attack and defense. To mitigate this problem, we establish a model robustness evaluation framework containing 23 comprehensive and rigorous metrics, which consider two key perspectives of adversarial learning (i.e., data and model). Through neuron coverage and data imperceptibility, we use data-oriented metrics to measure the integrity of test examples; by delving into model structure and behavior, we exploit model-oriented metrics to further evaluate robustness in the adversarial setting. To fully demonstrate the effectiveness of our framework, we conduct large-scale experiments on multiple datasets including CIFAR-10, SVHN, and ImageNet using different models and defenses with our open-source platform. Overall, our paper provides a comprehensive evaluation framework, where researchers could conduct comprehensive and fast evaluations using the open-source toolkit, and the analytical results could inspire deeper understanding and further improvement to the model robustness. [ABSTRACT FROM AUTHOR]

• We suggest a Lipschitz regularization based learning method with noisy labels, which prevents the model from over-fitting to noisy labels especially with severe noise. • We present a novel method for adaptive modeling and detection of label noise to dynamically identify the clean samples. • We combine the strongly-augmented strategy with semi-supervised learning and improve the robustness and stability of the model effectively. Despite the success of learning with noisy labels, existing approaches show limited performance when the noise level is extremely high, since deep neural networks (DNNs) are easily overfit to the training set with corrupted labels. In this paper, we introduce Lipschitz regularization to prevent the DNNs from over-fitting to noisy labels quickly. Meanwhile, to better detect and leverage the noisy samples, we propose a Lipschitz regularization based framework with a combination of adaptive modeling and detection module and improved semi-supervised learning. We propose to adaptively model the real distribution of the training set, and the implicit individual clean/noisy distribution, instead of parametric models. With Bayes' rule, we then compute the posterior probability of a sample being clean, which provides a dynamic threshold for the detection of noisy labels. To reduce training instability caused by less labeled data with severe label noise, we improve the semi-supervised learning by combining the advantages of Mixup and FixMatch. It can not only increase the diversity of unlabeled samples, but also improve the generalization capability of the DNNs to avoid over-fitting. Experiments on several benchmarks demonstrate that our approach achieves comparable results with the state-of-the-art methods in the less-noisy environment, and obtains a substantial improvement (∼ 8% and ∼ 6% in accuracy on CIFAR-10 and CIFAR-100 respectively) with severe noise. [ABSTRACT FROM AUTHOR]

• We propose a network compression model based on ℓ 1 / 2 regularization. To the best of our knowledge, it is the first work utilizing non-Lipschitz continuous regularization to compress DNNs. • We strictly prove the correspondence between ℓ p (0 < p < 1) and Hyper-Laplacian prior. Based on this prior, we suggest utilizing ℓ 1 / 2 , as the single regularizer, to sparsify the connections and neurons of the network simultaneously. • We give a closed-form, threshold solution to the proximal operator of ℓ 1 / 2 , and consequently design a stochastic proximal gradient algorithm to train the resulting model. • We conduct experiments to validate the performance of the proposed method. The results demonstrate that our method outperforms benchmark methods in terms of accuracy, computation and memory costs. Deep neural network(DNN) has achieved unprecedented success in many fields. However, its large model parameters which bring a great burden on storage and calculation hinder the development and application of DNNs. It is worthy of compressing the model to reduce the complexity of the DNN. Sparsity-inducing regularizer is one of the most common tools for compression. In this paper, we propose utilizing the ℓ 1 / 2 quasi-norm to zero out weights of neural networks and compressing the networks automatically during the learning process. To our knowledge, it is the first work applying the non-Lipschitz continuous regularizer for the compression of DNNs. The resulting sparse optimization problem is solved by stochastic proximal gradient algorithm. For further convenience of calculation, an approximation of the threshold-form solution to the proximal operator with ℓ 1 / 2 is given at the same time. Extensive experiments with various datasets and baselines demonstrate the advantages of our new method. [ABSTRACT FROM AUTHOR]

• We analyze shortcomings of graph construction strategies in previous algorithms. • We propose to boost graph matching by learning reliable graph patterns without input graph structures. • We integrate the learning of graph structures, node representations and matching decisions into a unified framework. • We provide insightful analysis and discussion about the effectiveness of learnt graph patterns. In recent years, powered by the learned discriminative representation via graph neural network (GNN) models, deep graph matching methods have made great progresses in the task of matching semantic features. However, these methods usually rely on heuristically generated graph patterns, which may introduce unreliable relationships to hurt the matching performance. In this paper, we propose a joint graph learning and matching network, named GLAM, to explore reliable graph structures for boosting graph matching. GLAM adopts a pure attention-based framework for both graph learning and graph matching. Specifically, it employs two types of attention mechanisms, self-attention and cross-attention for the task. The self-attention discovers the relationships between features to further update feature representations over the learnt structures; and the cross-attention computes cross-graph correlations between the two feature sets to be matched for feature reconstruction. Moreover, the final matching solution is directly derived from the output of the cross-attention layer, without employing a specific matching decision module. The proposed method is evaluated on three popular visual matching benchmarks (Pascal VOC, Willow Object and SPair-71k), and it outperforms previous state-of-the-art graph matching methods on all benchmarks. Furthermore, the graph patterns learnt by our model are validated to be able to remarkably enhance previous deep graph matching methods by replacing their handcrafted graph structures with the learnt ones. [ABSTRACT FROM AUTHOR]

• We explore the flexible versions of NP-Attack, when combined with the surrogate models. Our method could show a better query efficiency, demonstrating that NP-Attack outperforms with or without surrogate models. • We add some tiling tricks on NP-Attack to improve query efficiency. Moreover, we also design some ablation study experiments on tiling parameters. • We also evaluate NP-Attack on adversarial defense models to further discuss the capability of our method. Extensive experiments on benchmark demonstrate that our NP-Attack still outperforms existing evolution strategy methods in these black-box attack tasks. Deep neural networks (DNNs) have demonstrated excellent performance on various tasks, yet they are under the risk of adversarial examples that can be easily generated when the target model is accessible to an attacker (white-box setting). As plenty of machine learning models have been deployed via online services that only provide query outputs from inaccessible models (e.g. , Google Cloud Vision API2), black-box adversarial attacks raise critical security concerns in practice rather than white-box ones. However, existing query-based black-box adversarial attacks often require excessive model queries to maintain a high attack success rate. Therefore, in order to improve query efficiency, we explore the distribution of adversarial examples around benign inputs with the help of image structure information characterized by a Neural Process, and propose a Neural Process based black-box adversarial attack (NP-Attack) in this paper. Our proposed NP-Attack could be further boosted when applied with surrogate models or tiling tricks. Extensive experiments show that NP-Attack could greatly decrease the query counts under the black-box setting. [ABSTRACT FROM AUTHOR]

Annotation is a crucial first step in the semantic segmentation of urban images that facilitates the development of autonomous navigation systems. However, annotating complex urban images is time-consuming and challenging. It requires significant human effort making it expensive and error-prone. To reduce human effort during annotation, multiple images need to be annotated in a short time-span. In this paper, we introduce ScribbleNet, an interactive image segmentation algorithm to address this issue. Our approach provides users with a pre-segmented image that iteratively improves the segmentation using scribble as an annotation input. This method is based on conditional inference and exploits the learnt correlations in a deep neural network (DNN). ScribbleNet can: (1) work with urban city scenes captured in unseen environments, (2) annotate new classes not present in the training set, and (3) correct several labels at once. We compare this method with other interactive segmentation approaches on multiple datasets such as CityScapes, BDD, Mapillary Vistas, KITTI, and IDD. ScribbleNet reduces the annotation time of an image by up to 14.7 × over manual annotation and up to 5.4 × over the current approaches. The algorithm is integrated into the publicly available LabelMe image annotation tool and will be released as an open-source software. [ABSTRACT FROM AUTHOR]

• We propose a new proxy-based loss, which can fully consider the intra-class variance of each class. We set a learnable margin for each class, so as to better maintain the intra-class distribution in the embedding space. • Our loss increases the consideration of semantic relations between proxies, which can ensure the separation between classes, thus further maintaining the intra-class distribution in the embedding space. • The standard embedding network trained with our loss achieves state-of-the-art performance on several common benchmarks of metric learning. With the deepening of deep neural network research, deep metric learning has been further developed and achieved good results in many computer vision tasks. Deep metric learning trains the deep neural network by designing appropriate loss functions, and the deep neural network projects the training samples into an embedding space, where similar samples are very close, while dissimilar samples are far away. In the past two years, the proxy-based loss achieves remarkable improvements, boosts the speed of convergence and is robust against noisy labels and outliers due to the introduction of proxies. In the previous proxy-based losses, fixed margins were used to achieve the goal of metric learning, but the intra-class variance of fine-grained images were not fully considered. In this paper, a new proxy-based loss is proposed, which aims to set a learnable margin for each class, so that the intra-class variance can be better maintained in the final embedding space. Moreover, we also add a loss between proxies, so as to improve the discrimination between classes and further maintain the intra-class distribution. Our method is evaluated on fine-grained image retrieval, person re-identification and remote sensing image retrieval common benchmarks. The standard network trained by our loss achieves state-of-the-art performance. Thus, the possibility of extending our method to different fields of pattern recognition is confirmed. [ABSTRACT FROM AUTHOR]

• First, we propose novel procedures based on sub-pixel convolutions and mathematical morphology to construct pseudo-morphological operations using standard convolution layers. • We integrate these procedures into deep networks using morphological layers and NAS algorithms. We demonstrate empirically that our architecture tailored to morphological layers can outperform conventional convolutional layers. • We outline some current issues in NAS and introduce the problem of choosing the backbone, i.e.the higher-level architecture design on which the search will be performed. We offer novel network space descriptions suitable for the edge identification job. • We are the first to examine architectural search mixed with morphological procedures for edge detection. Our new specialized architecture achieves state-of-the-art performance for edge detection. Deep Neural Networks (DNNs) are generated by sequentially performing linear and non-linear processes. The combination of linear and non-linear procedures is critical for generating a sufficiently deep feature space. Most non-linear operators are derivations of activation functions or pooling functions. Mathematical morphology is a branch of mathematics that provides non-linear operators for various image processing problems. This paper investigates the utility of integrating these operations into an end-to-end deep learning framework. DNNs are designed to acquire a realistic representation for a particular job. Morphological operators give topological descriptors that convey salient information about the shapes of objects depicted in images. We propose a method based on meta-learning to incorporate morphological operators into DNNs. The learned architecture demonstrates how our novel morphological operations significantly increase DNN performance on various tasks, including picture classification, edge detection, and semantic segmentation. Our codes are available at https://nao-morpho.github.io/. [ABSTRACT FROM AUTHOR]

• We present a timely and comprehensive survey on robust adversarial training. • This survey offers the fundamentals of adversarial training, a unified theory that can be used to interpret various methods, and a comprehensive summarization of different methodologies. • This survey also addresses three important research focus in adversarial training: interpretability, robust generalization, and robustness evaluation, which can stimulate future inspirations as well as research outlook. Deep neural networks have achieved remarkable success in machine learning, computer vision, and pattern recognition in the last few decades. Recent studies, however, show that neural networks (both shallow and deep) may be easily fooled by certain imperceptibly perturbed input samples called adversarial examples. Such security vulnerability has resulted in a large body of research in recent years because real-world threats could be introduced due to the vast applications of neural networks. To address the robustness issue to adversarial examples particularly in pattern recognition, robust adversarial training has become one mainstream. Various ideas, methods, and applications have boomed in the field. Yet, a deep understanding of adversarial training including characteristics, interpretations, theories, and connections among different models has remained elusive. This paper presents a comprehensive survey trying to offer a systematic and structured investigation on robust adversarial training in pattern recognition. We start with fundamentals including definition, notations, and properties of adversarial examples. We then introduce a general theoretical framework with gradient regularization for defending against adversarial samples - robust adversarial training with visualizations and interpretations on why adversarial training can lead to model robustness. Connections will also be established between adversarial training and other traditional learning theories. After that, we summarize, review, and discuss various methodologies with defense/training algorithms in a structured way. Finally, we present analysis, outlook, and remarks on adversarial training. [ABSTRACT FROM AUTHOR]

• A more flexible HCF neuron model for DNNs is introduced; it constructs geometries in an n-dimensional space by changing weights and hyper-parameters and thus, possesses higher variability and plasticity. Furthermore, the approximation theorem and proof for arbitrary continuous infinite functions are presented, and the fitting ability of the HCF neuron 95 model is demonstrated. • HCFNN architecture based on the HCF neuron is proposed; it is used to mine specific feature representations and achieve adaptive parameter learning. Next, a novel adaptive optimization method for weights and hyper-parameters is proposed to achieve effective network learning. The 100 learned network model has better expression and learning ability with fewer neurons. • We conduct experiments on nine datasets in several domains, including the two-spirals problem, natural object recognition, face recognition, and person re-ID. Experimental results show that the proposed method has better 105 learning performance and generalization ability than the commonly used M-P and RBF neural networks. In addition, our method can improve the performance of various image recognition tasks and acquire good generalization. Recent advances in deep neural networks (DNNs) have mainly focused on innovations in network architecture and loss function. In this paper, we introduce a flexible high-order coverage function (HCF) neuron model to replace the fully-connected (FC) layers. The approximation theorem and proof for the HCF are also presented to demonstrate its fitting ability. Unlike the FC layers, which cannot handle high-dimensional data well, the HCF utilizes weight coefficients and hyper-parameters to mine underlying geometries with arbitrary shapes in an n-dimensional space. To explore the power and potential of our HCF neuron model, a high-order coverage function neural network (HCFNN) is proposed, which incorporates the HCF neuron as the building block. Moreover, a novel adaptive optimization method for weights and hyper-parameters is designed to achieve effective network learning. Comprehensive experiments on nine datasets in several domains validate the effectiveness and generalizability of the HCF and HCFNN. The proposed method provides a new perspective for further developments in DNNs and ensures wide application in the field of image classification. The source code is available at https://github.com/Tough2011/HCFNet.git [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel part-level feature extraction method to enhance the discriminative ability of deep convolutional features for the task of fine-grained vehicle recognition. Generally, the challenges for fine-grained vehicle recognition are mainly caused by the subtle visual differences between part regions of vehicles. Therefore, it is essential to extract discriminative features from part regions. Many existing methods, especially deep convolutional neural networks (D-CNNs), tend to detect the discriminative part regions explicitly or learn the part information implicitly through network restructuring and neglect the abundant part-level information contained in the high-level features generated by CNNs. In light of this, we propose a simple and effective part-level feature extraction method to enhance the representation of part-level features within the global features of target object generated by the backbone networks. The proposed method is built on the deep convolutional layers from which the discriminative part features could be integrated and extracted accordingly. More specifically, a basic feature grouping module is adopted to integrate the feature maps of deep convolutional layers into groups in each of which the related discriminative parts are assembled. The feature grouping process is performed in a multi-stage manner to ensure the integration process. Then a fusion module follows to model the coarse-to-fine relationship of the part features and further ensure the integrity and effectiveness of the part features. We conduct comparison experiments on public datasets, and the results show that the proposed method achieves comparable performance with state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

• Current momentum-based methods usually suffer from the transferability saturation dilemma, which may degrade their capacity to break black-box models. • To mitigate the transferability saturation effect, we propose cyclical optimization that divides the generation process into multiple phases and treats the velocity from the previous phase as helpful knowledge to guide a new attack? We design cyclical augmentation algorithm for further improving fooling rates by stabilizing update directions and learning diverse boundary information without additional retraining costs. • Cyclical optimization and cyclical augmentation enhance black-box adversarial examples from the perspectives of the optimization and augmentation, respectively. We can apply them simultaneously to achieve more superior performance, referred to as cyclical adversarial attack. • Our method has well generalizability to integrate with existing methods, and establishes state-of-the-art for transferable attacks against normally trained models and defenses under both standalone and combination settings Deep neural networks (DNNs) have shown vulnerability to adversarial attacks. By exploiting the transferability of adversarial examples, attackers can fool models under black-box settings without accessing the underlying information. However, they often exhibit weak performance when transferring to defenses, which may give a false sense of security. In this paper, we propose Cyclical Adversarial Attack (CA 2), a general and straightforward method to boost the transferability to break defenders. We first revisit the momentum-based methods from the perspective of optimization and find that they usually suffer from the transferability saturation dilemma. To address this, CA 2 performs cyclical optimization algorithm to produce adversarial examples. Unlike the standard momentum policy that accumulates the velocity to continuously update the solution, we divide the generation process into multiple phases and treat the velocity vectors from the previous phase as proper knowledge to guide a new adversarial attack with larger steps. Moreover, CA 2 applies a novel and compatible augmentation algorithm at every optimization in a loop manner for enhancing the black-box transferability further, referred to as cyclical augmentation. Extensive experiments conducted on a variety of models not only validate the efficacy of each designed algorithm in CA 2 , but also illustrate the superiority of our method compared with the state-of-the-art transferable attacks. Our implemental code is publicly available at https://github.com/mesunhlf/CA2. [ABSTRACT FROM AUTHOR]

• A novel strategy to construct a fixed-size feature map via neighbor selection and ordering. • A new node-feature convolution (NFC) layer for graph convolutional network (GCN). • Insightful studies on varying aggregators, neighborhood size, and model depth. • Demonstration of the efficacy of NFC-based GCNs on benchmark datasets. Graph convolutional network (GCN) is an effective neural network model for graph representation learning. However, standard GCN suffers from three main limitations: (1) most real-world graphs have no regular connectivity and node degrees can range from one to hundreds or thousands, (2) neighboring nodes are aggregated with fixed weights, and (3) node features within a node feature vector are considered equally important. Several extensions have been proposed to tackle the limitations respectively. This paper focuses on tackling all the proposed limitations. Specifically, we propose a new node-feature convolutional (NFC) layer for GCN. The NFC layer first constructs a feature map using features selected and ordered from a fixed number of neighbors. It then performs a convolution operation on this feature map to learn the node representation. In this way, we can learn the usefulness of both individual nodes and individual features from a fixed-size neighborhood. Experiments on three benchmark datasets show that NFC-GCN consistently outperforms state-of-the-art methods in node classification. [ABSTRACT FROM AUTHOR]

• A table detection approach with heterogeneous formats for business documents working on anonymized data. • A new graph neural network architecture that poses the table detection problems in terms of node and edge classification. • A final consensus layer based on the belief propagation algorithm to marginalize the edge probability. • Extensive experimentation on three document datasets. Tabular structures in business documents offer a complementary dimension to the raw textual data. For instance, there is information about the relationships among pieces of information. Nowadays, digital mailroom applications have become a key service for workflow automation. Therefore, the detection and interpretation of tables is crucial. With the recent advances in information extraction, table detection and recognition has gained interest in document image analysis, in particular, with the absence of rule lines and unknown information about rows and columns. However, business documents usually contain sensitive contents limiting the amount of public benchmarking datasets. In this paper, we propose a graph-based approach for detecting tables in document images which do not require the raw content of the document. Hence, the sensitive content can be previously removed and, instead of using the raw image or textual content, we propose a purely structural approach to keep sensitive data anonymous. Our framework uses graph neural networks (GNNs) to describe the local repetitive structures that constitute a table. In particular, our main application domain are business documents. We have carefully validated our approach in two invoice datasets and a modern document benchmark. Our experiments demonstrate that tables can be detected by purely structural approaches. [ABSTRACT FROM AUTHOR]

• Derived an expression that estimates the false resemblance of two ECG samples. • Three classes of features such as interval, amplitude and angle are used. • The cumulative probability of false ECG resemblance with 14 features is reported as 1.3 × 10 − 20 . • Compared empirical and theoretical performance of ECG independence. • Successfully prove the admissibility of ECG to use as biometrics with scientific evidence. The biometric authentication system using electrocardiogram (ECG) may protect individuals' privacy and prevent identity frauds. Researchers have demonstrated that ECG is suitable for biometrics use due to its pervasiveness, immutability, measurability, acceptance, and individuality. However, ECG's statistical independence for biometric authentication has yet to be substantiated. Thereby, this paper proposes a novel model to evaluate the statistical independence of ECG among individuals using heartbeat morphological features. The signal is qualitatively improved and heartbeat features are extracted using signal processing techniques. Three classes of features such as interval, amplitude, and angle are extracted from each heartbeat. The hypothesis estimating the probability of resemblance of interval, amplitude, and angle classes of features is derived. The accumulated effect of these classes of features measure the statistical independence of ECG. Further, the proposed model of statistical independence of ECG biometrics is validated by comparing the statistical performance with the empirical performance of the ECG verification system. The empirical performance is estimated using three different ECG biometric methods, i.e., traditional intraclass-interclass features, artificial neural network, and convolutional neural network. The false resemblance probabilities of heartbeats among individuals computed for four interval class features, five amplitude class features, and five angle class features are found to be 3.4 × 10 − 6 , 1.0 × 10 − 7 , and 3.9 × 10 − 8 , respectively. The cumulative probability of resemblance computed using fourteen heartbeat features of interval, amplitude, and angle classes is found as 1.3 × 10 − 20 . [ABSTRACT FROM AUTHOR]