Search

Most publicly available datasets for image classification are with single labels, while images are inherently multilabeled in our daily life. Such an annotation gap makes many pretrained single-label classification models fail in practical scenarios. For aerial images, this annotation issue is more concerned: Aerial data naturally cover a relatively large land area with multiple labels, while annotated aerial datasets currently publicly available (e.g., UCM and AID) are single-labeled. As manually annotating multilabel aerial images (MAIs) would be time-/ labor-consuming, we propose a novel self-correction integrated domain adaptation (SCIDA) method for automatic multilabel learning. SCIDA is weakly supervised, i.e., automatically learning the multilabel image classification model from using massive, publicly available single-label images. To achieve this goal, we propose a novel labelwise self-correction (LWC) module to better explore underlying label correlations. This module also makes the unsupervised domain adaptation (UDA) from single-label to multilabel data possible. For model training, the proposed method uses single-label information yet requires no prior knowledge of multilabeled data and predicts labels for MAIs. Through extensive evaluations, the proposed model, which is trained with single-labeled MAI-AID-s and MAI-UCM-s datasets, achieves much better performances than comparative methods on our collected multiscene aerial image dataset. The code and data are available on GitHub (https://github.com/Ryan315/Single2multi-DA). [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) decoding aims to identify the perceptual, semantic, and cognitive content of neural processing based on noninvasively measured brain activity. Traditional EEG decoding methods have achieved moderate success when applied to data acquired in static, well-controlled lab environments. However, an open-world environment is a more realistic setting, where situations affecting EEG recordings can emerge unexpectedly, significantly weakening the robustness of existing methods. In recent years, deep learning (DL) has emerged as a potential solution for such problems due to its superior capacity in feature extraction. It overcomes the limitations of defining handcrafted features or features extracted using shallow architectures but typically requires large amounts of costly, expertly labeled data, something not always obtainable. Combining DL with domain-specific knowledge may allow for the development of robust approaches that decode brain activity even with small-sample data. Although various DL techniques have been proposed to tackle some of the challenges in EEG decoding, a systematic tutorial overview, particularly for open-world applications, is currently lacking. This article therefore provides a comprehensive survey of DL methods for open-world EEG decoding and identifies promising research directions to inspire future studies for EEG decoding in real-world applications. [ABSTRACT FROM AUTHOR]

Automatic liver and tumor segmentation remain a challenging topic, which subjects to the exploration of 2D and 3D contexts in CT volume. Existing methods are either only focus on the 2D context by treating the CT volume as many independent image slices (but ignore the useful temporal information between adjacent slices), or just explore the 3D context lied in many little voxels (but damage the spatial detail in each slice). These factors lead an inadequate context exploration together for automatic liver and tumor segmentation. In this paper, we propose a novel full-context convolution neural network to bridge the gap between 2D and 3D contexts. The proposed network can utilize the temporal information along the Z axis in CT volume while retaining the spatial detail in each slice. Specifically, a 2D spatial network for intra-slice features extraction and a 3D temporal network for inter-slice features extraction are proposed separately and then are guided by the squeeze-and-excitation layer that allows the flow of 2D context and 3D temporal information. To address the severe class imbalance issue in the CT volume and meanwhile improve the segmentation performance, a loss function consisting of weighted cross-entropy and jaccard distance is proposed. During the network training, the 2D and 3D contexts are learned jointly in an end-to-end way. The proposed network achieves competitive results on the Liver Tumor Segmentation Challenge (LiTS) and the 3D-IRCADB datasets. This method should be a new promising paradigm to explore the contexts for liver and tumor segmentation. [ABSTRACT FROM AUTHOR]

Automatic skin lesion analysis of dermoscopy images remains a challenging topic. In this paper, we propose an end-to-end multi-task deep learning framework for automatic skin lesion analysis. The proposed framework can perform skin lesion detection, classification, and segmentation tasks simultaneously. To address the class imbalance issue in the dataset (as often observed in medical image datasets) and meanwhile to improve the segmentation performance, a loss function based on the focal loss and the jaccard distance is proposed. During the framework training, we employ a three-phase joint training strategy to ensure the efficiency of feature learning. The proposed framework outperforms state-of-the-art methods on the benchmarks ISBI 2016 challenge dataset towards melanoma classification and ISIC 2017 challenge dataset towards melanoma segmentation, especially for the segmentation task. The proposed framework should be a promising computer-aided tool for melanoma diagnosis. [ABSTRACT FROM AUTHOR]

Deep Convolutional Neural Networks (DCNNs) have been widely used in detection of global manipulations. However, designing effective DCNNs for specific image forensics tasks generally requires domain knowledge and experience gained from abundant experiments, which is time-consuming and labor-expensive. Approaches of automated network designing have been proposed for image classification tasks which are image-content focused, however they may not be suitable to image forensics tasks which rely on identifying subtle traces left by certain image operations. In this paper, we make the first attempt to automate the neural network architecture design for detection of global manipulations. The process of constructing a network is modeled as sequentially selecting optimal architecture modules to generate high-performing CNNs for specific forensic tasks through reinforcement learning. The module-based search space is proposed to make the designing process efficient. Advanced connection patterns (e.g., dense connectivity), which were shown preferred for global manipulation detections, are included in the modules to improve the representational power of the network. Experimental results show that the proposed approach can adaptively construct effective CNN architectures for two common forensic tasks, including multi-purpose forensics and the processing history detection. The auto-designed networks can outperform the state-of-the-art manually designed networks. [ABSTRACT FROM AUTHOR]

With the rapid development of deep learning models, their performances in various tasks have improved; meanwhile, their increasingly intricate architectures make them difficult to interpret. To tackle this challenge, model interpretability is essential and has been investigated in a wide range of applications. For end users, model interpretability can be used to build trust in the deployed machine learning models. For practitioners, interpretability plays a critical role in model explanation, model validation, and model improvement to develop a faithful model. In this paper, we propose a novel Multi-scale Interpretation (MINT) model for convolutional neural networks using both the perturbation-based and the gradient-based interpretation approaches. It learns the class-discriminative interpretable knowledge from the multi-scale perturbation of feature information in different layers of deep networks. The proposed MINT model provides the coarse-scale and the fine-scale interpretations for the attention in the deep layer and specific features in the shallow layer, respectively. Experimental results show that the MINT model presents the class-discriminative interpretation of the network decision and explains the significance of the hierarchical network structure. [ABSTRACT FROM AUTHOR]

In this paper, we present a Convolutional Neural Network (CNN) regression approach to address the two major limitations of existing intensity-based 2-D/3-D registration technology: 1) slow computation and 2) small capture range. Different from optimization-based methods, which iteratively optimize the transformation parameters over a scalar-valued metric function representing the quality of the registration, the proposed method exploits the information embedded in the appearances of the digitally reconstructed radiograph and X-ray images, and employs CNN regressors to directly estimate the transformation parameters. An automatic feature extraction step is introduced to calculate 3-D pose-indexed features that are sensitive to the variables to be regressed while robust to other factors. The CNN regressors are then trained for local zones and applied in a hierarchical manner to break down the complex regression task into multiple simpler sub-tasks that can be learned separately. Weight sharing is furthermore employed in the CNN regression model to reduce the memory footprint. The proposed approach has been quantitatively evaluated on 3 potential clinical applications, demonstrating its significant advantage in providing highly accurate real-time 2-D/3-D registration with a significantly enlarged capture range when compared to intensity-based methods. [ABSTRACT FROM AUTHOR]

Median filtering detection has recently drawn much attention in image editing and image anti-forensic techniques. Current image median filtering forensics algorithms mainly extract features manually. To deal with the challenge of detecting median filtering from small-size and compressed image blocks, by taking into account of the properties of median filtering, we propose a median filtering detection method based on convolutional neural networks (CNNs), which can automatically learn and obtain features directly from the image. To our best knowledge, this is the first work of applying CNNs in median filtering image forensics. Unlike conventional CNN models, the first layer of our CNN framework is a filter layer that accepts an image as the input and outputs its median filtering residual (MFR). Then, via alternating convolutional layers and pooling layers to learn hierarchical representations, we obtain multiple features for further classification. We test the proposed method on several experiments. The results show that the proposed method achieves significant performance improvements, especially in the cut-and-paste forgery detection. [ABSTRACT FROM PUBLISHER]

Corticomuscular coupling analysis, i.e., examining the relations between simultaneously recorded brain (e.g., electroencephalography—EEG) and muscle (e.g., electro-myography—EMG) signals, is a useful tool for understanding aspects of human motor control. Traditionally, the most popular method to assess corticomuscular coupling has been the pairwise magnitude-squared coherence (MSC) between EEG and concomitant EMG recordings. In this paper, we propose assessing corticomuscular coupling by combining partial least squares (PLS) and independent component analysis (ICA), which addresses many of the limitations of MSC, such as difficulty in robustly assessing group inference and relying on the biologically implausible assumption of pairwise interaction between brain and muscle recordings. In the proposed framework, response relevance and statistical independence are jointly incorporated into a multiobjective optimization function to meaningfully combine the goals of PLS and ICA under the same mathematical umbrella. Simulations, performed under realistic assumptions, illustrated the utility of such an approach. The method was extended to address intersubject variability to robustly discover common corticomuscular coupling patterns across subjects. We then applied the proposed framework to concurrent EEG and EMG data collected in a Parkinson’s disease (PD) study. The results from applying the proposed technique revealed temporal components in the EEG and EMG that were significantly correlated with one another. In addition to the expected motor areas, the corresponding spatial activation patterns demonstrated enhanced occipital connectivity in PD subjects, consistent with previous studies suggesting that PD subjects rely excessively on visual information to counteract the deficiency in being able to generate internal commands from their affected basal ganglia. [ABSTRACT FROM PUBLISHER]

Abstract: Using surface electromyography (sEMG) signal for efficient recognition of hand gestures has attracted increasing attention during the last decade, with most previous work being focused on recognition of upper arm and gross hand movements and some work on the classification of individual finger movements such as finger typing tasks. However, relatively few investigations can be found in the literature for automatic classification of multiple finger movements such as finger number gestures. This paper focuses on the recognition of number gestures based on a 4-channel wireless sEMG system. We investigate the effects of three popular feature types (i.e. Hudgins’ time–domain features (TD), autocorrelation and cross-correlation coefficients (ACCC) and spectral power magnitudes (SPM)) and four popular classification algorithms (i.e. k-nearest neighbor (k-NN), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA) and support vector machine (SVM)) in offline recognition. Motivated by the good performance of SVM, we further propose combining the three features and employing a new classification method, multiple kernel learning SVM (MKL-SVM). Real sEMG results from six subjects show that all combinations, except k-NN or LDA using ACCC features, can achieve above 91% average recognition accuracy, and the highest accuracy is 97.93% achieved by the proposed MKL-SVM method using the three feature combination (3F). Referring to the offline recognition results, we also implement a real-time recognition system. Our results show that all six subjects can achieve a real-time recognition accuracy higher than 90%. The number gestures are therefore promising for practical applications such as human–computer interaction (HCI). [Copyright &y& Elsevier]

Identification of constituent components of each sign gesture can be beneficial to the improved performance of sign language recognition (SLR), especially for large-vocabulary SLR systems. Aiming at developing such a system using portable accelerometer (ACC) and surface electromyographic (sEMG) sensors, we propose a framework for automatic Chinese SLR at the component level. In the proposed framework, data segmentation, as an important preprocessing operation, is performed to divide a continuous sign language sentence into subword segments. Based on the features extracted from ACC and sEMG data, three basic components of sign subwords, namely the hand shape, orientation, and movement, are further modeled and the corresponding component classifiers are learned. At the decision level, a sequence of subwords can be recognized by fusing the likelihoods at the component level. The overall classification accuracy of 96.5% for a vocabulary of 120 signs and 86.7% for 200 sentences demonstrate the feasibility of interpreting sign components from ACC and sEMG data and clearly show the superior recognition performance of the proposed method when compared with the previous SLR method at the subword level. The proposed method seems promising for implementing large-vocabulary portable SLR systems. [ABSTRACT FROM PUBLISHER]

Local feature points have been widely investigated in solving problems in computer vision, such as robust matching and object detection. However, its investigation in the area of image hashing is still limited. In this paper, we propose a novel shape-contexts-based image hashing approach using robust local feature points. The contributions are twofold: 1) The robust SIFT-Harris detector is proposed to select the most stable SIFT keypoints under various content-preserving distortions. 2) Compact and robust image hashes are generated by embedding the detected local features into shape-contexts-based descriptors. Experimental results show that the proposed image hashing is robust to a wide range of distortions and attacks, due to the benefits of robust salient keypoints detection and the shape-contexts-based feature descriptors. When compared with the current state-of-the-art schemes, the proposed scheme yields better identification performances under geometric attacks such as rotation attacks and brightness changes, and provides comparable performances under classical distortions such as additive noise, blurring, and compression. Also, we demonstrate that the proposed approach could be applied for image tampering detection. [ABSTRACT FROM PUBLISHER]

In this letter, we propose an eigenspectra-based feature extraction technique for classification of multivariate surface electromyographic (sEMG) recordings. The proposed method exploits the maximum eigenvalue vectors of the time-varying covariance patterns between sEMG channels. Together with a support vector machine (SVM) classifier, the proposed feature extraction technique is shown to be more reliable and robust, and it enhances classification between stroke and normal subjects, compared to the conventional univariate analysis methods that examine each muscle individually. In addition, analysis results show that the spatial whitening operation enhances the discriminability of eigenspectral features. This simple, easily-implemented, biologically-inspired approach is able to succinctly capture the subtle differences in muscle recruitment patterns between healthy and disease states. It appears to be a promising means to monitor motor performance in disease subjects. [ABSTRACT FROM AUTHOR]

Deep learning-based fusion of spectral-spatial information is increasingly dominant for hyperspectral image (HSI) classification. However, due to insufficient samples, current feature fusion methods often neglect joint interactions. In this paper, to further improve the classification accuracy, we propose a dual-attention-guided interactive multi-scale residual network (DA-IMRN) to explore the joint spectral-spatial information and assign pixel-wise labels for HSIs without information leakage. In DA-IMRN, two branches focusing on spatial and spectral information separately are employed for feature extraction. A bidirectional-attention mechanism is employed to guide the interactive feature learning between two branches and promote refined feature maps. In addition, we extract deep multi-scale features corresponding to multiple receptive fields from limited samples via a multi-scale spectral/spatial residual block, to improve classification performance. Experimental results on three benchmark datasets (i.e., Salinas Valley, Pavia University, and Indian Pines) support that attention-guided multi-scale feature learning can effectively explore the joint spectral-spatial information. The proposed method outperforms state-of-the-art methods with the overall accuracy of 91.26%, 93.33%, and 82.38%, and the average accuracy of 94.22%, 89.61%, and 80.35%, respectively. [ABSTRACT FROM AUTHOR]

To improve image quality, subsampling or refining images generated from (unconditional) generative adversarial networks (GANs) has received much attention recently. These methods are less effective or inefficient, however, for conditional GANs (cGANs) that either condition on a class (i.e., class-conditional GANs) or a continuous variable (i.e., continuous cGANs or CcGANs). Thus, we introduce a novel subsampling scheme for cGANs: conditional density ratio rejection sampling (cDR-RS). Specifically, cDR-RS comprises an improved feature extraction mechanism and a conditional Softplus loss (cSP). We also derive an error bound for the density ratio model trained with the cSP loss. Fake images are accepted or rejected based on the improved estimated conditional density ratio. An additional filtering scheme further increases fake images' label consistency without losing diversity when sampling from CcGANs. We extensively test the effectiveness and efficiency of cDR-RS in sampling from both class-conditional GANs and CcGANs on six benchmark datasets. Experimental results demonstrate that the proposed method can achieve state-of-the-art performances in subsampling both types of conditional GANs. [ABSTRACT FROM AUTHOR]

Epilepsy is the second common brain disorder affecting 70 million people worldwide. Electroencephalogram (EEG) has been widely used for the diagnosis of epileptic seizures. However, most of the existing EEG-based seizure detection methods cannot maintain robust performance in real life conditions. This is where the EEG data are corrupted with different sources of noise and artifacts. This study presents a robust seizure detection system that works efficiently under the real life conditions as well as the ideal conditions. A feature learning method based on L1-penalized robust regression is developed and applied to the EEG spectra to recognize the most prominent features pertinent to epileptic seizures. The extracted features are then fed into the random forest classifier for seizure detection. Results on a public benchmark dataset show that the performance of this seizure detection system is superior to prior work. It first achieves seizure detection rates of 100.00% sensitivity, 100.00% specificity, and 100.00% classification accuracy under the ideal conditions. The proposed method is also proven to be robust in the presence of white noise and EEG artifacts mainly those arising from muscle activities and eyes-blinking. It is found to achieve seizure detection accuracies in the range of 90.00–100.00% when applied to noisy EEG data corrupted with high noise levels. To the best of our knowledge, there exists no work in the literature that detects seizures under these conditions. [ABSTRACT FROM AUTHOR]