Search

One of the challenges in hyperspectral image (HSI) classification is that there are limited labeled samples to train a classifier for very high-dimensional data. In practical applications, we often encounter an HSI domain (called target domain) with very few labeled data, while another HSI domain (called source domain) may have enough labeled data. Classes between the two domains may not be the same. This article attempts to use source class data to help classify the target classes, including the same and new unseen classes. To address this classification paradigm, a meta-learning paradigm for few-shot learning (FSL) is usually adopted. However, existing FSL methods do not account for domain shift between source and target domain. To solve the FSL problem under domain shift, a novel deep cross-domain few-shot learning (DCFSL) method is proposed. For the first time, DCFSL tackles FSL and domain adaptation issues in a unified framework. Specifically, a conditional adversarial domain adaptation strategy is utilized to overcome domain shift, which can achieve domain distribution alignment. In addition, FSL is executed in source and target classes at the same time, which can not only discover transferable knowledge in the source classes but also learn a discriminative embedding model to the target classes. Experiments conducted on four public HSI data sets demonstrate that DCFSL outperforms the existing FSL methods and deep learning methods for HSI classification. Our source code is available at https://github.com/Li-ZK/DCFSL-2021.

Due to powerful feature extraction capability, convolutional neural networks (CNNs) have been widely used for hyperspectral image (HSI) classification. However, because of a large number of parameters that need to be trained, sufficient training samples are usually required for deep CNN-based methods. Unfortunately, limited training samples are a common issue in the remote sensing community. In this study, a dual graph convolutional network (DGCN) is proposed for the supervised classification of HSI with limited training samples. The first GCN fully extracts features existing in and among HSI samples, while the second GCN utilizes label distribution learning, and thus, it potentially reduces the number of required training samples. The two GCNs are integrated through several iterations to decrease interclass distances, which leads to a more accurate classification step. Moreover, a new idea entitled multiscale feature cutout is proposed as a regularization technique for HSI classification (DGCN-M). Different from the regularization methods (e.g., dropout and DropBlock), the proposed multiscale feature cutout could randomly mask out multiscale region sizes in a feature map, which further reduces the overfitting problem and yields consistent improvement. Experimental results on the four popular hyperspectral data sets (i.e., Salinas, Indian Pines, Pavia, and Houston) indicate that the proposed method obtains good classification performance compared to state-of-the-art methods, which shows the potential of GCN for HSI classification.

Deep learning has achieved great success in hyperspectral image classification. However, its success relies on the availability of sufficient training samples. Unfortunately, the collection of training samples is expensive, time-consuming, and even impossible in some cases. Natural image data sets that are different from HSI, such as Image Net and mini-ImageNet, have abundant texture and structure information. Effective knowledge transfer between two heterogeneous data sets can significantly improve the accuracy of HSI classification. In this letter, heterogeneous few-shot learning (HFSL) for HSI classification is proposed with only a few labeled samples per class. Firstly, few-shot learning is performed on the mini-ImageNet data sets to learn the transferable knowledge. Then, to make full use of the spatial and spectral information, a spectral-spatial fusion network is devised. Spectral information is obtained by the residual network with pure one-dimensional operators. Spatial information is extracted by a convolution network with pure two-dimensional operators, and the weights of the spatial network are initialized by the weights of the model trained on the mini-ImageNet data sets. Finally, few-shot learning is fine-tuned on HSI to extract discriminative spectral-spatial features and individual knowledge, which can improve the classification performance of the new classification task. Experiments conducted on two public HSI data sets demonstrate that the HFSL outperforms the existing few-shot learning methods and supervised learning methods for HSI classification with only a few labeled samples. Our source code is available at https://github.com/Li-ZK/HFSL.

The lack of training samples remains one of the major obstacles in applying convolutional neural networks (CNNs) to the hyperspectral image (HSI) classification. In this letter, the accurate classification of HSI with limited training samples is investigated. Due to the advantages of minimizing the distance between samples in the same class and maximizing the distance between samples in different classes, siamese CNN is used for HSI classification with limited training samples. After that, to improve the classification performance, data augmentation is investigated for siamese CNN-based HSI classification. Specifically, pair data augmentation based on CutMix is proposed to generate the training pairs of the same or different classes and the new generated training pairs are used to train siamese CNN. Traditional data augmentation methods simply generate new training samples. It is not proper when data augmentation methods keep the loss function and change the content of the input at the same time. Therefore, a soft-loss-based siamese CNN, which changes its loss according to the coupled replacement data augmentation, is proposed to further address the HSI classification with limited training samples. In the experimental part, on two widely used hyperspectral datasets, the influences of different training samples and window sizes are discussed, and the experimental results reveal that the proposed soft-augmentation-based siamese CNN provides competitive results with limited training samples compared with state-of-the-art methods.

Recently, convolutional neural networks (CNNs) have been widely used for light detection and ranging (LiDAR) data classification. Although CNNs achieve good classification performance for LiDAR data classification, a lot of efforts are needed to design a proper architecture. In this letter, the automatic modularized design of CNN is explored for LiDAR data classification for the first time. First, a searchable architecture containing convolution and pooling operations is used to establish the search space. Then, the optimal building block (i.e., cell), which is the basic part of a deep CNN, is obtained from search space by a gradient decent-based algorithm. At last, by stacking several optimal building blocks, a deep CNN can be formulated for LiDAR data classification. Moreover, in order to mitigate the overfitting problem in training a CNN, improved label smoothing and feature regularization are proposed to further improve the classification performance of LiDAR data. The proposed classification models are evaluated on two popular LiDAR data sets (i.e., the Bayview Park and Houston data sets). The experimental results show that the proposed models provide the competitive results compared to the state-of-the-art methods.

In recent years, deep convolutional neural networks (CNNs) have been widely investigated for hyperspectral image (HSI) classification. The CNN-based HSI classifiers obtained good performance under the condition of sufficient training samples. In order to address the problem of limited training samples, in this letter, transfer learning is combined with CNN to address the issue of HSI classification. Pretrained models on large-scale data sets (e.g., ImageNet) can extract the general and discriminative features. Due to the fact that the extracted low-level and mid-level features can be reused for the HSI feature extraction, the CNN-based methods usually obtain good classification performance with insufficient training samples. The ImageNet data set has three channels, while the HSI data set contains hundreds of channels. Therefore, three channels of the HSI are randomly selected to formulate a transferring CNN. Then, several transferring CNNs are combined to establish an ensemble classification system with diversity. Moreover, an improved label smoothing technique is proposed to further improve the classification accuracy of the HSI. Experimental results on two popular hyperspectral data sets [i.e., Indian Pines and Kennedy Space Center (KSC)] show that the transferring CNN ensemble obtains good classification performance compared to the state-of-the-art methods.

In recent years, deep convolutional neural networks (CNNs) have been widely used for hyperspectral image (HSI) classification. The powerful feature extraction capability and high classification performance of CNN highly depend on sufficient training samples. Unfortunately, it is not a common situation because collecting training samples is time-consuming and expensive. In this letter, in order to make the most of deep CNN with limited training samples, dual-path siamese CNN (Dual-SCNN) is proposed for HSI classification. Specifically, the proposed classification framework is a combination of extended morphological profiles, CNN, siamese network, and spectral–spatial feature fusion. In order to solve the problem of insufficiency in hard negative pairs during the training of a siamese network, adversarial training is combined with Dual-SCNN (Dual-SCNN-AT) for HSI classification. Moreover, a data augmentation method titled mixup is combined with Dual-SCNN and Dual-SCNN-AT to further improve the classification performance of HSI. The obtained results on widely used hyperspectral data sets reveal that the proposed methods provide the competitive results in terms of classification accuracy, especially with limited training samples.

Deep convolutional neural networks (CNNs) have shown their outstanding performance in the hyperspectral image (HSI) classification. The success of CNN-based HSI classification relies on the availability sufficient training samples. However, the collection of training samples is expensive and time consuming. Besides, there are many pretrained models on large-scale data sets, which extract the general and discriminative features. The proper reusage of low-level and midlevel representations will significantly improve the HSI classification accuracy. The large-scale ImageNet data set has three channels, but HSI contains hundreds of channels. Therefore, there are several difficulties to simply adapt the pretrained models for the classification of HSIs. In this article, heterogeneous transfer learning for HSI classification is proposed. First, a mapping layer is used to handle the issue of having different numbers of channels. Then, the model architectures and weights of the CNN trained on the ImageNet data sets are used to initialize the model and weights of the HSI classification network. Finally, a well-designed neural network is used to perform the HSI classification task. Furthermore, attention mechanism is used to adjust the feature maps due to the difference between the heterogeneous data sets. Moreover, controlled random sampling is used as another training sample selection method to test the effectiveness of the proposed methods. Experimental results on four popular hyperspectral data sets with two training sample selection strategies show that the transferred CNN obtains better classification accuracy than that of state-of-the-art methods. In addition, the idea of heterogeneous transfer learning may open a new window for further research.

Recently, many convolutional neural network (CNN)-based methods have been proposed to tackle the classification task of hyperspectral images (HSI). In fact, CNN has become the de-facto standard for HSI classification. It seems that the traditional neural networks such as multi-layer perceptron (MLP) are not competitive for HSI classification. However, in this study, we try to prove that the MLP can achieve good classification performance of HSI if it is properly designed and improved. The proposed Modified-MLP for HSI classification contains two special parts: spectral–spatial feature mapping and spectral–spatial information mixing. Specifically, for spectral–spatial feature mapping, each input sample of HSI is divided into a sequence of 3D patches with fixed length and then a linear layer is used to map the 3D patches to spectral–spatial features. For spectral–spatial information mixing, all the spectral–spatial features within a single sample are feed into the solely MLP architecture to model the spectral–spatial information across patches for following HSI classification. Furthermore, to obtain the abundant spectral–spatial information with different scales, Multiscale-MLP is proposed to aggregate neighboring patches with multiscale shapes for acquiring abundant spectral–spatial information. In addition, the Soft-MLP is proposed to further enhance the classification performance by applying soft split operation, which flexibly capture the global relations of patches at different positions in the input HSI sample. Finally, label smoothing is introduced to mitigate the overfitting problem in the Soft-MLP (Soft-MLP-L), which greatly improves the classification performance of MLP-based method. The proposed Modified-MLP, Multiscale-MLP, Soft-MLP, and Soft-MLP-L are tested on the three widely used hyperspectral datasets. The proposed Soft-MLP-L leads to the highest OA, which outperforms CNN by 5.76%, 2.55%, and 2.5% on the Salinas, Pavia, and Indian Pines datasets, respectively. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, which shows that the MLP-based methods are still competitive for HSI classification.

The accurate classification of radar jamming signal is a core step of anti-jamming. Recently, convolutional neural network (CNN) based methods have shown their powerfulness in signal processing. In this paper, a deep fusion method based on CNN is proposed to classify jamming signal acting on pulse compression radar. The proposed method consists of three subnetworks (i.e., 1D-CNN, 2D-CNN, and fusion network). 1D-CNN is used to extract deep features of original radar jamming signal. Meanwhile, in order to extract the time-frequency features, short time Fourier transform (STFT) is applied to jamming signal to obtain time-frequency spectrograms. Then, 2D-CNN is used to extract deep time-frequency features, which are useful for further features fusion processing. Fusion network is used to deeply fuse the extracted features of the aforementioned CNNs and softmax is used to finish the task of radar jamming signal classification. In addition, in order to alleviate the problem of overfitting and improve the generalization ability of proposed model, soft label smoothing is proposed. The experimental results reveal that the proposed method provides competitive results in term of classification accuracy.

Deep learning-based methods, especially deep convolutional neural networks (CNNs), have shown their effectiveness for hyperspectral image (HSI) classification. In previous deep CNN-based HSI classification methods, a cuboid is empirically determined as the input. The dimensionalities of the cuboid, including height and weight, are crucial to the final classification results. Unfortunately, these superparameters (i.e., the dimensionalities of input cube) are hand-crafted, which means the inputs of a classifier are not optimized according to the specific hyperspectral dataset. In this letter, spatial transformation network (STN) is explored to obtain the optimal input for CNN-based HSI classification for the first time. STN is used to translate, rotate, and scale the original input to obtain optimized input for the following CNN. Moreover, in order to mitigate the overfitting problem in CNN-based HSI classification, DropBlock is introduced as a regularization technique for HSI accurate classification. Compared with dropout, which is a popular regularization technique, DropBlock obtains better classification accuracy. The proposed methods are tested on two widely used hyperspectral data sets (i.e., Salinas and Kennedy Space Center). The obtained experimental results show that the proposed methods provide competitive results compared with state-of-the-art methods including deep CNN-based methods.

Deep learning-based methods, especially deep convolutional neural network (CNN), have proven their powerfulness in hyperspectral image (HSI) classification. On the other hand, ensemble learning is ...

In recent years, deep convolutional neural networks (CNNs) have been widely used for hyperspectral image (HSI) classification. Besides, ensemble learning is a useful way to enhance the classification performance. Therefore, in this study, a new method titled Boosting-CNN is proposed for HSI classification, which fully explored the advantages of deep CNN and ensemble learning. Specifically, several deep CNNs are well-designed to classify HSI. The samples are misclassified in a CNN have more weighs in the following CNN through adaptive boosting. The final classification result is obtained by weighted voting of several CNNs. For HSI classification issue, the number of samples in different classes varies greatly, however, traditional classification methods cannot handle this issue well. In order to address imbalance training samples in HSI classification, soft class balanced loss is proposed to mitigate the influence of imbalance training samples. Experimental results on two popular hyperspectral datasets (i.e., Salinas and Pavia University) show that the proposed method obtain better classification accuracy compared to comparison methods.

Jamming is a big threat to the survival of a radar system. Therefore, the recognition of radar jamming signal type is a part of radar countermeasure. Recently, convolutional neural networks (CNNs) have shown their effectiveness in radar signal processing, including jamming signal recognition. However, most of existing CNN methods do not regard radar jamming as a complex value signal. In this study, a complex-valued CNN (CV-CNN) is investigated to fully explore the inherent characteristics of a radar jamming signal, and we find that we can obtain better recognition accuracy using this method compared with a real-valued CNN (RV-CNN). CV-CNNs contain more parameters, which need more inference time. To reduce the parameter redundancy and speed up the recognition time, a fast CV-CNN (F-CV-CNN), which is based on pruning, is proposed for radar jamming signal fast recognition. The experimental results show that the CV-CNN and F-CV-CNN methods obtain good recognition performance in terms of accuracy and speed. The proposed methods open a new window for future research, which shows a huge potential of CV-CNN-based methods for radar signal processing.

Recently, a great many deep convolutional neural network (CNN)-based methods have been proposed for hyperspectral image (HSI) classification. Although the proposed CNN-based methods have the advantages of spatial feature extraction, they are difficult to handle the sequential data with and CNNs are not good at modeling the long-range dependencies. However, the spectra of HSI are a kind of sequential data, and HSI usually contains hundreds of bands. Therefore, it is difficult for CNNs to handle HSI processing well. On the other hand, the Transformer model, which is based on an attention mechanism, has proved its advantages in processing sequential data. To address the issue of capturing relationships of sequential spectra in HSI in a long distance, in this study, Transformer is investigated for HSI classification. Specifically, in this study, a new classification framework titled spatial-spectral Transformer (SST) is proposed for HSI classification. In the proposed SST, a well-designed CNN is used to extract the spatial features, and a modified Transformer (a Transformer with dense connection, i.e., DenseTransformer) is proposed to capture sequential spectra relationships, and multilayer perceptron is used to finish the final classification task. Furthermore, dynamic feature augmentation, which aims to alleviate the overfitting problem and therefore generalize the model well, is proposed and added to the SST (SST-FA). In addition, to address the issue of limited training samples in HSI classification, transfer learning is combined with SST, and another classification framework titled transferring-SST (T-SST) is proposed. At last, to mitigate the overfitting problem and improve the classification accuracy, label smoothing is introduced for the T-SST-based classification framework (T-SST-L). The proposed SST, SST-FA, T-SST, and T-SST-L are tested on three widely used hyperspectral datasets. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, which shows that the concept of Transformer opens a new window for HSI classification.

This letter presents a study of vehicle detection in high-resolution images using superpixel segmentation and iterative convolutional neural network strategy. First, a novel superpixel segmentation integrated with multiple local information constraints method is proposed to improve the segmentation results with a low breakage rate. To make training and detection more efficient, we extract meaningful and nonredundant patches based on the centers of the segmented superpixels. For reducing the instability in detection performance because of manual or random selection of samples, a training sample iterative selection strategy based on convolutional neural network is proposed. After a compact training sample subset is obtained from the original entire training set, a representative feature set with high discrimination ability between vehicle and background is extracted from these selected samples for detection. To further avoid overfitting the training and promote the detection efficiency, data augment and a main direction estimation method are used. Comparative experimental results on Toronto data indicated the effectiveness of our proposed method.

Light detection and ranging (LiDAR) is a useful data acquisition technique, which is widely used in a variety of practical applications. The classification of LiDAR-derived rasterized digital surface model (LiDAR-DSM) is a fundamental technique in LiDAR data processing. In recent years, deep learning methods, especially convolutional neural networks (CNNs), have shown their capability in remote sensing areas, including LiDAR data processing. Traditional deep models empirically use a fixed neighborhood system as input to the network. Therefore, the weight and height of the input rectangle may not be optimal. In order to modify such handcrafted setting, a spatial transformation network is used here to identify optimal inputs. The transformed inputs are fed into a well-designed CNN to obtain the final classification results. Furthermore, morphological profiles are combined with spatial transformation CNN to further improve the classification accuracy. The proposed frameworks are tested on two LiDAR-DSMs (i.e., the Recology and Houston data sets). The experimental results show that the proposed models provide competitive results compared to the state-of-the-art methods. Furthermore, the proposed optimal input identification approach can also be found beneficial for other remote sensing applications.

Airborne and spaceborne hyperspectral imaging systems have advanced in recent years in terms of spectral and spatial resolution, which makes data sets produced by them a valuable source for land-cover classification. The availability of hyperspectral data with fine spatial resolution has revolutionized hyperspectral image classification techniques by taking advantage of both spectral and spatial information in a single classification framework. The ECHO (Extraction and Classification of Homogeneous Objects) classifier, which was proposed in 1976, might be the first spectral-spatial classification approach of its kind in the remote sensing community. Since then and especially in the latest years, increasing attention has been dedicated to developing sophisticated spectral-spatial classification methods. There is now a rich literature on this particular topic in the remote sensing community, composing of several fast-growing branches. In this paper, the latest advances in spectral-spatial classification of hyperspectral data are critically reviewed. More than 25 approaches based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning are addressed with an emphasis on discussing their methodological foundations. Examples of experimental results on three benchmark hyperspectral data sets, including both well-known long-used data and a recent data set resulting from an international contest, are also presented. Moreover, the utilized training and test sets for the aforementioned data sets as well as several codes and libraries are also shared online with the community.

A generative adversarial network (GAN) usually contains a generative network and a discriminative network in competition with each other. The GAN has shown its capability in a variety of applications. In this paper, the usefulness and effectiveness of GAN for classification of hyperspectral images (HSIs) are explored for the first time. In the proposed GAN, a convolutional neural network (CNN) is designed to discriminate the inputs and another CNN is used to generate so-called fake inputs. The aforementioned CNNs are trained together: the generative CNN tries to generate fake inputs that are as real as possible, and the discriminative CNN tries to classify the real and fake inputs. This kind of adversarial training improves the generalization capability of the discriminative CNN, which is really important when the training samples are limited. Specifically, we propose two schemes: 1) a well-designed 1D-GAN as a spectral classifier and 2) a robust 3D-GAN as a spectral–spatial classifier. Furthermore, the generated adversarial samples are used with real training samples to fine-tune the discriminative CNN, which improves the final classification performance. The proposed classifiers are carried out on three widely used hyperspectral data sets: Salinas, Indiana Pines, and Kennedy Space Center. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods. In addition, the proposed GANs open new opportunities in the remote sensing community for the challenging task of HSI classification and also reveal the huge potential of GAN-based methods for the analysis of such complex and inherently nonlinear data.

In recent years, deep learning-based methods, especially convolutional neural networks (CNNs), have shown their capabilities in remote sensing data processing. The efficacy of light detection and ranging (LiDAR) has been already proven in a wide variety of research areas. Most of the existing methods do not extract the informative features from LiDAR-derived rasterized digital surface models (LiDAR-DSM) data in a deep manner. In order to utilize the advantages of deep models for the classification of LiDAR-derived features, deep CNN is proposed here to hierarchically extract the robust and discriminant features of the input data. Moreover, morphological profiles and multiattribute profiles (MAPs) are investigated to enrich the inputs of the CNN and further to improve the ultimate classification performance. Furthermore, a new activation function, sigmoid-weighted linear units (SiLUs), is introduced. The proposed frameworks are tested on two LiDAR-DSMs (i.e., Bayview Park and Houston data sets). The MAP-CNNs with SiLU outperform original CNNs by 6.62% and 6.88% in terms of overall accuracy on Bayview Park and Houston data sets, respectively, when the number of training samples of each class is 40.

Hyperspectral remote sensing obtains abundant spectral and spatial information of the observed object simultaneously. It is an opportunity to classify hyperspectral imagery (HSI) with a fine-grained manner. In this study, the fine-grained classification of HSI, which contains a large number of classes, is investigated. On one hand, traditional classification methods cannot handle fine-grained classification of HSI well, on the other hand, deep learning methods have shown their powerfulness in fine-grained classification. So, in this paper, deep learning is explored for HSI supervised and semi-supervised fine-grained classification. For supervised HSI fine-grained classification, densely connected convolutional neural network (DenseNet) is explored for accurate classification. Moreover, DenseNet is combined with pre-processing technique (i.e., principal component analysis or auto-encoder) or post-processing technique (i.e., conditional random field) to further improve classification performance. For semi-supervised HSI fine-grained classification, a generative adversarial network (GAN), which includes a discriminative CNN and a generative CNN, is carefully designed. The GAN fully uses the labeled and unlabeled samples to improve classification accuracy. The proposed methods were tested on the Indian Pines data set, which contains 33,3951 samples with 52 classes. The experimental results show that the deep learning-based methods provide great improvements compared with other traditional methods, which demonstrate that deep models have huge potential for HSI fine-grained classification.

In this letter, a self-improving convolutional neural network (CNN) based method is proposed for the classification of hyperspectral data. This approach solves the so-called curse of dimensionality and the lack of available training samples by iteratively selecting the most informative bands suitable for the designed network via fractional order Darwinian particle swarm optimization. The selected bands are then fed to the classification system to produce the final classification map. Experimental results have been conducted with two well-known hyperspectral data sets: Indian Pines and Pavia University. Results indicate that the proposed approach significantly improves a CNN-based classification method in terms of classification accuracy. In addition, this letter uses the concept of dither for the first time in the remote sensing community to tackle overfitting.

Due to the advantages of deep learning, in this paper, a regularized deep feature extraction (FE) method is presented for hyperspectral image (HSI) classification using a convolutional neural network (CNN). The proposed approach employs several convolutional and pooling layers to extract deep features from HSIs, which are nonlinear, discriminant, and invariant. These features are useful for image classification and target detection. Furthermore, in order to address the common issue of imbalance between high dimensionality and limited availability of training samples for the classification of HSI, a few strategies such as L2 regularization and dropout are investigated to avoid overfitting in class data modeling. More importantly, we propose a 3-D CNN-based FE model with combined regularization to extract effective spectral-spatial features of hyperspectral imagery. Finally, in order to further improve the performance, a virtual sample enhanced method is proposed. The proposed approaches are carried out on three widely used hyperspectral data sets: Indian Pines, University of Pavia, and Kennedy Space Center. The obtained results reveal that the proposed models with sparse constraints provide competitive results to state-of-the-art methods. In addition, the proposed deep FE opens a new window for further research.

Publisher's version (útgefin grein), Capsule networks can be considered to be the next era of deep learning and have recently shown their advantages in supervised classification. Instead of using scalar values to represent features, the capsule networks use vectors to represent features, which enriches the feature presentation capability. This paper introduces a deep capsule network for hyperspectral image (HSI) classification to improve the performance of the conventional convolutional neural networks (CNNs). Furthermore, a modification of the capsule network named Conv-Capsule is proposed. Instead of using full connections, local connections and shared transform matrices, which are the core ideas of CNNs, are used in the Conv-Capsule network architecture. In Conv-Capsule, the number of trainable parameters is reduced compared to the original capsule, which potentially mitigates the overfitting issue when the number of available training samples is limited. Specifically, we propose two schemes: (1) A 1D deep capsule network is designed for spectral classification, as a combination of principal component analysis, CNN, and the Conv-Capsule network, and (2) a 3D deep capsule network is designed for spectral-spatial classification, as a combination of extended multi-attribute profiles, CNN, and the Conv-Capsule network. The proposed classifiers are tested on three widely-used hyperspectral data sets. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, including kernel support vector machines, CNNs, and recurrent neural network., This research was funded by Natural Science Foundation of China under the Grant 61771171.

Hyperspectral image (HSI) classification is a core task in the remote sensing community, and recently, deep learning-based methods have shown their capability of accurate classification of HSIs. Among the deep learning-based methods, deep convolutional neural networks (CNNs) have been widely used for the HSI classification. In order to obtain a good classification performance, substantial efforts are required to design a proper deep learning architecture. Furthermore, the manually designed architecture may not fit a specific data set very well. In this paper, the idea of automatic CNN for the HSI classification is proposed for the first time. First, a number of operations, including convolution, pooling, identity, and batch normalization, are selected. Then, a gradient descent-based search algorithm is used to effectively find the optimal deep architecture that is evaluated on the validation data set. After that, the best CNN architecture is selected as the model for the HSI classification. Specifically, the automatic 1-D Auto-CNN and 3-D Auto-CNN are used as spectral and spectral–spatial HSI classifiers, respectively. Furthermore, the cutout is introduced as a regularization technique for the HSI spectral–spatial classification to further improve the classification accuracy. The experiments on four widely used hyperspectral data sets (i.e., Salinas, Pavia University, Kennedy Space Center, and Indiana Pines) show that the automatically designed data-dependent CNNs obtain competitive classification accuracy compared with the state-of-the-art methods. In addition, the automatic design of the deep learning architecture opens a new window for future research, showing the huge potential of using neural architectures’ optimization capabilities for the accurate HSI classification.

Deep learning models, especially deep convolutional neural networks (CNNs), have been intensively investigated for hyperspectral image (HSI) classification due to their powerful feature extraction ability. In the same manner, ensemble-based learning systems have demonstrated high potential to effectively perform supervised classification. In order to boost the performance of deep learning-based HSI classification, the idea of deep learning ensemble framework is proposed here, which is loosely based on the integration of deep learning model and random subspace-based ensemble learning. Specifically, two deep learning ensemble-based classification methods (i.e., CNN ensemble and deep residual network ensemble) are proposed. CNNs or deep residual networks are used as individual classifiers and random subspaces contribute to diversify the ensemble system in a simple yet effective manner. Moreover, to further improve the classification accuracy, transfer learning is investigated in this study to transfer the learnt weights from one individual classifier to another (i.e., CNNs). This mechanism speeds up the learning stage. Experimental results with widely used hyperspectral datasets indicate that the proposed deep learning ensemble system provides competitive results compared with state-of-the-art methods in terms of classification accuracy. The combination of deep learning and ensemble learning provides a significant potential for reliable HSI classification.

Recovering hyperspectral image (HSI) from mixed noise degradation is a challenging and promising theme in remote sensing, particularly when stripes and deadlines exist in several contiguous bands. This paper proposes a HSI’s restoration method making use of adaptive morphological filtering (AMF) and fusing structure information of an auxiliary color image. An adaptive structuring element (ASE) indicating morphological features of each pixel is generated through information fusion, to simultaneously remove the mixed noise and preserve fine spatial structures. This key technology contains three main steps. First, edges are extracted from the auxiliary image exploiting its color information; then, an edge-constraint growing algorithm is used to generate the clustering kernel; finally, the ASE is obtained via goal-guided k-means clustering. The ASE has extensive application value, for it can be an enhancing module for most filters-based restoration methods, to mitigate the structural damage due to the fixed mask. Among these methods, Gaussian filter for preprocessing and majority voting for postprocessing are introduced in this paper as representatives. In addition, the auxiliary image can be both visible image of multisensor and false RGB component of the undamaged bands of the HSI, so it is relatively available. Experiments on simulated and real data sets show obvious effects on denoising and destriping both subjectively and objectively. The advantage of ASE on structure details preserving, compared to conventional approaches, is clearly demonstrated. The application value of the proposed restoration frame and ASE is further proved through the decision-level postprocessing experiments.

Hyperspectral data classification is a hot topic in remote sensing community. In recent years, significant effort has been focused on this issue. However, most of the methods extract the features of original data in a shallow manner. In this paper, we introduce a deep learning approach into hyperspectral image classification. A new feature extraction (FE) and image classification framework are proposed for hyperspectral data analysis based on deep belief network (DBN). First, we verify the eligibility of restricted Boltzmann machine (RBM) and DBN by the following spectral information-based classification. Then, we propose a novel deep architecture, which combines the spectral–spatial FE and classification together to get high classification accuracy. The framework is a hybrid of principal component analysis (PCA), hierarchical learning-based FE, and logistic regression (LR). Experimental results with hyperspectral data indicate that the classifier provide competitive solution with the state-of-the-art methods. In addition, this paper reveals that deep learning system has huge potential for hyperspectral data classification.

A novel object tracking algorithm based on hierarchical convolutional features was proposed in this paper. Firstly, the tracking algorithm uses the hierarchical networks of VGG-Net-19 to extract the hierarchical convolutional features of image, having a greater improvement than using only one layer to do that. Secondly, the algorithm obtains features by using correlation filtering method with weighted fusion, so as to determine the real position of the target according to the characteristics of different layers. The experimental results show that, compared with the current four popular object tracking algorithms, the proposed algorithm achieves better accuracy and success rate, and the results are consistent in OPE (one-pass evaluation), SRE (spatial robustness evaluation) and TRE (temporal robustness evaluation).

Recently, the capability of deep learning-based approaches, especially deep convolutional neural networks (CNNs), has been investigated for hyperspectral remote sensing feature extraction (FE) and classification. Due to the large number of learnable parameters in convolutional filters, lots of training samples are needed in deep CNNs to avoid the overfitting problem. On the other hand, Gabor filtering can effectively extract spatial information including edges and textures, which may reduce the FE burden of the CNNs. In this letter, in order to make the most of deep CNN and Gabor filtering, a new strategy, which combines Gabor filters with convolutional filters, is proposed for hyperspectral image classification to mitigate the problem of overfitting. The obtained results reveal that the proposed model provides competitive results in terms of classification accuracy, especially when only a limited number of training samples are available.

The multisensory fusion of remote sensing data has obtained a great attention in recent years. In this letter, we propose a new feature fusion framework based on deep neural networks (DNNs). The proposed framework employs deep convolutional neural networks (CNNs) to effectively extract features of multi-/hyperspectral and light detection and ranging data. Then, a fully connected DNN is designed to fuse the heterogeneous features obtained by the previous CNNs. Through the aforementioned deep networks, one can extract the discriminant and invariant features of remote sensing data, which are useful for further processing. At last, logistic regression is used to produce the final classification results. Dropout and batch normalization strategies are adopted in the deep fusion framework to further improve classification accuracy. The obtained results reveal that the proposed deep fusion model provides competitive results in terms of classification accuracy. Furthermore, the proposed deep learning idea opens a new window for future remote sensing data fusion.

Classification is one of the most popular topics in hyperspectral remote sensing. In the last two decades, a huge number of methods were proposed to deal with the hyperspectral data classification problem. However, most of them do not hierarchically extract deep features. In this paper, the concept of deep learning is introduced into hyperspectral data classification for the first time. First, we verify the eligibility of stacked autoencoders by following classical spectral information-based classification. Second, a new way of classifying with spatial-dominated information is proposed. We then propose a novel deep learning framework to merge the two features, from which we can get the highest classification accuracy. The framework is a hybrid of principle component analysis (PCA), deep learning architecture, and logistic regression. Specifically, as a deep learning architecture, stacked autoencoders are aimed to get useful high-level features. Experimental results with widely-used hyperspectral data indicate that classifiers built in this deep learning-based framework provide competitive performance. In addition, the proposed joint spectral-spatial deep neural network opens a new window for future research, showcasing the deep learning-based methods' huge potential for accurate hyperspectral data classification.

In hyperspectral remote sensing image classification, ensemble systems with support vector machine (SVM), such as the Random Subspace SVM Ensemble (RSSE), have significantly outperformed single SVM on the robustness and overall accuracy. In this paper, we introduce a novel subspace mechanism, the Optimizing Subspace SVM Ensemble (OSSE), to improve RSSE by selecting discriminating subspaces for individual SVMs. The framework is based on Genetic Algorithm (GA), adopting the Jeffries-Matusita (JM) distance as a criterion, to optimize the selected subspaces. The combination of optimizing subspaces is more suitable for classification than the random one, at the same time having the ability to accommodate requisite diversity within the ensemble. The modifications have improved the accuracies of individual classifiers; as a result, better overall accuracies are present. Experiments on the classification of two hyperspectral datasets reveal that our proposed OSSE obtains sound performances compared with RSSE, single SVM, and other ensemble with GA to optimize SVM.

Existing deep convolutional neural networks (CNNs) have shown their great success on image classification. CNNs mainly consist of convolutional and pooling layers, both of which are performed on local image areas without considering the dependencies among different image regions. However, such dependencies are very important for generating explicit image representation. In contrast, recurrent neural networks (RNNs) are well known for their ability of encoding contextual information among sequential data, and they only require a limited number of network parameters. General RNNs can hardly be directly applied on non-sequential data. Thus, we proposed the hierarchical RNNs (HRNNs). In HRNNs, each RNN layer focuses on modeling spatial dependencies among image regions from the same scale but different locations. While the cross RNN scale connections target on modeling scale dependencies among regions from the same location but different scales. Specifically, we propose two recurrent neural network models: 1) hierarchical simple recurrent network (HSRN), which is fast and has low computational cost; and 2) hierarchical long-short term memory recurrent network (HLSTM), which performs better than HSRN with the price of more computational cost. In this manuscript, we integrate CNNs with HRNNs, and develop end-to-end convolutional hierarchical recurrent neural networks (C-HRNNs). C-HRNNs not only make use of the representation power of CNNs, but also efficiently encodes spatial and scale dependencies among different image regions. On four of the most challenging object/scene image classification benchmarks, our C-HRNNs achieve state-of-the-art results on Places 205, SUN 397, MIT indoor, and competitive results on ILSVRC 2012.

Hyperspectral image classification has been a vibrant area of research in recent years. Given a set of observations, i.e., pixel vectors in a hyperspectral image, classification approaches try to allocate a unique label to each pixel vector. However, the classification of hyperspectral images is a challenging task for a number of reasons, such as the presence of redundant features, the imbalance among the limited number of available training samples, and the high dimensionality of the data.

Recently, the fusion of hyperspectral and light detection and ranging (LiDAR) data has obtained a great attention in the remote sensing community. In this paper, we propose a new feature fusion framework using deep neural network (DNN). The proposed framework employs a novel 3D convolutional neural network (CNN) to extract the spectral-spatial features of hyperspectral data, a deep 2D CNN to extract the elevation features of LiDAR data, and then a fully connected deep neural network to fuse the extracted features in the previous CNNs. Through the aforementioned three deep networks, one can extract the discriminant and invariant features of hyperspectral and LiDAR data. At last, logistic regression is used to produce the final classification results. The experimental results reveal that the proposed deep fusion model provides competitive results. Furthermore, the proposed deep fusion idea opens a new window for future research.

Hyperspectral images (HSIs) are often contaminated by noise, in order to remove the image noise efficiently and acquire excellent results. We propose a new denoising method based on 3D sparse coding. Firstly, to make full use of spectral information of hyperspectral data, we extract patches from HSIs and each patch contains the same area of different band. Secondly, we use aforementioned method to extract all patches and train these patches, the dictionary can be obtained, further calculate sparse coefficients. Finally, we can restore the HISs through the dictionary and the sparse coefficients. Experiments are implemented using the HSIs collected by AVIRIS and ROSIS. Results indicate that compared with common 2D sparse coding method, 3D sparse method can effectively improve the restoration performance for both subjective visual and objective evaluation criterion.

In existing convolutional neural networks (CNNs), both convolution and pooling are locally performed for image regions separately, no contextual dependencies between different image regions have been taken into consideration. Such dependencies represent useful spatial structure information in images. Whereas recurrent neural networks (RNNs) are designed for learning contextual dependencies among sequential data by using the recurrent (feedback) connections. In this work, we propose the convolutional recurrent neural network (C-RNN), which learns the spatial dependencies between image regions to enhance the discriminative power of image representation. The C-RNN is trained in an end-to-end manner from raw pixel images. CNN layers are firstly processed to generate middle level features. RNN layer is then learned to encode spatial dependencies. The C-RNN can learn better image representation, especially for images with obvious spatial contextual dependencies. Our method achieves competitive performance on ILSVRC 2012, SUN 397, and MIT indoor.

In recent years, the specialized destriping methods for remote-sensing image emphasize efficiency and flexibility but lose the particulars in wide stripes. Though many excellent inpainting algorithms have been proposed, most of them are too complex and inefficient for hyperspectral image (HSI). In this paper, we propose a novel automatic completing method which is appropriate for the feature of HSI and easy to be implemented in parallel. The boundary information in broken region is restored primarily based on SVM with the input of the surrounding detected edges. Then we introduce an adaptive morphological filling operator restrained by the boundaries to estimate the pixels in the same position of all bands synchronously. Experimental results show satisfactory performance of the proposed system and its significance on HSI completion.

In hyperspectral images, there is abundant spectral information for classification. In most cases, spectral information based methods yield good classification results. However, different objects may have similar spectrums due to similar physical property. Spectral information based classification methods can't give accurate results for the similar physical property among objects belonging to the same main category. On the other hand, as the resolutions of sensors increase in recent years, more spatial information, such as shape and texture information can be extracted and described more precisely. In this paper, we proposed a spatial information aided similar spectral classification. In our method, spatial features, including the pixel shape index (PSI), and the gray-level co-occurrence matrix (GLCM), are extracted for accurate classification. We compared the Bhattachary Distance before and after spatial information being aided and we found that the B Distance was amplified sharply, which indicated that the separability among classes increased. Classification is practiced on two hyperspectral data sets, Kennedy Space Center and Pavia City, and the proposed method is compared with classification based on different features. It is found that each feature makes contribution to classification and the accuracy of the proposed method is the highest among all methods, which certifies the effectiveness of our algorithm.

The paper presents a novel ensemble system which unites Adaboost with multifeature to increase diversity among individual classifiers. Adaboost gives rise to convenience for hyperspectral data classification. To improve the method further, we propose joint Adaboost and multifeature based ensemble (JAME), which assigns different multifeature sets to individual classifiers in Adaboost. Diverse spectral and spatial feature sets are integrated to form multifeature sets. As a result, compared with Adaboost the method has increased the diversity of ensemble system, and better overall accuracies are present. Experiments on hyperspectral data sets reveal that the proposed JAME obtains sound performances comparing with original Adaboost and single classifier.

Hyperspectral image (HSI) classification is a hot topic in the remote sensing community. This paper proposes a new framework of spectral-spatial feature extraction for HSI classification, in which for the first time the concept of deep learning is introduced. Specifically, the model of autoencoder is exploited in our framework to extract various kinds of features. First we verify the eligibility of autoencoder by following classical spectral information based classification and use autoencoders with different depth to classify hyperspectral image. Further in the proposed framework, we combine PCA on spectral dimension and autoencoder on the other two spatial dimensions to extract spectral-spatial information for classification. The experimental results show that this framework achieves the highest classification accuracy among all methods, and outperforms classical classifiers such as SVM and PCA-based SVM., Comment: Accepted as a conference paper at ICICS 2013, an updated version. Codes published. 9 pages, 6 figures

The nonlinear characteristics in hyperspectral data is considered as an influential factor curtailing the classification accuracy. To deal with the problem, a new method for classification is developed, especially for hyperspectral imagery (HSI). It is a supervised method based on Locally Linear Embedding (LLE) and k-Nearest Neighbor (KNN), named with KNN based supervised LLE (S-LLE KNN). We use two real HIS dataset of AVIRIS in experiment section and compare overall classification accuracy and accuracy of each class in different methods, the results shows that the supervised nonlinear feature extraction method contributes more to classification accuracies methods.

Semisupervised feature selection methods can improve classification performance and enhance model comprehensibility with few labeled objects. However, most of the existing methods require graph construction beforehand, and the resulting heavy computational cost may bring about the failure to accurately capture the local geometry of data. To overcome the problem, adaptive semisupervised feature selection (ASFS) is proposed. In ASFS, the goodness of each feature is measured by linear objective functions based on loss functions and probability distribution matrices. By alternatively optimizing model parameters and automatically adjusting the probabilities of boundary objects, ASFS can measure the genuine characteristics of the data and then rank and select features. The experimental results attest to the effectiveness and practicality of the method in comparison with the latest and state-of-the-art methods on a Worldview II image and a Quickbird II image.

Achieving high target recognition accuracy is a pursuing and challenging issue for hyperspectral data analysis. The complicated imaging environment and noise interference lead to heterogeneous spectra within the homogeneous object, which makes the current spectral target recognition methods be lack of robustness. In this paper, a robust spectral target recognition method is proposed based on the combined spectral signatures, in which two key techniques are concerned. One is support vector data description (SVDD), which can tolerate the spectral variations of different pixels in the same object. Another is an effective spectral signature combined of spectral reflectance and spectral derivative, which can be robust to data characteristics with different spectral-amplitude variation. The proposed method outperforms the classical methods with only spectral reflective information in term of target recognition accuracy and robustness.

Vector Quantization (VQ) is widely used for Hyper Spectral Image (HSI) compression and VQ based algorithms yield good results for reducing the amount of the data. However, the VQ based algorithms have the shortcoming of computing expensive. Many fast VQ algorithms have been proposed to reduce the computing complexity, while the algorithms consider the HSI feature rarely. We present a new framework of fast vector quantization for HSI compression, which uses the spectra characteristics of HSI adequately. The breakthrough codebook training method is calculated at the HSI feature domain, which is a low dimension structure without losing significant information, to get much lower complexity. The experimental results demonstrate that the proposed algorithms can reduce the computing time dramatically while keep the comparable reconstruction fidelity.

Hyperspectral images (HSI) regularly contain hundreds of bands, which are of different importance in the application. Most HSI compression methods usually deal with most bands in the same way, and they do not take the difference of different bands into consideration, which may cause the loss of important spectral information. In order to preserve the spectral information of interest for applications, a new band-of-interest (BOI)-preserving-based HSI compression method is proposed. The conception of BOI is proposed because some bands are significant in the specific applications, and BOI selection methods are chosen according to application requirements. BOI selection is first performed according to application measurements. Then, BOI information is fed into recursive bidirection prediction (RBP) and set partition in hierarchical trees (SPIHT) compression scheme which uses RBP for spectral decorrelation followed by SPIHT algorithm for coding the resulting decorrelated residual images. More bits are allocated to BOI to preserve BOI by two approaches, respectively. Compress BOI and non-BOI bands directly with low distortion and high distortion, respectively, and compress all bands with low distortion and perform a postcompression truncation. Experiments are implemented with different settings using AVIRIS images. Results indicate that the proposed two methods both can achieve excellent compression efficiency and reconstructed quality. In addition, they can improve the application effect in both material classification and target recognition. Compared with non-BOI compression algorithm, at the compression ratio of 80, the proposed methods improve the classification accuracy by 2% and target recognition accuracy by 9%.

In this paper, we propose an algorithm for reference bands selection in hyperspectral image (HSI) compression. In HSI compression, many algorithms need to select the reference bands, but all those algorithms uniformly select the reference bands. The uniform selection method isn't optimal, so we propose a genetic algorithm (GA) based reference band selection method to get lower prediction residual. The proposed method has the advantage of being simple, yielding compression fidelity better than the classical methods.