Search

Recently, many deep convolutional neural network (DCNN)-based methods have been proposed for remote sensing (RS) image scene classification (SC). In general, DCNNs obtain good generalization capabilities under the condition of correct labels. Unfortunately, the given samples are sometimes mislabeled. In this letter, the classification of RS images with noisy labels is investigated. First, complementary learning (CL), which learns from complementary labels rather than the original labels, is introduced for RS image classification with noisy labels. CL can decrease the probability of learning from incorrect information, and therefore, it is robust to noisy labels. Then, soft CL, which randomly disturbs the complementary labels of the training samples, is proposed to prevent the overfitting issue in training a DCNN. Moreover, an RS image scene classification framework combining ordinary learning (OL) and CL (RS-COCL) is proposed, which uses CL to obtain a good model and OL to fine-tune the deep model. Additionally, noisy labels filtering is used in RS-COCL (RS-COCL-NLF) to detected and corrected noisy samples. At last, soft CL is used in RS-COCL-NLF to obtain better classification performance. The proposed methods are tested on two widely used datasets (i.e., Northwestern Polytechnical University (NWPU)-RESISC45 and PatternNet) and the obtained results show that the proposed methods provide competitive classification accuracy compared to the state-of-the-art methods.

One of the challenges in hyperspectral image (HSI) classification is that there are limited labeled samples to train a classifier for very high-dimensional data. In practical applications, we often encounter an HSI domain (called target domain) with very few labeled data, while another HSI domain (called source domain) may have enough labeled data. Classes between the two domains may not be the same. This article attempts to use source class data to help classify the target classes, including the same and new unseen classes. To address this classification paradigm, a meta-learning paradigm for few-shot learning (FSL) is usually adopted. However, existing FSL methods do not account for domain shift between source and target domain. To solve the FSL problem under domain shift, a novel deep cross-domain few-shot learning (DCFSL) method is proposed. For the first time, DCFSL tackles FSL and domain adaptation issues in a unified framework. Specifically, a conditional adversarial domain adaptation strategy is utilized to overcome domain shift, which can achieve domain distribution alignment. In addition, FSL is executed in source and target classes at the same time, which can not only discover transferable knowledge in the source classes but also learn a discriminative embedding model to the target classes. Experiments conducted on four public HSI data sets demonstrate that DCFSL outperforms the existing FSL methods and deep learning methods for HSI classification. Our source code is available at https://github.com/Li-ZK/DCFSL-2021.

Due to powerful feature extraction capability, convolutional neural networks (CNNs) have been widely used for hyperspectral image (HSI) classification. However, because of a large number of parameters that need to be trained, sufficient training samples are usually required for deep CNN-based methods. Unfortunately, limited training samples are a common issue in the remote sensing community. In this study, a dual graph convolutional network (DGCN) is proposed for the supervised classification of HSI with limited training samples. The first GCN fully extracts features existing in and among HSI samples, while the second GCN utilizes label distribution learning, and thus, it potentially reduces the number of required training samples. The two GCNs are integrated through several iterations to decrease interclass distances, which leads to a more accurate classification step. Moreover, a new idea entitled multiscale feature cutout is proposed as a regularization technique for HSI classification (DGCN-M). Different from the regularization methods (e.g., dropout and DropBlock), the proposed multiscale feature cutout could randomly mask out multiscale region sizes in a feature map, which further reduces the overfitting problem and yields consistent improvement. Experimental results on the four popular hyperspectral data sets (i.e., Salinas, Indian Pines, Pavia, and Houston) indicate that the proposed method obtains good classification performance compared to state-of-the-art methods, which shows the potential of GCN for HSI classification.

Deep learning has achieved great success in hyperspectral image classification. However, its success relies on the availability of sufficient training samples. Unfortunately, the collection of training samples is expensive, time-consuming, and even impossible in some cases. Natural image data sets that are different from HSI, such as Image Net and mini-ImageNet, have abundant texture and structure information. Effective knowledge transfer between two heterogeneous data sets can significantly improve the accuracy of HSI classification. In this letter, heterogeneous few-shot learning (HFSL) for HSI classification is proposed with only a few labeled samples per class. Firstly, few-shot learning is performed on the mini-ImageNet data sets to learn the transferable knowledge. Then, to make full use of the spatial and spectral information, a spectral-spatial fusion network is devised. Spectral information is obtained by the residual network with pure one-dimensional operators. Spatial information is extracted by a convolution network with pure two-dimensional operators, and the weights of the spatial network are initialized by the weights of the model trained on the mini-ImageNet data sets. Finally, few-shot learning is fine-tuned on HSI to extract discriminative spectral-spatial features and individual knowledge, which can improve the classification performance of the new classification task. Experiments conducted on two public HSI data sets demonstrate that the HFSL outperforms the existing few-shot learning methods and supervised learning methods for HSI classification with only a few labeled samples. Our source code is available at https://github.com/Li-ZK/HFSL.

The lack of training samples remains one of the major obstacles in applying convolutional neural networks (CNNs) to the hyperspectral image (HSI) classification. In this letter, the accurate classification of HSI with limited training samples is investigated. Due to the advantages of minimizing the distance between samples in the same class and maximizing the distance between samples in different classes, siamese CNN is used for HSI classification with limited training samples. After that, to improve the classification performance, data augmentation is investigated for siamese CNN-based HSI classification. Specifically, pair data augmentation based on CutMix is proposed to generate the training pairs of the same or different classes and the new generated training pairs are used to train siamese CNN. Traditional data augmentation methods simply generate new training samples. It is not proper when data augmentation methods keep the loss function and change the content of the input at the same time. Therefore, a soft-loss-based siamese CNN, which changes its loss according to the coupled replacement data augmentation, is proposed to further address the HSI classification with limited training samples. In the experimental part, on two widely used hyperspectral datasets, the influences of different training samples and window sizes are discussed, and the experimental results reveal that the proposed soft-augmentation-based siamese CNN provides competitive results with limited training samples compared with state-of-the-art methods.

Recently, convolutional neural networks (CNNs) have been widely used for light detection and ranging (LiDAR) data classification. Although CNNs achieve good classification performance for LiDAR data classification, a lot of efforts are needed to design a proper architecture. In this letter, the automatic modularized design of CNN is explored for LiDAR data classification for the first time. First, a searchable architecture containing convolution and pooling operations is used to establish the search space. Then, the optimal building block (i.e., cell), which is the basic part of a deep CNN, is obtained from search space by a gradient decent-based algorithm. At last, by stacking several optimal building blocks, a deep CNN can be formulated for LiDAR data classification. Moreover, in order to mitigate the overfitting problem in training a CNN, improved label smoothing and feature regularization are proposed to further improve the classification performance of LiDAR data. The proposed classification models are evaluated on two popular LiDAR data sets (i.e., the Bayview Park and Houston data sets). The experimental results show that the proposed models provide the competitive results compared to the state-of-the-art methods.

In recent years, deep convolutional neural networks (CNNs) have been widely investigated for hyperspectral image (HSI) classification. The CNN-based HSI classifiers obtained good performance under the condition of sufficient training samples. In order to address the problem of limited training samples, in this letter, transfer learning is combined with CNN to address the issue of HSI classification. Pretrained models on large-scale data sets (e.g., ImageNet) can extract the general and discriminative features. Due to the fact that the extracted low-level and mid-level features can be reused for the HSI feature extraction, the CNN-based methods usually obtain good classification performance with insufficient training samples. The ImageNet data set has three channels, while the HSI data set contains hundreds of channels. Therefore, three channels of the HSI are randomly selected to formulate a transferring CNN. Then, several transferring CNNs are combined to establish an ensemble classification system with diversity. Moreover, an improved label smoothing technique is proposed to further improve the classification accuracy of the HSI. Experimental results on two popular hyperspectral data sets [i.e., Indian Pines and Kennedy Space Center (KSC)] show that the transferring CNN ensemble obtains good classification performance compared to the state-of-the-art methods.

In recent years, deep convolutional neural networks (CNNs) have been widely used for hyperspectral image (HSI) classification. The powerful feature extraction capability and high classification performance of CNN highly depend on sufficient training samples. Unfortunately, it is not a common situation because collecting training samples is time-consuming and expensive. In this letter, in order to make the most of deep CNN with limited training samples, dual-path siamese CNN (Dual-SCNN) is proposed for HSI classification. Specifically, the proposed classification framework is a combination of extended morphological profiles, CNN, siamese network, and spectral–spatial feature fusion. In order to solve the problem of insufficiency in hard negative pairs during the training of a siamese network, adversarial training is combined with Dual-SCNN (Dual-SCNN-AT) for HSI classification. Moreover, a data augmentation method titled mixup is combined with Dual-SCNN and Dual-SCNN-AT to further improve the classification performance of HSI. The obtained results on widely used hyperspectral data sets reveal that the proposed methods provide the competitive results in terms of classification accuracy, especially with limited training samples.

Deep convolutional neural networks (CNNs) have shown their outstanding performance in the hyperspectral image (HSI) classification. The success of CNN-based HSI classification relies on the availability sufficient training samples. However, the collection of training samples is expensive and time consuming. Besides, there are many pretrained models on large-scale data sets, which extract the general and discriminative features. The proper reusage of low-level and midlevel representations will significantly improve the HSI classification accuracy. The large-scale ImageNet data set has three channels, but HSI contains hundreds of channels. Therefore, there are several difficulties to simply adapt the pretrained models for the classification of HSIs. In this article, heterogeneous transfer learning for HSI classification is proposed. First, a mapping layer is used to handle the issue of having different numbers of channels. Then, the model architectures and weights of the CNN trained on the ImageNet data sets are used to initialize the model and weights of the HSI classification network. Finally, a well-designed neural network is used to perform the HSI classification task. Furthermore, attention mechanism is used to adjust the feature maps due to the difference between the heterogeneous data sets. Moreover, controlled random sampling is used as another training sample selection method to test the effectiveness of the proposed methods. Experimental results on four popular hyperspectral data sets with two training sample selection strategies show that the transferred CNN obtains better classification accuracy than that of state-of-the-art methods. In addition, the idea of heterogeneous transfer learning may open a new window for further research.

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link. In recent years, the state-of-the-art video coding standard - Versatile Video Coding (VVC) has been widely investigated. VVC achieves impressive performance by adopting more flexible partitioning method compared to its predecessor High Efficiency Video Coding (HEVC). However, the superior performance is realized at the expense of huge time consumption and increasing hardware costs, which obstructs its applications in real-time scenarios. To ad- dress this problem, we present a fast implementation for the decision process of the nested multi-type tree (QTMT) partitioning, and it significantly reduces the run-time of encoder while maintaining almost the same coding performance. Firstly, the inherent texture property of source frame is utilized to identify the prediction depth for Coding Tree Unit (CTU). Then, the spatial correlation is used to further narrow the depth range down. Finally, we skip unnecessary par- tition types according to the predicted Coding Unit (CU) depth, which is deter- mined by the above predicted CTU depth and adjacent CU’s depth together. Experimental results demonstrate the effectiveness of our proposed method in VVC Test Model (VTM). Compared with the original implementation of the VTM4.0 anchor, the proposed algorithm achieves an average of 49.01% encoding time savings, accompanied by only an increase of 2.18% in Bj ntegaard delta Bitrate (BDBR) and a loss of 0.138dB in Bjontegaard delta PSNR (BDPSNR).

Recently, many convolutional neural network (CNN)-based methods have been proposed to tackle the classification task of hyperspectral images (HSI). In fact, CNN has become the de-facto standard for HSI classification. It seems that the traditional neural networks such as multi-layer perceptron (MLP) are not competitive for HSI classification. However, in this study, we try to prove that the MLP can achieve good classification performance of HSI if it is properly designed and improved. The proposed Modified-MLP for HSI classification contains two special parts: spectral–spatial feature mapping and spectral–spatial information mixing. Specifically, for spectral–spatial feature mapping, each input sample of HSI is divided into a sequence of 3D patches with fixed length and then a linear layer is used to map the 3D patches to spectral–spatial features. For spectral–spatial information mixing, all the spectral–spatial features within a single sample are feed into the solely MLP architecture to model the spectral–spatial information across patches for following HSI classification. Furthermore, to obtain the abundant spectral–spatial information with different scales, Multiscale-MLP is proposed to aggregate neighboring patches with multiscale shapes for acquiring abundant spectral–spatial information. In addition, the Soft-MLP is proposed to further enhance the classification performance by applying soft split operation, which flexibly capture the global relations of patches at different positions in the input HSI sample. Finally, label smoothing is introduced to mitigate the overfitting problem in the Soft-MLP (Soft-MLP-L), which greatly improves the classification performance of MLP-based method. The proposed Modified-MLP, Multiscale-MLP, Soft-MLP, and Soft-MLP-L are tested on the three widely used hyperspectral datasets. The proposed Soft-MLP-L leads to the highest OA, which outperforms CNN by 5.76%, 2.55%, and 2.5% on the Salinas, Pavia, and Indian Pines datasets, respectively. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, which shows that the MLP-based methods are still competitive for HSI classification.

The accurate classification of radar jamming signal is a core step of anti-jamming. Recently, convolutional neural network (CNN) based methods have shown their powerfulness in signal processing. In this paper, a deep fusion method based on CNN is proposed to classify jamming signal acting on pulse compression radar. The proposed method consists of three subnetworks (i.e., 1D-CNN, 2D-CNN, and fusion network). 1D-CNN is used to extract deep features of original radar jamming signal. Meanwhile, in order to extract the time-frequency features, short time Fourier transform (STFT) is applied to jamming signal to obtain time-frequency spectrograms. Then, 2D-CNN is used to extract deep time-frequency features, which are useful for further features fusion processing. Fusion network is used to deeply fuse the extracted features of the aforementioned CNNs and softmax is used to finish the task of radar jamming signal classification. In addition, in order to alleviate the problem of overfitting and improve the generalization ability of proposed model, soft label smoothing is proposed. The experimental results reveal that the proposed method provides competitive results in term of classification accuracy.

Deep learning-based methods, especially deep convolutional neural networks (CNNs), have shown their effectiveness for hyperspectral image (HSI) classification. In previous deep CNN-based HSI classification methods, a cuboid is empirically determined as the input. The dimensionalities of the cuboid, including height and weight, are crucial to the final classification results. Unfortunately, these superparameters (i.e., the dimensionalities of input cube) are hand-crafted, which means the inputs of a classifier are not optimized according to the specific hyperspectral dataset. In this letter, spatial transformation network (STN) is explored to obtain the optimal input for CNN-based HSI classification for the first time. STN is used to translate, rotate, and scale the original input to obtain optimized input for the following CNN. Moreover, in order to mitigate the overfitting problem in CNN-based HSI classification, DropBlock is introduced as a regularization technique for HSI accurate classification. Compared with dropout, which is a popular regularization technique, DropBlock obtains better classification accuracy. The proposed methods are tested on two widely used hyperspectral data sets (i.e., Salinas and Kennedy Space Center). The obtained experimental results show that the proposed methods provide competitive results compared with state-of-the-art methods including deep CNN-based methods.

Deep learning-based methods, especially deep convolutional neural network (CNN), have proven their powerfulness in hyperspectral image (HSI) classification. On the other hand, ensemble learning is ...

Successive approximation register (SAR) analogue-to-digital converter (ADC) is a good choice for low-power applications owing to its high energy-efficiency. However, in the biomedical applications which require ultra-low power consumption, the existing switching schemes are still not suitable. A highly energy-efficient, area-efficient switching scheme for SAR ADC is proposed with a novel hybrid switching technique. The novel hybrid switching technique makes use of the floating capacitor, split capacitor and C-2C dummy capacitor techniques. As a result, the proposed switching scheme can achieve high energy saving and area saving. Simulation results show that this proposed scheme lifts the efficiency of switching energy by 99.3% and reduces the number of capacitors by 87% compared with the conventional switching scheme. Besides, this scheme has no reset energy consumption. In addition, the proposed scheme has better performance in linearity of SAR ADC.

An ultra-low power consumption two-stage mixed switching scheme for successive approximation register (SAR) analog-to-digital converter (ADC) is presented. Using two simple switches, the novel switching scheme divides the capacitor arrays into two sub-arrays: stage-one and stage-two arrays. The two sub-arrays convert high and low bits cycles, respectively. Once the high bits conversion cycles are completed, the corresponding sub-arrays are split off from the capacitor arrays. This decreases the number of capacitors used in the rest conversion procedure. Thus, the proposed method improves the energy efficiency of SAR ADC. Thanks to C–2C dummy capacitors and two-stage capacitor arrays, the novel architecture achieves 86% reduction in capacitor area than conventional SAR ADC. Furthermore, based on the charge sharing technique and monotonic switching method, the proposed switching scheme does not consume reset energy and achieves 99.8% less switching energy than the conventional switching method. In addition, the proposed scheme is less sensitive to capacitor mismatch because of its great performance in linearity.

In recent years, deep convolutional neural networks (CNNs) have been widely used for hyperspectral image (HSI) classification. Besides, ensemble learning is a useful way to enhance the classification performance. Therefore, in this study, a new method titled Boosting-CNN is proposed for HSI classification, which fully explored the advantages of deep CNN and ensemble learning. Specifically, several deep CNNs are well-designed to classify HSI. The samples are misclassified in a CNN have more weighs in the following CNN through adaptive boosting. The final classification result is obtained by weighted voting of several CNNs. For HSI classification issue, the number of samples in different classes varies greatly, however, traditional classification methods cannot handle this issue well. In order to address imbalance training samples in HSI classification, soft class balanced loss is proposed to mitigate the influence of imbalance training samples. Experimental results on two popular hyperspectral datasets (i.e., Salinas and Pavia University) show that the proposed method obtain better classification accuracy compared to comparison methods.

Jamming is a big threat to the survival of a radar system. Therefore, the recognition of radar jamming signal type is a part of radar countermeasure. Recently, convolutional neural networks (CNNs) have shown their effectiveness in radar signal processing, including jamming signal recognition. However, most of existing CNN methods do not regard radar jamming as a complex value signal. In this study, a complex-valued CNN (CV-CNN) is investigated to fully explore the inherent characteristics of a radar jamming signal, and we find that we can obtain better recognition accuracy using this method compared with a real-valued CNN (RV-CNN). CV-CNNs contain more parameters, which need more inference time. To reduce the parameter redundancy and speed up the recognition time, a fast CV-CNN (F-CV-CNN), which is based on pruning, is proposed for radar jamming signal fast recognition. The experimental results show that the CV-CNN and F-CV-CNN methods obtain good recognition performance in terms of accuracy and speed. The proposed methods open a new window for future research, which shows a huge potential of CV-CNN-based methods for radar signal processing.

In recent years, many convolutional neural network (CNN)-based methods have been proposed to address the scene classification tasks of remote sensing images. Since the number of training samples in RS datasets is generally small, data augmentation is often used to expand the training set. It is, however, not appropriate when original data augmentation methods keep the label and change the content of the image at the same time. In this study, label augmentation (LA) is presented to fully utilize the training set by assigning a joint label to each generated image, which considers the label and data augmentation at the same time. Moreover, the output of images obtained by different data augmentation is aggregated in the test process. However, the augmented samples increase the intra-class diversity of the training set, which is a challenge to complete the following classification process. To address the above issue and further improve classification accuracy, Kullback–Leibler divergence (KL) is used to constrain the output distribution of two training samples with the same scene category to generate a consistent output distribution. Extensive experiments were conducted on widely-used UCM, AID and NWPU datasets. The proposed method can surpass the other state-of-the-art methods in terms of classification accuracy. For example, on the challenging NWPU dataset, competitive overall accuracy (i.e., 91.05%) is obtained with a 10% training ratio.

Recently, a great many deep convolutional neural network (CNN)-based methods have been proposed for hyperspectral image (HSI) classification. Although the proposed CNN-based methods have the advantages of spatial feature extraction, they are difficult to handle the sequential data with and CNNs are not good at modeling the long-range dependencies. However, the spectra of HSI are a kind of sequential data, and HSI usually contains hundreds of bands. Therefore, it is difficult for CNNs to handle HSI processing well. On the other hand, the Transformer model, which is based on an attention mechanism, has proved its advantages in processing sequential data. To address the issue of capturing relationships of sequential spectra in HSI in a long distance, in this study, Transformer is investigated for HSI classification. Specifically, in this study, a new classification framework titled spatial-spectral Transformer (SST) is proposed for HSI classification. In the proposed SST, a well-designed CNN is used to extract the spatial features, and a modified Transformer (a Transformer with dense connection, i.e., DenseTransformer) is proposed to capture sequential spectra relationships, and multilayer perceptron is used to finish the final classification task. Furthermore, dynamic feature augmentation, which aims to alleviate the overfitting problem and therefore generalize the model well, is proposed and added to the SST (SST-FA). In addition, to address the issue of limited training samples in HSI classification, transfer learning is combined with SST, and another classification framework titled transferring-SST (T-SST) is proposed. At last, to mitigate the overfitting problem and improve the classification accuracy, label smoothing is introduced for the T-SST-based classification framework (T-SST-L). The proposed SST, SST-FA, T-SST, and T-SST-L are tested on three widely used hyperspectral datasets. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, which shows that the concept of Transformer opens a new window for HSI classification.

This letter presents a study of vehicle detection in high-resolution images using superpixel segmentation and iterative convolutional neural network strategy. First, a novel superpixel segmentation integrated with multiple local information constraints method is proposed to improve the segmentation results with a low breakage rate. To make training and detection more efficient, we extract meaningful and nonredundant patches based on the centers of the segmented superpixels. For reducing the instability in detection performance because of manual or random selection of samples, a training sample iterative selection strategy based on convolutional neural network is proposed. After a compact training sample subset is obtained from the original entire training set, a representative feature set with high discrimination ability between vehicle and background is extracted from these selected samples for detection. To further avoid overfitting the training and promote the detection efficiency, data augment and a main direction estimation method are used. Comparative experimental results on Toronto data indicated the effectiveness of our proposed method.

Light detection and ranging (LiDAR) is a useful data acquisition technique, which is widely used in a variety of practical applications. The classification of LiDAR-derived rasterized digital surface model (LiDAR-DSM) is a fundamental technique in LiDAR data processing. In recent years, deep learning methods, especially convolutional neural networks (CNNs), have shown their capability in remote sensing areas, including LiDAR data processing. Traditional deep models empirically use a fixed neighborhood system as input to the network. Therefore, the weight and height of the input rectangle may not be optimal. In order to modify such handcrafted setting, a spatial transformation network is used here to identify optimal inputs. The transformed inputs are fed into a well-designed CNN to obtain the final classification results. Furthermore, morphological profiles are combined with spatial transformation CNN to further improve the classification accuracy. The proposed frameworks are tested on two LiDAR-DSMs (i.e., the Recology and Houston data sets). The experimental results show that the proposed models provide competitive results compared to the state-of-the-art methods. Furthermore, the proposed optimal input identification approach can also be found beneficial for other remote sensing applications.

Airborne and spaceborne hyperspectral imaging systems have advanced in recent years in terms of spectral and spatial resolution, which makes data sets produced by them a valuable source for land-cover classification. The availability of hyperspectral data with fine spatial resolution has revolutionized hyperspectral image classification techniques by taking advantage of both spectral and spatial information in a single classification framework. The ECHO (Extraction and Classification of Homogeneous Objects) classifier, which was proposed in 1976, might be the first spectral-spatial classification approach of its kind in the remote sensing community. Since then and especially in the latest years, increasing attention has been dedicated to developing sophisticated spectral-spatial classification methods. There is now a rich literature on this particular topic in the remote sensing community, composing of several fast-growing branches. In this paper, the latest advances in spectral-spatial classification of hyperspectral data are critically reviewed. More than 25 approaches based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning are addressed with an emphasis on discussing their methodological foundations. Examples of experimental results on three benchmark hyperspectral data sets, including both well-known long-used data and a recent data set resulting from an international contest, are also presented. Moreover, the utilized training and test sets for the aforementioned data sets as well as several codes and libraries are also shared online with the community.

A generative adversarial network (GAN) usually contains a generative network and a discriminative network in competition with each other. The GAN has shown its capability in a variety of applications. In this paper, the usefulness and effectiveness of GAN for classification of hyperspectral images (HSIs) are explored for the first time. In the proposed GAN, a convolutional neural network (CNN) is designed to discriminate the inputs and another CNN is used to generate so-called fake inputs. The aforementioned CNNs are trained together: the generative CNN tries to generate fake inputs that are as real as possible, and the discriminative CNN tries to classify the real and fake inputs. This kind of adversarial training improves the generalization capability of the discriminative CNN, which is really important when the training samples are limited. Specifically, we propose two schemes: 1) a well-designed 1D-GAN as a spectral classifier and 2) a robust 3D-GAN as a spectral–spatial classifier. Furthermore, the generated adversarial samples are used with real training samples to fine-tune the discriminative CNN, which improves the final classification performance. The proposed classifiers are carried out on three widely used hyperspectral data sets: Salinas, Indiana Pines, and Kennedy Space Center. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods. In addition, the proposed GANs open new opportunities in the remote sensing community for the challenging task of HSI classification and also reveal the huge potential of GAN-based methods for the analysis of such complex and inherently nonlinear data.

In recent years, deep learning-based methods, especially convolutional neural networks (CNNs), have shown their capabilities in remote sensing data processing. The efficacy of light detection and ranging (LiDAR) has been already proven in a wide variety of research areas. Most of the existing methods do not extract the informative features from LiDAR-derived rasterized digital surface models (LiDAR-DSM) data in a deep manner. In order to utilize the advantages of deep models for the classification of LiDAR-derived features, deep CNN is proposed here to hierarchically extract the robust and discriminant features of the input data. Moreover, morphological profiles and multiattribute profiles (MAPs) are investigated to enrich the inputs of the CNN and further to improve the ultimate classification performance. Furthermore, a new activation function, sigmoid-weighted linear units (SiLUs), is introduced. The proposed frameworks are tested on two LiDAR-DSMs (i.e., Bayview Park and Houston data sets). The MAP-CNNs with SiLU outperform original CNNs by 6.62% and 6.88% in terms of overall accuracy on Bayview Park and Houston data sets, respectively, when the number of training samples of each class is 40.

Hyperspectral remote sensing obtains abundant spectral and spatial information of the observed object simultaneously. It is an opportunity to classify hyperspectral imagery (HSI) with a fine-grained manner. In this study, the fine-grained classification of HSI, which contains a large number of classes, is investigated. On one hand, traditional classification methods cannot handle fine-grained classification of HSI well, on the other hand, deep learning methods have shown their powerfulness in fine-grained classification. So, in this paper, deep learning is explored for HSI supervised and semi-supervised fine-grained classification. For supervised HSI fine-grained classification, densely connected convolutional neural network (DenseNet) is explored for accurate classification. Moreover, DenseNet is combined with pre-processing technique (i.e., principal component analysis or auto-encoder) or post-processing technique (i.e., conditional random field) to further improve classification performance. For semi-supervised HSI fine-grained classification, a generative adversarial network (GAN), which includes a discriminative CNN and a generative CNN, is carefully designed. The GAN fully uses the labeled and unlabeled samples to improve classification accuracy. The proposed methods were tested on the Indian Pines data set, which contains 33,3951 samples with 52 classes. The experimental results show that the deep learning-based methods provide great improvements compared with other traditional methods, which demonstrate that deep models have huge potential for HSI fine-grained classification.

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework which divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral-spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments.

A reconfigurable low noise amplifier with sub-amplifier cascode miller compensation is proposed to obtain stability both in action potential (AP) and local field potential (LFP) applications. The phase margin of more than 60 degrees can be guaranteed in both two modes compared with traditional method. Simulations have been performed in standard $0.18\mu m$ technology parameters and the simulated characteristics of the proposed amplifier show that the input referred noise and the power consumption is 3.44/6.SS$\mu$Vrms and $2.4S/5.46\mu W$ in both AP and LFP bands.

In this letter, a self-improving convolutional neural network (CNN) based method is proposed for the classification of hyperspectral data. This approach solves the so-called curse of dimensionality and the lack of available training samples by iteratively selecting the most informative bands suitable for the designed network via fractional order Darwinian particle swarm optimization. The selected bands are then fed to the classification system to produce the final classification map. Experimental results have been conducted with two well-known hyperspectral data sets: Indian Pines and Pavia University. Results indicate that the proposed approach significantly improves a CNN-based classification method in terms of classification accuracy. In addition, this letter uses the concept of dither for the first time in the remote sensing community to tackle overfitting.

Due to the advantages of deep learning, in this paper, a regularized deep feature extraction (FE) method is presented for hyperspectral image (HSI) classification using a convolutional neural network (CNN). The proposed approach employs several convolutional and pooling layers to extract deep features from HSIs, which are nonlinear, discriminant, and invariant. These features are useful for image classification and target detection. Furthermore, in order to address the common issue of imbalance between high dimensionality and limited availability of training samples for the classification of HSI, a few strategies such as L2 regularization and dropout are investigated to avoid overfitting in class data modeling. More importantly, we propose a 3-D CNN-based FE model with combined regularization to extract effective spectral-spatial features of hyperspectral imagery. Finally, in order to further improve the performance, a virtual sample enhanced method is proposed. The proposed approaches are carried out on three widely used hyperspectral data sets: Indian Pines, University of Pavia, and Kennedy Space Center. The obtained results reveal that the proposed models with sparse constraints provide competitive results to state-of-the-art methods. In addition, the proposed deep FE opens a new window for further research.

Recovering hyperspectral image (HSI) from mixed noise degradation is a challenging and promising theme in remote sensing, particularly when stripes and deadlines exist in several contiguous bands. This paper proposes a HSI’s restoration method making use of adaptive morphological filtering (AMF) and fusing structure information of an auxiliary color image. An adaptive structuring element (ASE) indicating morphological features of each pixel is generated through information fusion, to simultaneously remove the mixed noise and preserve fine spatial structures. This key technology contains three main steps. First, edges are extracted from the auxiliary image exploiting its color information; then, an edge-constraint growing algorithm is used to generate the clustering kernel; finally, the ASE is obtained via goal-guided k-means clustering. The ASE has extensive application value, for it can be an enhancing module for most filters-based restoration methods, to mitigate the structural damage due to the fixed mask. Among these methods, Gaussian filter for preprocessing and majority voting for postprocessing are introduced in this paper as representatives. In addition, the auxiliary image can be both visible image of multisensor and false RGB component of the undamaged bands of the HSI, so it is relatively available. Experiments on simulated and real data sets show obvious effects on denoising and destriping both subjectively and objectively. The advantage of ASE on structure details preserving, compared to conventional approaches, is clearly demonstrated. The application value of the proposed restoration frame and ASE is further proved through the decision-level postprocessing experiments.

Capacitor mismatch plays an important part in the spurious-free dynamic range (SFDR) performance of high-resolution successive approximation register (SAR) analog-to-digital converters (ADCs). This paper presents a symmetric group control dynamic element matching (SGCDEM) method aiming at enhancing SFDR of SAR ADCs applied in biomedical applications. The proposed scheme symmetrically divides the capacitor array into four sub-DACs, using quantization results corresponding to each sub-DAC as rotary control codes to symmetrically control the rotation of the capacitors. In order to improve the energy efficiency and save chip area, the proposed method replaces the pseudo-random number generator (PRNG) with quantization results of ADC as the rotary control code. Moreover, split rotator technology is applied in the proposed scheme to reduce the number of multiplexers (MUX) used in the rotator, which introduces a low logic depth. The proposed architecture has been implemented on a 12-bit SAR ADC model in MATLAB in order to verify its performances. After taking a standard deviation of 1% for the unit capacitance into consideration, this method achieves above 77.3[Formula: see text]dB SFDR, enabling to correct the SFDR by more than 23.6[Formula: see text]dB compared with the structure without DEM.

Publisher's version (útgefin grein), Capsule networks can be considered to be the next era of deep learning and have recently shown their advantages in supervised classification. Instead of using scalar values to represent features, the capsule networks use vectors to represent features, which enriches the feature presentation capability. This paper introduces a deep capsule network for hyperspectral image (HSI) classification to improve the performance of the conventional convolutional neural networks (CNNs). Furthermore, a modification of the capsule network named Conv-Capsule is proposed. Instead of using full connections, local connections and shared transform matrices, which are the core ideas of CNNs, are used in the Conv-Capsule network architecture. In Conv-Capsule, the number of trainable parameters is reduced compared to the original capsule, which potentially mitigates the overfitting issue when the number of available training samples is limited. Specifically, we propose two schemes: (1) A 1D deep capsule network is designed for spectral classification, as a combination of principal component analysis, CNN, and the Conv-Capsule network, and (2) a 3D deep capsule network is designed for spectral-spatial classification, as a combination of extended multi-attribute profiles, CNN, and the Conv-Capsule network. The proposed classifiers are tested on three widely-used hyperspectral data sets. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods, including kernel support vector machines, CNNs, and recurrent neural network., This research was funded by Natural Science Foundation of China under the Grant 61771171.

Hyperspectral image (HSI) classification is a core task in the remote sensing community, and recently, deep learning-based methods have shown their capability of accurate classification of HSIs. Among the deep learning-based methods, deep convolutional neural networks (CNNs) have been widely used for the HSI classification. In order to obtain a good classification performance, substantial efforts are required to design a proper deep learning architecture. Furthermore, the manually designed architecture may not fit a specific data set very well. In this paper, the idea of automatic CNN for the HSI classification is proposed for the first time. First, a number of operations, including convolution, pooling, identity, and batch normalization, are selected. Then, a gradient descent-based search algorithm is used to effectively find the optimal deep architecture that is evaluated on the validation data set. After that, the best CNN architecture is selected as the model for the HSI classification. Specifically, the automatic 1-D Auto-CNN and 3-D Auto-CNN are used as spectral and spectral–spatial HSI classifiers, respectively. Furthermore, the cutout is introduced as a regularization technique for the HSI spectral–spatial classification to further improve the classification accuracy. The experiments on four widely used hyperspectral data sets (i.e., Salinas, Pavia University, Kennedy Space Center, and Indiana Pines) show that the automatically designed data-dependent CNNs obtain competitive classification accuracy compared with the state-of-the-art methods. In addition, the automatic design of the deep learning architecture opens a new window for future research, showing the huge potential of using neural architectures’ optimization capabilities for the accurate HSI classification.

Deep learning models, especially deep convolutional neural networks (CNNs), have been intensively investigated for hyperspectral image (HSI) classification due to their powerful feature extraction ability. In the same manner, ensemble-based learning systems have demonstrated high potential to effectively perform supervised classification. In order to boost the performance of deep learning-based HSI classification, the idea of deep learning ensemble framework is proposed here, which is loosely based on the integration of deep learning model and random subspace-based ensemble learning. Specifically, two deep learning ensemble-based classification methods (i.e., CNN ensemble and deep residual network ensemble) are proposed. CNNs or deep residual networks are used as individual classifiers and random subspaces contribute to diversify the ensemble system in a simple yet effective manner. Moreover, to further improve the classification accuracy, transfer learning is investigated in this study to transfer the learnt weights from one individual classifier to another (i.e., CNNs). This mechanism speeds up the learning stage. Experimental results with widely used hyperspectral datasets indicate that the proposed deep learning ensemble system provides competitive results compared with state-of-the-art methods in terms of classification accuracy. The combination of deep learning and ensemble learning provides a significant potential for reliable HSI classification.

Hyperspectral data classification is a hot topic in remote sensing community. In recent years, significant effort has been focused on this issue. However, most of the methods extract the features of original data in a shallow manner. In this paper, we introduce a deep learning approach into hyperspectral image classification. A new feature extraction (FE) and image classification framework are proposed for hyperspectral data analysis based on deep belief network (DBN). First, we verify the eligibility of restricted Boltzmann machine (RBM) and DBN by the following spectral information-based classification. Then, we propose a novel deep architecture, which combines the spectral–spatial FE and classification together to get high classification accuracy. The framework is a hybrid of principal component analysis (PCA), hierarchical learning-based FE, and logistic regression (LR). Experimental results with hyperspectral data indicate that the classifier provide competitive solution with the state-of-the-art methods. In addition, this paper reveals that deep learning system has huge potential for hyperspectral data classification.

A novel object tracking algorithm based on hierarchical convolutional features was proposed in this paper. Firstly, the tracking algorithm uses the hierarchical networks of VGG-Net-19 to extract the hierarchical convolutional features of image, having a greater improvement than using only one layer to do that. Secondly, the algorithm obtains features by using correlation filtering method with weighted fusion, so as to determine the real position of the target according to the characteristics of different layers. The experimental results show that, compared with the current four popular object tracking algorithms, the proposed algorithm achieves better accuracy and success rate, and the results are consistent in OPE (one-pass evaluation), SRE (spatial robustness evaluation) and TRE (temporal robustness evaluation).

Recently, the capability of deep learning-based approaches, especially deep convolutional neural networks (CNNs), has been investigated for hyperspectral remote sensing feature extraction (FE) and classification. Due to the large number of learnable parameters in convolutional filters, lots of training samples are needed in deep CNNs to avoid the overfitting problem. On the other hand, Gabor filtering can effectively extract spatial information including edges and textures, which may reduce the FE burden of the CNNs. In this letter, in order to make the most of deep CNN and Gabor filtering, a new strategy, which combines Gabor filters with convolutional filters, is proposed for hyperspectral image classification to mitigate the problem of overfitting. The obtained results reveal that the proposed model provides competitive results in terms of classification accuracy, especially when only a limited number of training samples are available.

The multisensory fusion of remote sensing data has obtained a great attention in recent years. In this letter, we propose a new feature fusion framework based on deep neural networks (DNNs). The proposed framework employs deep convolutional neural networks (CNNs) to effectively extract features of multi-/hyperspectral and light detection and ranging data. Then, a fully connected DNN is designed to fuse the heterogeneous features obtained by the previous CNNs. Through the aforementioned deep networks, one can extract the discriminant and invariant features of remote sensing data, which are useful for further processing. At last, logistic regression is used to produce the final classification results. Dropout and batch normalization strategies are adopted in the deep fusion framework to further improve classification accuracy. The obtained results reveal that the proposed deep fusion model provides competitive results in terms of classification accuracy. Furthermore, the proposed deep learning idea opens a new window for future remote sensing data fusion.

Classification is one of the most popular topics in hyperspectral remote sensing. In the last two decades, a huge number of methods were proposed to deal with the hyperspectral data classification problem. However, most of them do not hierarchically extract deep features. In this paper, the concept of deep learning is introduced into hyperspectral data classification for the first time. First, we verify the eligibility of stacked autoencoders by following classical spectral information-based classification. Second, a new way of classifying with spatial-dominated information is proposed. We then propose a novel deep learning framework to merge the two features, from which we can get the highest classification accuracy. The framework is a hybrid of principle component analysis (PCA), deep learning architecture, and logistic regression. Specifically, as a deep learning architecture, stacked autoencoders are aimed to get useful high-level features. Experimental results with widely-used hyperspectral data indicate that classifiers built in this deep learning-based framework provide competitive performance. In addition, the proposed joint spectral-spatial deep neural network opens a new window for future research, showcasing the deep learning-based methods' huge potential for accurate hyperspectral data classification.

Existing deep convolutional neural networks (CNNs) have shown their great success on image classification. CNNs mainly consist of convolutional and pooling layers, both of which are performed on local image areas without considering the dependencies among different image regions. However, such dependencies are very important for generating explicit image representation. In contrast, recurrent neural networks (RNNs) are well known for their ability of encoding contextual information among sequential data, and they only require a limited number of network parameters. General RNNs can hardly be directly applied on non-sequential data. Thus, we proposed the hierarchical RNNs (HRNNs). In HRNNs, each RNN layer focuses on modeling spatial dependencies among image regions from the same scale but different locations. While the cross RNN scale connections target on modeling scale dependencies among regions from the same location but different scales. Specifically, we propose two recurrent neural network models: 1) hierarchical simple recurrent network (HSRN), which is fast and has low computational cost; and 2) hierarchical long-short term memory recurrent network (HLSTM), which performs better than HSRN with the price of more computational cost. In this manuscript, we integrate CNNs with HRNNs, and develop end-to-end convolutional hierarchical recurrent neural networks (C-HRNNs). C-HRNNs not only make use of the representation power of CNNs, but also efficiently encodes spatial and scale dependencies among different image regions. On four of the most challenging object/scene image classification benchmarks, our C-HRNNs achieve state-of-the-art results on Places 205, SUN 397, MIT indoor, and competitive results on ILSVRC 2012.

Hyperspectral image classification has been a vibrant area of research in recent years. Given a set of observations, i.e., pixel vectors in a hyperspectral image, classification approaches try to allocate a unique label to each pixel vector. However, the classification of hyperspectral images is a challenging task for a number of reasons, such as the presence of redundant features, the imbalance among the limited number of available training samples, and the high dimensionality of the data.

Recently, the fusion of hyperspectral and light detection and ranging (LiDAR) data has obtained a great attention in the remote sensing community. In this paper, we propose a new feature fusion framework using deep neural network (DNN). The proposed framework employs a novel 3D convolutional neural network (CNN) to extract the spectral-spatial features of hyperspectral data, a deep 2D CNN to extract the elevation features of LiDAR data, and then a fully connected deep neural network to fuse the extracted features in the previous CNNs. Through the aforementioned three deep networks, one can extract the discriminant and invariant features of hyperspectral and LiDAR data. At last, logistic regression is used to produce the final classification results. The experimental results reveal that the proposed deep fusion model provides competitive results. Furthermore, the proposed deep fusion idea opens a new window for future research.

Hyperspectral images (HSIs) are often contaminated by noise, in order to remove the image noise efficiently and acquire excellent results. We propose a new denoising method based on 3D sparse coding. Firstly, to make full use of spectral information of hyperspectral data, we extract patches from HSIs and each patch contains the same area of different band. Secondly, we use aforementioned method to extract all patches and train these patches, the dictionary can be obtained, further calculate sparse coefficients. Finally, we can restore the HISs through the dictionary and the sparse coefficients. Experiments are implemented using the HSIs collected by AVIRIS and ROSIS. Results indicate that compared with common 2D sparse coding method, 3D sparse method can effectively improve the restoration performance for both subjective visual and objective evaluation criterion.

In existing convolutional neural networks (CNNs), both convolution and pooling are locally performed for image regions separately, no contextual dependencies between different image regions have been taken into consideration. Such dependencies represent useful spatial structure information in images. Whereas recurrent neural networks (RNNs) are designed for learning contextual dependencies among sequential data by using the recurrent (feedback) connections. In this work, we propose the convolutional recurrent neural network (C-RNN), which learns the spatial dependencies between image regions to enhance the discriminative power of image representation. The C-RNN is trained in an end-to-end manner from raw pixel images. CNN layers are firstly processed to generate middle level features. RNN layer is then learned to encode spatial dependencies. The C-RNN can learn better image representation, especially for images with obvious spatial contextual dependencies. Our method achieves competitive performance on ILSVRC 2012, SUN 397, and MIT indoor.

In hyperspectral images, there is abundant spectral information for classification. In most cases, spectral information based methods yield good classification results. However, different objects may have similar spectrums due to similar physical property. Spectral information based classification methods can't give accurate results for the similar physical property among objects belonging to the same main category. On the other hand, as the resolutions of sensors increase in recent years, more spatial information, such as shape and texture information can be extracted and described more precisely. In this paper, we proposed a spatial information aided similar spectral classification. In our method, spatial features, including the pixel shape index (PSI), and the gray-level co-occurrence matrix (GLCM), are extracted for accurate classification. We compared the Bhattachary Distance before and after spatial information being aided and we found that the B Distance was amplified sharply, which indicated that the separability among classes increased. Classification is practiced on two hyperspectral data sets, Kennedy Space Center and Pavia City, and the proposed method is compared with classification based on different features. It is found that each feature makes contribution to classification and the accuracy of the proposed method is the highest among all methods, which certifies the effectiveness of our algorithm.

The paper presents a novel ensemble system which unites Adaboost with multifeature to increase diversity among individual classifiers. Adaboost gives rise to convenience for hyperspectral data classification. To improve the method further, we propose joint Adaboost and multifeature based ensemble (JAME), which assigns different multifeature sets to individual classifiers in Adaboost. Diverse spectral and spatial feature sets are integrated to form multifeature sets. As a result, compared with Adaboost the method has increased the diversity of ensemble system, and better overall accuracies are present. Experiments on hyperspectral data sets reveal that the proposed JAME obtains sound performances comparing with original Adaboost and single classifier.

Hyperspectral image (HSI) classification is a hot topic in the remote sensing community. This paper proposes a new framework of spectral-spatial feature extraction for HSI classification, in which for the first time the concept of deep learning is introduced. Specifically, the model of autoencoder is exploited in our framework to extract various kinds of features. First we verify the eligibility of autoencoder by following classical spectral information based classification and use autoencoders with different depth to classify hyperspectral image. Further in the proposed framework, we combine PCA on spectral dimension and autoencoder on the other two spatial dimensions to extract spectral-spatial information for classification. The experimental results show that this framework achieves the highest classification accuracy among all methods, and outperforms classical classifiers such as SVM and PCA-based SVM., Comment: Accepted as a conference paper at ICICS 2013, an updated version. Codes published. 9 pages, 6 figures

3D city models typically consist of thousands of buildings in different types. We usually reconstruct these buildings automatically from high-resolution satellite or airborne imagery. However, for detailed roof reconstruction, 2D information offered by imagery data is not enough while DSM data is necessary. In this paper, we propose a novel level set framework for 3D building models in LOD2 with geometry structure of typical roofs. Local information is introduced towards multiphase and multichannel level set method. Its energy function is minimized when each part of roof data corresponds to the same normal vector as feature values for level set segmentation. The advantage of this method is that for complex building models, roof primitives as well as roof topology graph can be extracted from high-resolution DSM data with high accuracy, evaluated by completeness of segmentation and RMSE of 3D reconstruction. Thus, LOD2 building models can be reconstructed automatically with good performance. The very promising experimental results demonstrate the potentials of our method for large-scale building reconstruction in LOD2.