Search

Scene classification of remote sensing images has drawn great attention because of its wide applications. In this paper, with the guidance of the human visual system (HVS), we explore the attention mechanism and propose a novel end-to-end attention recurrent convolutional network (ARCNet) for scene classification. It can learn to focus selectively on some key regions or locations and just process them at high-level features, thereby discarding the noncritical information and promoting the classification performance. The contributions of this paper are threefold. First, we design a novel recurrent attention structure to squeeze high-level semantic and spatial features into several simplex vectors for the reduction of learning parameters. Second, an end-to-end network named ARCNet is proposed to adaptively select a series of attention regions and then to generate powerful predictions by learning to process them sequentially. Third, we construct a new data set named OPTIMAL-31, which contains more categories than popular data sets and gives researchers an extra platform to validate their algorithms. The experimental results demonstrate that our model makes great promotion in comparison with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Change detection has been a hotspot in the remote sensing technology for a long time. With the increasing availability of multi-temporal remote sensing images, numerous change detection algorithms have been proposed. Among these methods, image transformation methods with feature extraction and mapping could effectively highlight the changed information and thus has a better change detection performance. However, the changes of multi-temporal images are usually complex, and the existing methods are not effective enough. In recent years, the deep network has shown its brilliant performance in many fields, including feature extraction and projection. Therefore, in this paper, based on the deep network and slow feature analysis (SFA) theory, we proposed a new change detection algorithm for multi-temporal remotes sensing images called deep SFA (DSFA). In the DSFA model, two symmetric deep networks are utilized for projecting the input data of bi-temporal imagery. Then, the SFA module is deployed to suppress the unchanged components and highlight the changed components of the transformed features. The change vector analysis pre-detection is employed to find unchanged pixels with high confidence as training samples. Finally, the change intensity is calculated with chi-square distance and the changes are determined by threshold algorithms. The experiments are performed on two real-world data sets and a public hyperspectral data set. The visual comparison and the quantitative evaluation have shown that DSFA could outperform the other state-of-the-art algorithms, including other SFA-based and deep learning methods. [ABSTRACT FROM AUTHOR]

For hyperspectral image classification, there is a large gap between the theoretical method and the practical application. Hyperspectral image classification in theoretical research trains a new classifier for each data set, which is ineffective and even infeasible in large-scale applications. In this paper, we make a preliminary attempt to recycle the classification model to new data sets in an unsupervised way. Specially, we propose a cross-data set hyperspectral image classification method based on deep domain adaptation. The proposed method contains three modules: domain alignment module that learns to minimize the domain discrepancy with the guide of an irrelevant task, task allocation module that learns to classify on the source domain with the regulation of domain alignment, and domain adaptation module that transfers both the alignment ability and classification ability to the target domain by an adaptation strategy. As a result, with the information of an irrelevant task on dual-domain data sets, we can minimize the domain discrepancy and transfer the task-relevant knowledge from the source domain to the target domain in an unsupervised way. Extensive experiments on three hyperspectral images demonstrate the effectiveness of our method compared with other related methods when dealing with new data sets. [ABSTRACT FROM AUTHOR]

Road detection and centerline extraction from very high-resolution (VHR) remote sensing imagery are of great significance in various practical applications. Road detection and centerline extraction operations depend on each other, to a certain extent. The road detection constrains the appearance of the centerline, and the centerline enhances the linear features of the road detection. However, most of the previous works have addressed these two tasks separately and have not considered the symbiotic relationship between them, making it difficult to obtain smooth and complete roads. In this paper, a novel multi-scale and multi-task deep learning framework for automatic road extraction (MSMT-RE) is proposed to build the relationship between them and simultaneously complete the road detection and centerline extraction tasks. U-Net is selected as the basic network for multi-task learning due to its strong ability to preserve spatial details. Multi-scale feature integration is also applied in the framework to increase the robustness of the feature extraction. Meanwhile, an adaptive loss function is introduced to solve the problems of roads taking up a small percentage of the training samples, and the fact that the positive samples of the two tasks are unbalanced. Finally, experiments were conducted on two public road data sets and two large images from Google Earth, and the proposed framework was compared with other state-of-the-art deep learning-based road extraction methods, both quantitatively and qualitatively. The proposed approach outperformed all the compared methods, confirming its advantages in automatic road extraction. [ABSTRACT FROM AUTHOR]

Recently, deep neural networks have received intense interests in polarimetric synthetic aperture radar (PolSAR) image classification. However, its success is subject to the availability of large amounts of annotated data which require great efforts of experienced human annotators. Aiming at improving the classification performance with greatly reduced annotation cost, this paper presents an active deep learning approach for minimally supervised PolSAR image classification, which integrates active learning and fine-tuned convolutional neural network (CNN) into a principled framework. Starting from a CNN trained using a very limited number of labeled pixels, we iteratively and actively select the most informative candidates for annotation, and incrementally fine-tune the CNN by incorporating the newly annotated pixels. Moreover, to boost the performance and robustness of the proposed method, we employ Markov random field (MRF) to enforce class label smoothness, and data augmentation technique to enlarge the training set. We conducted extensive experiments on four real benchmark PolSAR images, and experiments demonstrated that our approach achieved state-of-the-art classification results with significantly reduced annotation cost. [ABSTRACT FROM AUTHOR]

For hyperspectral remote sensing image (HSI) classification, the learning process of deep neural networks has been progressively advanced in depth, but the fine features are often largely lost or even disappear in the process of depth transfer. With the increase in feature aggregation and connectivity, the complexity of the network and the training parameters increases greatly, requiring more training time. This paper proposed a multi-scale dense network (MSDN) for HSI classification that made full use of different scale information in the network structure and combined scale information throughout the network. It implemented feature extraction of HSIs in two dimensions, including the features at fine and coarse levels. In the horizontal direction, it considered the deep extraction of HSI features, and the 3-D dense connection structure was used for aggregating features at different levels. In the vertical direction, scale information was considered, and three-scale feature maps at low, middle, and high levels were generated based on the first layer of the network. The MSDN used stride convolution for downsampling and combined feature information at different scale levels. The MSDN extracted features along the diagonal line. The network implemented the reconstruction of deep feature extraction and multi-scale fusion for HSI classification. The MSDN model performed well on representative HSI datasets, namely, the Indian Pines, Pavia University, Salinas, Botswana, and Kennedy Space Center datasets. It improved the training speed and accuracy for HSI classification and especially improved the convergence speed, which effectively saved computing resources and had high stability. [ABSTRACT FROM AUTHOR]

Semantic segmentation requires methods capable of learning high-level features while dealing with large volume of data. Toward such goal, convolutional networks can learn specific and adaptable features based on the data. However, these networks are not capable of processing a whole remote sensing image, given its huge size. To overcome such limitation, the image is processed using fixed size patches. The definition of the input patch size is usually performed empirically (evaluating several sizes) or imposed (by network constraint). Both strategies suffer from drawbacks and could not lead to the best patch size. To alleviate this problem, several works exploited multicontext information by combining networks or layers. This process increases the number of parameters, resulting in a more difficult model to train. In this paper, we propose a novel technique to perform semantic segmentation of remote sensing images that exploits a multicontext paradigm without increasing the number of parameters while defining, in training time, the best patch size. The main idea is to train a dilated network with distinct patch sizes, allowing it to capture multicontext characteristics from heterogeneous contexts. While processing these varying patches, the network provides a score for each patch size, helping in the definition of the best size for the current scenario. A systematic evaluation of the proposed algorithm is conducted using four high-resolution remote sensing data sets with very distinct properties. Our results show that the proposed algorithm provides improvements in pixelwise classification accuracy when compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The shortage of training samples remains one of the main obstacles in applying the neural networks to the hyperspectral images classification. To fuse the spatial and spectral information, pixel patches are often utilized to train a model, which may further aggregate this problem. In the existing works, an artificial neural network (ANN) model supervised by centerloss (ANNC) was introduced. Training merely with spectral information, the ANNC yields discriminative spectral features suitable for the subsequent classification tasks. In this paper, we propose a novel convolutional neural network (CNN)-based spatial feature fusion (CSFF) algorithm, which allows a smart integration of spatial information to the spectral features extracted by ANNC. As a critical part of CSFF, a CNN-based discriminant model is introduced to estimate whether two pixels belong to the same class. At the testing stage, by applying the discriminant model to the pixel pairs generated by a test pixel and each of its neighbors, the local structure is estimated and represented as a customized convolutional kernel. The spectral–spatial feature is generated by a convolutional operation between the estimated kernel and the corresponding spectral features within a local region. The final label is determined by classifying the resulting spectral–spatial feature. Without increasing the number of training samples or involving pixel patches at the training stage, the CSFF framework achieves the state of the art by declining 20%–50% classification failures in experiments on three well-known hyperspectral images. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is a core task in the remote sensing community, and recently, deep learning-based methods have shown their capability of accurate classification of HSIs. Among the deep learning-based methods, deep convolutional neural networks (CNNs) have been widely used for the HSI classification. In order to obtain a good classification performance, substantial efforts are required to design a proper deep learning architecture. Furthermore, the manually designed architecture may not fit a specific data set very well. In this paper, the idea of automatic CNN for the HSI classification is proposed for the first time. First, a number of operations, including convolution, pooling, identity, and batch normalization, are selected. Then, a gradient descent-based search algorithm is used to effectively find the optimal deep architecture that is evaluated on the validation data set. After that, the best CNN architecture is selected as the model for the HSI classification. Specifically, the automatic 1-D Auto-CNN and 3-D Auto-CNN are used as spectral and spectral–spatial HSI classifiers, respectively. Furthermore, the cutout is introduced as a regularization technique for the HSI spectral–spatial classification to further improve the classification accuracy. The experiments on four widely used hyperspectral data sets (i.e., Salinas, Pavia University, Kennedy Space Center, and Indiana Pines) show that the automatically designed data-dependent CNNs obtain competitive classification accuracy compared with the state-of-the-art methods. In addition, the automatic design of the deep learning architecture opens a new window for future research, showing the huge potential of using neural architectures’ optimization capabilities for the accurate HSI classification. [ABSTRACT FROM AUTHOR]

Supervised deep neural networks (DNNs) have been extensively used in diverse tasks. Generally, training such DNNs with superior performance requires a large amount of labeled data. However, it is time-consuming and expensive to manually label the data, especially for tasks in remote sensing, e.g., change detection. The situation motivates us to resort to the existing related images with labels, from which the concept of change can be adapted to new images. However, the distributions of the related labeled images (source domain) and unlabeled new images (target domain) are similar but not identical. It impedes a change detection model learned from source domains being well applied to the target domain. In this paper, we propose a transferred deep learning-based change detection framework to solve this problem. It consists of pretraining and fine-tuning stages. In the pretraining process, we propose two tasks to be learned simultaneously, namely, change detection for the source domain with labels and reconstruction of the unlabeled target data. The auxiliary task aims to reconstruct the difference image (DI) for the target domain. DI is an effective feature, such that the auxiliary task is of much relevance to change detection. The lower layers are shared between these two tasks in the training process. It mitigates the distribution discrepancy between the source and target domains and makes the concept of change from the source domain adapt to the target domain. In addition, we evaluate three modes of the U-net architecture to merge the information for a pair of patches. To fine-tune the change detection network (CDN) for the target domain, two strategies are exploited to select the pixels that have a high possibility of being correctly classified by an unsupervised approach. The proposed method demonstrates an excellent capacity for adapting the concept of change from the source domain to the target domain. It outperforms the state-of-the-art change detection methods via experimental results on real remote sensing data sets. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework that divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral–spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments. [ABSTRACT FROM AUTHOR]

With the rapid development of various technologies of satellite sensor, synthetic aperture radar (SAR) image has been an import source of data in the application of change detection. In this paper, a novel method based on a convolutional neural network (CNN) for SAR image change detection is proposed. The main idea of our method is to generate the classification results directly from the original two SAR images through a CNN without any preprocessing operations, which also eliminate the process of generating the difference image (DI), thus reducing the influence of the DI on the final classification result. In CNN, the spatial characteristics of the raw image can be extracted and captured by automatic learning and the results with stronger robustness can be obtained. The basic idea of the proposed method includes three steps: it first produces false labels through unsupervised spatial fuzzy clustering. Then we train the CNN through proper samples that are selected from the samples with false labels. Finally, the final detection results are obtained by the trained convolutional network. Although training the convolutional network is a supervised learning fashion, the whole process of the algorithm is an unsupervised process without priori knowledge. The theoretical analysis and experimental results demonstrate the validity, robustness, and potential of our algorithm in simulated and real data sets. In addition, we try to apply our algorithm to the change detection of heterogeneous images, which also achieves satisfactory results. [ABSTRACT FROM AUTHOR]

By considering the spectral signature as a sequence, recurrent neural networks (RNNs) have been successfully used to learn discriminative features from hyperspectral images (HSIs) recently. However, most of these models only input the whole spectral bands into RNNs directly, which may not fully explore the specific properties of HSIs. In this paper, we propose a cascaded RNN model using gated recurrent units to explore the redundant and complementary information of HSIs. It mainly consists of two RNN layers. The first RNN layer is used to eliminate redundant information between adjacent spectral bands, while the second RNN layer aims to learn the complementary information from nonadjacent spectral bands. To improve the discriminative ability of the learned features, we design two strategies for the proposed model. Besides, considering the rich spatial information contained in HSIs, we further extend the proposed model to its spectral–spatial counterpart by incorporating some convolutional layers. To test the effectiveness of our proposed models, we conduct experiments on two widely used HSIs. The experimental results show that our proposed models can achieve better results than the compared models. [ABSTRACT FROM AUTHOR]

Recently, deep learning-based methods have brought new ideas for ship detection in synthetic aperture radar (SAR) images. However, several challenges still exist: 1) deep models contain millions of parameters, whereas the available annotated samples are not sufficient in number for training. Therefore, most deep detectors have to fine-tune networks pre-trained on ImageNet, which incurs learning bias due to the huge domain mismatch between SAR images and ImageNet images. Furthermore, it has a little flexibility to redesign the network structure; and 2) ships in SAR images are relatively small in size and densely clustered, whereas most deep detectors have poor performance with small objects due to the rough feature map used for detection and the extreme foreground–background imbalance. To address these problems, this paper proposes an effective approach to learn deep ship detector from scratch. First, we design a condensed backbone network, which consists of several dense blocks. Hence, earlier layers can receive additional supervision from the objective function through the dense connections, which makes it easy to train. In addition, feature reuse strategy is adopted to make it highly parameter efficient. Therefore, the backbone network could be freely designed and effectively trained from scratch without using a large amount of annotated samples. Second, we improve the cross-entropy loss to address the foreground–background imbalance and predict multi-scale ship proposals from several intermediate layers to improve the recall rate. Then, position-sensitive score maps are adopted to encode position information into each ship proposal for discrimination. The comparison results on the Sentinel-1 data set show that: 1) learning ship detector from scratch achieved better performance than ImageNet pre-trained model-based detectors and 2) our method is more effective than existing algorithms for detecting the small and densely clustered ships. [ABSTRACT FROM AUTHOR]

Recently, deep learning (DL) is of great interest in hyperspectral image (HSI) classification. Although many effective frameworks exist in the literature, the generally limited availability of training samples poses great challenges in applying DL to HSI classification. In this paper, we present a novel DL framework, namely, semisupervised stacked autoencoders (Semi-SAEs) with cotraining, for HSI classification. First, two SAEs are pretrained based on the hyperspectral features and the spatial features, respectively. Second, fine-tuning is alternatively conducted for the two SAEs in a semisupervised cotraining fashion, where the initial training set is enlarged by designing an effective region growing method. Finally, the classification probabilities obtained by the two SAEs are fused using a Markov random field model solved by iterated conditional modes. Experimental results based on three popular hyperspectral data sets demonstrate that the proposed method outperforms other state-of-the-art DL methods. [ABSTRACT FROM AUTHOR]

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial–spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min–max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen–Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a spectral-spatial feature extraction and classification framework based on an artificial neuron network in the context of hyperspectral imagery. With limited labeled samples, only spectral information is exploited for training and spatial context is integrated posteriorly at the testing stage. Taking advantage of recent advances in face recognition, a joint supervision symbol that combines softmax loss and center loss is adopted to train the proposed network, by which intraclass features are gathered while interclass variations are enlarged. Based on the learned architecture, the extracted spectrum-based features are classified by a center classifier. Moreover, to fuse the spectral and spatial information, an adaptive spectral-spatial center classifier is developed, where multiscale neighborhoods are considered simultaneously, and the final label is determined using an adaptive voting strategy. Finally, experimental results on three well-known data sets validate the effectiveness of the proposed methods compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Deep learning has recently attracted significant attention in the field of hyperspectral images (HSIs) classification. However, the construction of an efficient deep neural network mostly relies on a large number of labeled samples being available. To address this problem, this paper proposes a unified deep network, combined with active transfer learning (TL) that can be well-trained for HSIs classification using only minimally labeled training data. More specifically, deep joint spectral–spatial feature is first extracted through hierarchical stacked sparse autoencoder (SSAE) networks. Active TL is then exploited to transfer the pretrained SSAE network and the limited training samples from the source domain to the target domain, where the SSAE network is subsequently fine-tuned using the limited labeled samples selected from both source and target domains by the corresponding active learning (AL) strategies. The advantages of our proposed method are threefold: 1) the network can be effectively trained using only limited labeled samples with the help of novel AL strategies; 2) the network is flexible and scalable enough to function across various transfer situations, including cross data set and intraimage; and 3) the learned deep joint spectral–spatial feature representation is more generic and robust than many joint spectral–spatial feature representations. Extensive comparative evaluations demonstrate that our proposed method significantly outperforms many state-of-the-art approaches, including both traditional and deep network-based methods, on three popular data sets. [ABSTRACT FROM AUTHOR]

Satellite images are widely applied in humanitarian mapping that labels buildings, roads, and so on for humanitarian aid and economic development. However, the labeling now is mostly done by volunteers. In this paper, we utilize deep learning to solve humanitarian mapping tasks of a mobile software named MapSwipe. The current deep learning techniques, e.g., convolutional neural network (CNN), can recognize ground objects from satellite images but rely on numerous labels for training for each specific task. We solve this problem by fusing multiple freely accessible crowdsourced geographic data and propose an active learning-based CNN training framework named MC-CNN to deal with the quality issues of the labels extracted from these data, including incompleteness (e.g., some kinds of object are not labeled) and heterogeneity (e.g., different spatial granularities). The method is evaluated with building mapping in South Malawi and road mapping in Guinea with level-18 satellite images provided by Bing Map and volunteered geographic information from OpenStreetMap, MapSwipe, and OsmAnd. The results based on multiple metrics, including Precision, Recall, F1 Score, and area under the receiver operating characteristic curve, show that MC-CNN can fuse the crowdsourced labels for higher prediction performance and be successfully applied in MapSwipe for humanitarian mapping with 85% labor saved and an overall accuracy of 0.86 achieved. [ABSTRACT FROM AUTHOR]

Image classification is considered to be one of the critical tasks in hyperspectral remote sensing image processing. Recently, a convolutional neural network (CNN) has established itself as a powerful model in classification by demonstrating excellent performances. The use of a graphical model such as a conditional random field (CRF) contributes further in capturing contextual information and thus improving the classification performance. In this paper, we propose a method to classify hyperspectral images by considering both spectral and spatial information via a combined framework consisting of CNN and CRF. We use multiple spectral band groups to learn deep features using CNN, and then formulate deep CRF with CNN-based unary and pairwise potential functions to effectively extract the semantic correlations between patches consisting of 3-D data cubes. Furthermore, we introduce a deep deconvolution network that improves the final classification performance. We also introduced a new data set and experimented our proposed method on it along with several widely adopted benchmark data sets to evaluate the effectiveness of our method. By comparing our results with those from several state-of-the-art models, we show the promising potential of our method. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is an active and important research task driven by many practical applications. To leverage deep learning models especially convolutional neural networks (CNNs) for HSI classification, this paper proposes a simple yet effective method to extract hierarchical deep spatial feature for HSI classification by exploring the power of off-the-shelf CNN models, without any additional retraining or fine-tuning on the target data set. To obtain better classification accuracy, we further propose a unified metric learning-based framework to alternately learn discriminative spectral–spatial features, which have better representation capability and train support vector machine (SVM) classifiers. To this end, we design a new objective function that explicitly embeds a metric learning regularization term into SVM training. The metric learning regularization term is used to learn a powerful spectral–spatial feature representation by fusing spectral feature and deep spatial feature, which has small intraclass scatter but big between class separation. By transforming HSI data into new spectral–spatial feature space through CNN and metric learning, we can pull the pixels from the same class closer, while pushing the different class pixels farther away. In the experiments, we comprehensively evaluate the proposed method on three commonly used HSI benchmark data sets. State-of-the-art results are achieved when compared with the existing HSI classification methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a spectral–spatial unified network (SSUN) with an end-to-end architecture for the hyperspectral image (HSI) classification. Different from traditional spectral–spatial classification frameworks where the spectral feature extraction (FE), spatial FE, and classifier training are separated, these processes are integrated into a unified network in our model. In this way, both FE and classifier training will share a uniform objective function and all the parameters in the network can be optimized at the same time. In the implementation of the SSUN, we propose a band grouping-based long short-term memory model and a multiscale convolutional neural network as the spectral and spatial feature extractors, respectively. In the experiments, three benchmark HSIs are utilized to evaluate the performance of the proposed method. The experimental results demonstrate that the SSUN can yield a competitive performance compared with existing methods. [ABSTRACT FROM AUTHOR]

A generative adversarial network (GAN) usually contains a generative network and a discriminative network in competition with each other. The GAN has shown its capability in a variety of applications. In this paper, the usefulness and effectiveness of GAN for classification of hyperspectral images (HSIs) are explored for the first time. In the proposed GAN, a convolutional neural network (CNN) is designed to discriminate the inputs and another CNN is used to generate so-called fake inputs. The aforementioned CNNs are trained together: the generative CNN tries to generate fake inputs that are as real as possible, and the discriminative CNN tries to classify the real and fake inputs. This kind of adversarial training improves the generalization capability of the discriminative CNN, which is really important when the training samples are limited. Specifically, we propose two schemes: 1) a well-designed 1D-GAN as a spectral classifier and 2) a robust 3D-GAN as a spectral–spatial classifier. Furthermore, the generated adversarial samples are used with real training samples to fine-tune the discriminative CNN, which improves the final classification performance. The proposed classifiers are carried out on three widely used hyperspectral data sets: Salinas, Indiana Pines, and Kennedy Space Center. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods. In addition, the proposed GANs open new opportunities in the remote sensing community for the challenging task of HSI classification and also reveal the huge potential of GAN-based methods for the analysis of such complex and inherently nonlinear data. [ABSTRACT FROM AUTHOR]

The supervised deep networks have shown great potential in improving the classification performance. However, training these supervised deep networks is very challenging for hyperspectral image given the fact that usually only a small amount of labeled samples are available. In order to overcome this problem and enhance the discriminative ability of the network, in this paper, we propose a deep network architecture for a super-resolution (SR)-aided hyperspectral image classification with classwise loss (SRCL). First, a three-layer SR convolutional neural network (SRCNN) is employed to reconstruct a high-resolution image from a low-resolution image. Second, an unsupervised triplet-pipeline CNN (TCNN) with an improved classwise loss is built to encourage intraclass similarity and interclass dissimilarity. Finally, SRCNN, TCNN, and a classification module are integrated to define the SRCL, which can be fine-tuned in an end-to-end manner with a small amount of training data. Experimental results on real hyperspectral images demonstrate that the proposed SRCL approach outperforms other state-of-the-art classification methods, especially for the task in which only a small amount of training data are available. [ABSTRACT FROM AUTHOR]

It is a feasible and promising way to utilize deep neural networks to learn and extract valuable features from synthetic aperture radar (SAR) images for SAR automatic target recognition (ATR). However, it is too difficult to effectively train the deep neural networks with limited raw SAR images. In this paper, we propose a new approach to do SAR ATR, in which a multiview deep learning framework was employed. Based on the multiview SAR ATR pattern, we first present a flexible mean to generate adequate multiview SAR data, which can guarantee a large amount of inputs for network training without needing many raw SAR images. Then, a unique deep convolutional neural network containing a parallel network topology with multiple inputs is adopted. The features of input SAR images from different views will be learned by the proposed network layer by layer; meanwhile, the learned features from the distinct views are fused in different layers progressively. Therefore, the proposed framework is able to achieve a superior recognition performance, and requires only a small number of raw SAR images for network training samples generation. Experimental results have shown the superiority of the proposed framework based on the Moving and Stationary Target Acquisition and Recognition data set. [ABSTRACT FROM AUTHOR]

Deep learning based land cover classification algorithms have recently been proposed in the literature. In hyperspectral images (HSIs), they face the challenges of large dimensionality, spatial variability of spectral signatures, and scarcity of labeled data. In this paper, we propose an end-to-end deep learning architecture that extracts band specific spectral-spatial features and performs land cover classification. The architecture has fewer independent connection weights and thus requires fewer training samples. The method is found to outperform the highest reported accuracies on popular HSI data sets. [ABSTRACT FROM PUBLISHER]

Feature extraction is of significance for hyperspectral image (HSI) classification. Compared with conventional hand-crafted feature extraction, deep learning can automatically learn features with discriminative information. However, two issues exist in applying deep learning to HSIs. One issue is how to jointly extract spectral features and spatial features, and the other one is how to train the deep model when training samples are scarce. In this paper, a deep convolutional neural network with two-branch architecture is proposed to extract the joint spectral–spatial features from HSIs. The two branches of the proposed network are devoted to features from the spectral domain as well as the spatial domain. The learned spectral features and spatial features are then concatenated and fed to fully connected layers to extract the joint spectral–spatial features for classification. When the training samples are limited, we investigate the transfer learning to improve the performance. Low and mid-layers of the network are pretrained and transferred from other data sources; only top layers are trained with limited training samples extracted from the target scene. Experiments on Airborne Visible/Infrared Imaging Spectrometer and Reflective Optics System Imaging Spectrometer data demonstrate that the learned deep joint spectral–spatial features are discriminative, and competitive classification results can be achieved when compared with state-of-the-art methods. The experiments also reveal that the transferred features boost the classification performance. [ABSTRACT FROM PUBLISHER]

In this paper, we study self-taught learning for hyperspectral image (HSI) classification. Supervised deep learning methods are currently state of the art for many machine learning problems, but these methods require large quantities of labeled data to be effective. Unfortunately, existing labeled HSI benchmarks are too small to directly train a deep supervised network. Alternatively, we used self-taught learning, which is an unsupervised method to learn feature extracting frameworks from unlabeled hyperspectral imagery. These models learn how to extract generalizable features by training on sufficiently large quantities of unlabeled data that are distinct from the target data set. Once trained, these models can extract features from smaller labeled target data sets. We studied two self-taught learning frameworks for HSI classification. The first is a shallow approach that uses independent component analysis and the second is a three-layer stacked convolutional autoencoder. Our models are applied to the Indian Pines, Salinas Valley, and Pavia University data sets, which were captured by two separate sensors at different altitudes. Despite large variation in scene type, our algorithms achieve state-of-the-art results across all the three data sets. [ABSTRACT FROM PUBLISHER]

The deep convolutional neural network (CNN) is of great interest recently. It can provide excellent performance in hyperspectral image classification when the number of training samples is sufficiently large. In this paper, a novel pixel-pair method is proposed to significantly increase such a number, ensuring that the advantage of CNN can be actually offered. For a testing pixel, pixel-pairs, constructed by combining the center pixel and each of the surrounding pixels, are classified by the trained CNN, and the final label is then determined by a voting strategy. The proposed method utilizing deep CNN to learn pixel-pair features is expected to have more discriminative power. Experimental results based on several hyperspectral image data sets demonstrate that the proposed method can achieve better classification performance than the conventional deep learning-based method. [ABSTRACT FROM AUTHOR]

Due to the advantages of deep learning, in this paper, a regularized deep feature extraction (FE) method is presented for hyperspectral image (HSI) classification using a convolutional neural network (CNN). The proposed approach employs several convolutional and pooling layers to extract deep features from HSIs, which are nonlinear, discriminant, and invariant. These features are useful for image classification and target detection. Furthermore, in order to address the common issue of imbalance between high dimensionality and limited availability of training samples for the classification of HSI, a few strategies such as L2 regularization and dropout are investigated to avoid overfitting in class data modeling. More importantly, we propose a 3-D CNN-based FE model with combined regularization to extract effective spectral–spatial features of hyperspectral imagery. Finally, in order to further improve the performance, a virtual sample enhanced method is proposed. The proposed approaches are carried out on three widely used hyperspectral data sets: Indian Pines, University of Pavia, and Kennedy Space Center. The obtained results reveal that the proposed models with sparse constraints provide competitive results to state-of-the-art methods. In addition, the proposed deep FE opens a new window for further research. [ABSTRACT FROM AUTHOR]

This paper presents an automated algorithm for rapidly and effectively detecting cars directly from large-volume 3-D point clouds. Rather than using low-order descriptors, a multilayer feature generation model is created to obtain high-order feature representations for 3-D local patches through deep learning techniques. To handle cars with different levels of incompleteness caused by data acquisition ways and occlusions, a hierarchical visibility estimation model is developed to augment Hough voting. Considering scale and orientation variations in the azimuth direction, a set of multiscale Hough forests is constructed to rotationally cast votes to estimate cars' centroids. Quantitative assessments show that the proposed algorithm achieves average completeness, correctness, quality, and F1-measure of 0.94, 0.96, 0.90, and 0.95, respectively, in detecting 3-D cars. Comparative studies also demonstrate that the proposed algorithm outperforms the other four existing algorithms in accurately and completely detecting 3-D cars from large-scale 3-D point clouds. [ABSTRACT FROM AUTHOR]

This paper introduces the use of single-layer and deep convolutional networks for remote sensing data analysis. Direct application to multi- and hyperspectral imagery of supervised (shallow or deep) convolutional networks is very challenging given the high input data dimensionality and the relatively small amount of available labeled data. Therefore, we propose the use of greedy layerwise unsupervised pretraining coupled with a highly efficient algorithm for unsupervised learning of sparse features. The algorithm is rooted on sparse representations and enforces both population and lifetime sparsity of the extracted features, simultaneously. We successfully illustrate the expressive power of the extracted representations in several scenarios: classification of aerial scenes, as well as land-use classification in very high resolution or land-cover classification from multi- and hyperspectral images. The proposed algorithm clearly outperforms standard principal component analysis (PCA) and its kernel counterpart (kPCA), as well as current state-of-the-art algorithms of aerial classification, while being extremely computationally efficient at learning representations of data. Results show that single-layer convolutional networks can extract powerful discriminative features only when the receptive field accounts for neighboring pixels and are preferred when the classification requires high resolution and detailed results. However, deep architectures significantly outperform single-layer variants, capturing increasing levels of abstraction and complexity throughout the feature hierarchy. [ABSTRACT FROM PUBLISHER]

Ship detection on spaceborne images has attracted great interest in the applications of maritime security and traffic control. Optical images stand out from other remote sensing images in object detection due to their higher resolution and more visualized contents. However, most of the popular techniques for ship detection from optical spaceborne images have two shortcomings: 1) Compared with infrared and synthetic aperture radar images, their results are affected by weather conditions, like clouds and ocean waves, and 2) the higher resolution results in larger data volume, which makes processing more difficult. Most of the previous works mainly focus on solving the first problem by improving segmentation or classification with complicated algorithms. These methods face difficulty in efficiently balancing performance and complexity. In this paper, we propose a ship detection approach to solving the aforementioned two issues using wavelet coefficients extracted from JPEG2000 compressed domain combined with deep neural network (DNN) and extreme learning machine (ELM). Compressed domain is adopted for fast ship candidate extraction, DNN is exploited for high-level feature representation and classification, and ELM is used for efficient feature pooling and decision making. Extensive experiments demonstrate that, in comparison with the existing relevant state-of-the-art approaches, the proposed method requires less detection time and achieves higher detection accuracy. [ABSTRACT FROM PUBLISHER]