Search

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

Scene classification of remote sensing images has drawn great attention because of its wide applications. In this paper, with the guidance of the human visual system (HVS), we explore the attention mechanism and propose a novel end-to-end attention recurrent convolutional network (ARCNet) for scene classification. It can learn to focus selectively on some key regions or locations and just process them at high-level features, thereby discarding the noncritical information and promoting the classification performance. The contributions of this paper are threefold. First, we design a novel recurrent attention structure to squeeze high-level semantic and spatial features into several simplex vectors for the reduction of learning parameters. Second, an end-to-end network named ARCNet is proposed to adaptively select a series of attention regions and then to generate powerful predictions by learning to process them sequentially. Third, we construct a new data set named OPTIMAL-31, which contains more categories than popular data sets and gives researchers an extra platform to validate their algorithms. The experimental results demonstrate that our model makes great promotion in comparison with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Deep models with multiple layers have demonstrated their potential in learning abstract and invariant features for better representation and classification of remote sensing images. Moreover, metric learning (ML) is usually introduced into the deep models to further increase the discrimination of deep representations. However, the usual deep ML methods treat the training samples in each training batch in the stochastic gradient descent-based learning procedure independently, and thus, they neglect the important contextual (structural) information in the training samples. In this paper, we first introduce deep structural ML (DSML) into the literature of remote sensing scene classification and specifically capture and use the structural information during the training on the remote sensing images. Further analysis demonstrates that DSML usually makes many learned metric parameters similar. This similarity leads to obvious model redundancy and thus decreases the representational ability of the model. To address this problem, this paper proposes a new diversity-promoting DSML (D-DSML) method by regularizing the learning procedure by a diversity-promoting prior over the parameter factors. The proposed D-DSML encourages the parameter factors to be uncorrelated, such that each factor can model unique information, and thus, the model's description ability and classification performance would be significantly improved. Experiments over six real-world remote sensing scene data sets demonstrate that the proposed method obtains much better results than those obtained by the original deep models and has comparable or even better performances when compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Panchromatic (PAN) and multispectral (MS) imagery classification is one of the hottest topics in the field of remote sensing. In recent years, deep learning techniques have been widely applied in many areas of image processing. In this paper, an end-to-end learning framework based on deep multiple instance learning (DMIL) is proposed for MS and PAN images' classification using the joint spectral and spatial information based on feature fusion. There are two instances in the proposed framework: one instance is used to capture the spatial information of PAN and the other is used to describe the spectral information of MS. The features obtained by the two instances are concatenated directly, which can be treated as simple fusion features. To fully fuse the spatial-spectral information for further classification, the simple fusion features are fed into a fusion network with three fully connected layers to learn the high-level fusion features. Classification experiments carried out on four different airborne MS and PAN images indicate that the classifier provides feasible and efficient solution. It demonstrates that DMIL performs better than using a convolutional neural network and a stacked autoencoder network separately. In addition, this paper shows that the DMIL model can learn and fuse spectral and spatial information effectively, and has huge potential for MS and PAN imagery classification. [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) image classification is a vital application in remote sensing image processing. In general, PolSAR image classification is actually a high-dimensional nonlinear mapping problem. The methods based on sparse representation and deep learning have shown a great potential for PolSAR image classification. Therefore, a novel PolSAR image classification method based on multilayer projective dictionary pair learning (MDPL) and sparse auto encoder (SAE) is proposed in this paper. First, MDPL is used to extract features, and the abstract degree of the extracted features is high. Second, in order to get the nonlinear relationship between elements of feature vectors in an adaptive way, SAE is also used in this paper. Three PolSAR images are used to test the effectiveness of our method. Compared with several state-of-the-art methods, our method achieves very competitive results in PolSAR image classification. [ABSTRACT FROM PUBLISHER]

Change detection has been a hotspot in the remote sensing technology for a long time. With the increasing availability of multi-temporal remote sensing images, numerous change detection algorithms have been proposed. Among these methods, image transformation methods with feature extraction and mapping could effectively highlight the changed information and thus has a better change detection performance. However, the changes of multi-temporal images are usually complex, and the existing methods are not effective enough. In recent years, the deep network has shown its brilliant performance in many fields, including feature extraction and projection. Therefore, in this paper, based on the deep network and slow feature analysis (SFA) theory, we proposed a new change detection algorithm for multi-temporal remotes sensing images called deep SFA (DSFA). In the DSFA model, two symmetric deep networks are utilized for projecting the input data of bi-temporal imagery. Then, the SFA module is deployed to suppress the unchanged components and highlight the changed components of the transformed features. The change vector analysis pre-detection is employed to find unchanged pixels with high confidence as training samples. Finally, the change intensity is calculated with chi-square distance and the changes are determined by threshold algorithms. The experiments are performed on two real-world data sets and a public hyperspectral data set. The visual comparison and the quantitative evaluation have shown that DSFA could outperform the other state-of-the-art algorithms, including other SFA-based and deep learning methods. [ABSTRACT FROM AUTHOR]

For hyperspectral image classification, there is a large gap between the theoretical method and the practical application. Hyperspectral image classification in theoretical research trains a new classifier for each data set, which is ineffective and even infeasible in large-scale applications. In this paper, we make a preliminary attempt to recycle the classification model to new data sets in an unsupervised way. Specially, we propose a cross-data set hyperspectral image classification method based on deep domain adaptation. The proposed method contains three modules: domain alignment module that learns to minimize the domain discrepancy with the guide of an irrelevant task, task allocation module that learns to classify on the source domain with the regulation of domain alignment, and domain adaptation module that transfers both the alignment ability and classification ability to the target domain by an adaptation strategy. As a result, with the information of an irrelevant task on dual-domain data sets, we can minimize the domain discrepancy and transfer the task-relevant knowledge from the source domain to the target domain in an unsupervised way. Extensive experiments on three hyperspectral images demonstrate the effectiveness of our method compared with other related methods when dealing with new data sets. [ABSTRACT FROM AUTHOR]

This paper applies a machine learning (ML) algorithm based on the multi-hidden layer neural network (MHL-NN) for ocean surface wind speed estimation using global navigation satellite system (GNSS) reflection measurements. Unlike conventional wind speed retrieval methods that often depend on limited scalar delay-Doppler map (DDM) observables, the proposed MHL-NN makes use of information captured by the entire DDM. Both simulated and real data sets are used to train and evaluate the performance of the MHL-NN and compare it to a conventional wind speed retrieval method and other prevailing ML algorithms. The results show that the MHL-NN algorithm outperforms the other methods in terms of the root mean square error (RMSE) and mean absolute percentage error (MAPE) of the wind speed estimation. [ABSTRACT FROM AUTHOR]

Accurate and robust detection of multi-class objects in optical remote sensing images is essential to many real-world applications, such as urban planning, traffic control, searching, and rescuing. However, the state-of-the-art object detection techniques designed for images captured using ground-level sensors usually experience a sharp performance drop when directly applied to remote sensing images, largely due to the object appearance differences in remote sensing images in terms of sparse texture, low contrast, arbitrary orientations, and large-scale variations. This paper presents a novel object detection network [(context-aware detection network (CAD-Net)] that exploits attention-modulated features as well as global and local contexts to address the new challenges in detecting objects from remote sensing images. The proposed CAD-Net learns global and local contexts of objects by capturing their correlations with the global scene (at scene level) and the local neighboring objects or features (at object level), respectively. In addition, it designs a spatial-and-scale-aware attention module that guides the network to focus on more informative regions and features as well as more appropriate feature scales. Experiments over two publicly available object detection data sets for remote sensing images demonstrate that the proposed CAD-Net achieves superior detection performance. The implementation codes will be made publicly available for facilitating future works. [ABSTRACT FROM AUTHOR]

While deep learning technologies for computer vision have developed rapidly since 2012, modeling of remote sensing systems has remained focused around human vision. In particular, remote sensing systems are usually constructed to optimize sensing cost-quality tradeoffs with respect to human image interpretability. While some recent studies have explored remote sensing system design as a function of simple computer vision algorithm performance, there has been little work relating this design to the state of the art in computer vision: deep learning with convolutional neural networks. We develop experimental systems to conduct this analysis, showing results with modern deep learning algorithms and recent overhead image data. Our results are compared to standard image quality measurements based on human visual perception, and we conclude not only that machine and human interpretability differ significantly but also that computer vision performance is largely self-consistent across a range of disparate conditions. This paper is presented as a cornerstone for a new generation of sensor design systems that focus on computer algorithm performance instead of human visual perception. [ABSTRACT FROM AUTHOR]

Road detection and centerline extraction from very high-resolution (VHR) remote sensing imagery are of great significance in various practical applications. Road detection and centerline extraction operations depend on each other, to a certain extent. The road detection constrains the appearance of the centerline, and the centerline enhances the linear features of the road detection. However, most of the previous works have addressed these two tasks separately and have not considered the symbiotic relationship between them, making it difficult to obtain smooth and complete roads. In this paper, a novel multi-scale and multi-task deep learning framework for automatic road extraction (MSMT-RE) is proposed to build the relationship between them and simultaneously complete the road detection and centerline extraction tasks. U-Net is selected as the basic network for multi-task learning due to its strong ability to preserve spatial details. Multi-scale feature integration is also applied in the framework to increase the robustness of the feature extraction. Meanwhile, an adaptive loss function is introduced to solve the problems of roads taking up a small percentage of the training samples, and the fact that the positive samples of the two tasks are unbalanced. Finally, experiments were conducted on two public road data sets and two large images from Google Earth, and the proposed framework was compared with other state-of-the-art deep learning-based road extraction methods, both quantitatively and qualitatively. The proposed approach outperformed all the compared methods, confirming its advantages in automatic road extraction. [ABSTRACT FROM AUTHOR]

Frequency filtering is widely used in routine processing of seismic data to improve the signal-to-noise ratio (SNR) of recorded signals and by doing so to improve subsequent analyses. In this paper, we develop a new denoising/decomposition method, DeepDenoiser, based on a deep neural network. This network is able to simultaneously learn a sparse representation of data in the time–frequency domain and a non-linear function that maps this representation into masks that decompose input data into a signal of interest and noise (defined as any non-seismic signal). We show that DeepDenoiser achieves impressive denoising of seismic signals even when the signal and noise share a common frequency band. Because the noise statistics are automatically learned from data and require no assumptions, our method properly handles white noise, a variety of colored noise, and non-earthquake signals. DeepDenoiser can significantly improve the SNR with minimal changes in the waveform shape of interest, even in the presence of high noise levels. We demonstrate the effect of our method on improving earthquake detection. There are clear applications of DeepDenoiser to seismic imaging, micro-seismic monitoring, and preprocessing of ambient noise data. We also note that the potential applications of our approach are not limited to these applications or even to earthquake data and that our approach can be adapted to diverse signals and applications in other settings. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel unsupervised change detection method of remote sensing (RS) images based on a unified framework for weighted collaborative representation (WCR) with robust deep dictionary learning (RDDL) and fuzzy clustering. Specifically, WCR is employed to collaboratively represent neighborhood features with lower computational complexity, for which the RDDL model is built to learn more effective and representative overcomplete dictionary and enhance the robustness against the noise and outliers. Meanwhile, in order to make the resulting collaborative coefficients more beneficial for clustering, the unified framework for WCR with RDDL and fuzzy clustering is designed. By doing so, our framework not only precludes the utilization of third-party clustering algorithm, but also achieves better detection performance. Subsequently, the spatial constraint is enforced on the membership matrix to yield the updated one for further improving the accuracy of change detection. Finally, a binary change mask (CM) is achieved by assigning the pixels into the changed and unchanged classes. Experiments are performed on five pairs of RS images, and experimental results demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Recently, deep neural networks have received intense interests in polarimetric synthetic aperture radar (PolSAR) image classification. However, its success is subject to the availability of large amounts of annotated data which require great efforts of experienced human annotators. Aiming at improving the classification performance with greatly reduced annotation cost, this paper presents an active deep learning approach for minimally supervised PolSAR image classification, which integrates active learning and fine-tuned convolutional neural network (CNN) into a principled framework. Starting from a CNN trained using a very limited number of labeled pixels, we iteratively and actively select the most informative candidates for annotation, and incrementally fine-tune the CNN by incorporating the newly annotated pixels. Moreover, to boost the performance and robustness of the proposed method, we employ Markov random field (MRF) to enforce class label smoothness, and data augmentation technique to enlarge the training set. We conducted extensive experiments on four real benchmark PolSAR images, and experiments demonstrated that our approach achieved state-of-the-art classification results with significantly reduced annotation cost. [ABSTRACT FROM AUTHOR]

For hyperspectral remote sensing image (HSI) classification, the learning process of deep neural networks has been progressively advanced in depth, but the fine features are often largely lost or even disappear in the process of depth transfer. With the increase in feature aggregation and connectivity, the complexity of the network and the training parameters increases greatly, requiring more training time. This paper proposed a multi-scale dense network (MSDN) for HSI classification that made full use of different scale information in the network structure and combined scale information throughout the network. It implemented feature extraction of HSIs in two dimensions, including the features at fine and coarse levels. In the horizontal direction, it considered the deep extraction of HSI features, and the 3-D dense connection structure was used for aggregating features at different levels. In the vertical direction, scale information was considered, and three-scale feature maps at low, middle, and high levels were generated based on the first layer of the network. The MSDN used stride convolution for downsampling and combined feature information at different scale levels. The MSDN extracted features along the diagonal line. The network implemented the reconstruction of deep feature extraction and multi-scale fusion for HSI classification. The MSDN model performed well on representative HSI datasets, namely, the Indian Pines, Pavia University, Salinas, Botswana, and Kennedy Space Center datasets. It improved the training speed and accuracy for HSI classification and especially improved the convergence speed, which effectively saved computing resources and had high stability. [ABSTRACT FROM AUTHOR]

In order to better interpret polarimetric synthetic aperture radar (PolSAR) images, many scholars tend to do target decomposition for PolSAR images and utilize the obtained features to perform subsequent classification. These target decomposition features play an important role in terrain classification but completely utilizing them produces a high computational complexity. Furthermore, some features have a negative impact on the classification task. Therefore, selecting the appropriate amount of high-quality features is of great significance to the classification task. In this paper, we propose a convolutional neural network (CNN)-based feature selection algorithm for PolSAR image classification. First, we design a 1-D CNN for feature selection, then train the designed network with all the decomposition features to obtain a trained model. Second, the Kullback–Leibler distance (KLD) between different features is utilized as a standard to select feature subsets. Third, feature subsets with excellent performance form the final results. Due to the special structure of the 1-D CNN, repetitively training model is avoided when the input changes. Different from traditional feature selection methods, our method considers the performance of features combination rather than single feature contribution. To this end, the feature subsets selected by the proposed method are more useful to the classification task. Innovatively introducing KLD in the selection stage avoids random selection and improves the selection efficiency. Finally, we validate the performance of selected feature subsets in traditional and deep learning classification frameworks. Experiments demonstrate that features selected by the proposed method have a good performance comparing with others on three real PolSAR data sets. [ABSTRACT FROM AUTHOR]

Feature learning-based polarimetric synthetic aperture radar (PolSAR) classification model will generally suffer from the challenge of deficient labeled pixels. In this paper, we propose a novel generative-discriminative network for PolSAR polar-spatial feature fusion learning and classification, which comprises of a deep generative network and a discriminative network with their bottom layers shared. With this architecture, it enables to make use of both labeled and unlabeled pixels in a PolSAR image for model learning in a semisupervised way. Moreover, the proposed network imposes a Gaussian random field prior and a conditional random field posterior on the learned fusion features and the output label configuration, respectively. Without the need of the complicated recurrent iterations, our network can still efficiently produce the structured fusion feature as well as a smoothed classification map by involving some auxiliary variables, and it is specifically optimized via variational inference within an alternating direction method of multipliers iteration scheme. Extensive experiments on different benchmark PolSAR imageries demonstrate the effectiveness and superiority of the proposed network. Compared with other state-of-the-art algorithms of PolSAR feature learning and classification, our model can achieve a much better performance in terms of the visual quality of the label map and overall classification accuracy, facilitating the much less labeling pixels. [ABSTRACT FROM AUTHOR]

Point cloud classification is one of the most important and time-consuming stages of airborne LiDAR (Light Detection and Ranging) data processing, playing a key role in the generation of cartographic products. This paper describes an innovative algorithm to perform LiDAR point-cloud classification, which relies on Convolutional Neural Networks (CNNs) and takes advantage of full-waveform data registered by modern laser scanners. The proposed method consists of two steps. First, a simple CNN is used to preprocess each waveform, providing a compact representation of the data. By exploiting the coordinates of the points associated with the waveforms, output vectors generated by the first CNN are then mapped into an image that is subsequently segmented by a Fully Convolutional Network (FCN): a label is assigned to each pixel and, consequently, to the point falling in the pixel. In this way, spatial positions and geometrical relationships between neighboring data are taken into account. These particular architectures allow to accurately identify even challenging classes such as power line and transmission tower. [ABSTRACT FROM AUTHOR]

Currently, deep neural networks (DNNs) are an important method for handling hyperspectral image (HSI) classification because of their good performance in image processing. However, DNNs’ performance depends on a massive number of training data and hyperparameters that are carefully fine-tuned, which results in structural complexity and a time-consuming process. Deep forest is a novel deep learning method that does not need much training data and has a simple structure. In this paper, we first design a deep forest for spectral-based HSI classification and then propose an improved deep forest algorithm, named deep multigrained cascade forest (dgcForest), for spatial-based HSI classification. On the one hand, the cascade forest in dgcForest is used in multigrained scanning, which enhances the performance; on the other hand, a pooling layer is added after the multigrained scanning to reduce the output dimensions. To demonstrate that our proposed algorithm presents a good performance in HSI classification, we analyze the hyperparameters of deep forest and dgcForest and compare them with other methods on the biased and unbiased data sets, which illustrates that our method is superior to other state-of-the-art deep learning methods. [ABSTRACT FROM AUTHOR]

The automatic classification of 3-D point clouds is publicly known as a challenging task in a complex road environment. Specifically, each point is automatically classified into a unique category label, and then, the labels are used as clues for semantic analysis and scene recognition. Instead of heuristically extracting handcrafted features in traditional methods to classify all points, we put forward an end-to-end octree-based fully convolutional network (FCN) to classify 3-D point clouds in an urban road environment. There are four contributions in this paper. The first is that the integration and comprehensive uses of OctNet and FCN greatly decrease the computing time and memory demands compared with a dense 3-D convolutional neural network (CNN). The second is that the octree-based network is strengthened by means of modifying the cross-entropy loss function to solve the problems of an unbalanced category distribution. The third is that an Inception-ResNet block is united with our network, which enables our 3-D CNN to effectively learn how to classify scenes containing objects at multiple scales and improve classification accuracy. The last is that an open source data set (HuangshiRoad data set) with ten different classes is introduced for 3-D point cloud classification. Three representative data sets [Semantic3D, WHU_MLS (blocks I and II), and HuangshiRoad] with different covered areas and numbers of points and classes are selected to evaluate our proposed method. The experimental results show that the overall classification accuracy is appreciable, with 89.4% for Semantic3D, 82.9% for WHU_MLS block I, 91.4% for WHU_MLS block II, and 94% for HuangshiRoad. Our deep learning approach can efficiently classify 3-D dense point clouds in an urban road environment measured by a mobile laser scanning (MLS) system or static LiDAR. [ABSTRACT FROM AUTHOR]

Semantic segmentation requires methods capable of learning high-level features while dealing with large volume of data. Toward such goal, convolutional networks can learn specific and adaptable features based on the data. However, these networks are not capable of processing a whole remote sensing image, given its huge size. To overcome such limitation, the image is processed using fixed size patches. The definition of the input patch size is usually performed empirically (evaluating several sizes) or imposed (by network constraint). Both strategies suffer from drawbacks and could not lead to the best patch size. To alleviate this problem, several works exploited multicontext information by combining networks or layers. This process increases the number of parameters, resulting in a more difficult model to train. In this paper, we propose a novel technique to perform semantic segmentation of remote sensing images that exploits a multicontext paradigm without increasing the number of parameters while defining, in training time, the best patch size. The main idea is to train a dilated network with distinct patch sizes, allowing it to capture multicontext characteristics from heterogeneous contexts. While processing these varying patches, the network provides a score for each patch size, helping in the definition of the best size for the current scenario. A systematic evaluation of the proposed algorithm is conducted using four high-resolution remote sensing data sets with very distinct properties. Our results show that the proposed algorithm provides improvements in pixelwise classification accuracy when compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

As the spatial resolution of remote sensing images is improving gradually, it is feasible to realize “scene-object” collaborative image interpretation. Unfortunately, this idea is not fully utilized in vehicle detection from high-resolution aerial images, and most of the existing methods may be promoted by considering the variability of vehicle spatial distribution in different image scenes and treating vehicle detection tasks scene-specific. With this motivation, a scene context-driven vehicle detection method is proposed in this paper. At first, we perform scene classification using the deep learning method and, then, detect vehicles in roads and parking lots separately through different vehicle detectors. Afterward, we further optimize the detection results using different postprocessing rules according to different scene types. Experimental results show that the proposed approach outperforms the state-of-the-art algorithms in terms of higher detection accuracy rate and lower false alarm rate. [ABSTRACT FROM AUTHOR]

Road information extraction based on aerial images is a critical task for many applications, and it has attracted considerable attention from researchers in the field of remote sensing. The problem is mainly composed of two subtasks, namely, road detection and centerline extraction. Most of the previous studies rely on multistage-based learning methods to solve the problem. However, these approaches may suffer from the well-known problem of propagation errors. In this paper, we propose a novel deep learning model, recurrent convolution neural network U-Net (RCNN-UNet), to tackle the aforementioned problem. Our proposed RCNN-UNet has three distinct advantages. First, the end-to-end deep learning scheme eliminates the propagation errors. Second, a carefully designed RCNN unit is leveraged to build our deep learning architecture, which can better exploit the spatial context and the rich low-level visual features. Thereby, it alleviates the detection problems caused by noises, occlusions, and complex backgrounds of roads. Third, as the tasks of road detection and centerline extraction are strongly correlated, a multitask learning scheme is designed so that two predictors can be simultaneously trained to improve both effectiveness and efficiency. Extensive experiments were carried out based on two publicly available benchmark data sets, and nine state-of-the-art baselines were used in a comparative evaluation. Our experimental results demonstrate the superiority of the proposed RCNN-UNet model for both the road detection and the centerline extraction tasks. [ABSTRACT FROM AUTHOR]

The shortage of training samples remains one of the main obstacles in applying the neural networks to the hyperspectral images classification. To fuse the spatial and spectral information, pixel patches are often utilized to train a model, which may further aggregate this problem. In the existing works, an artificial neural network (ANN) model supervised by centerloss (ANNC) was introduced. Training merely with spectral information, the ANNC yields discriminative spectral features suitable for the subsequent classification tasks. In this paper, we propose a novel convolutional neural network (CNN)-based spatial feature fusion (CSFF) algorithm, which allows a smart integration of spatial information to the spectral features extracted by ANNC. As a critical part of CSFF, a CNN-based discriminant model is introduced to estimate whether two pixels belong to the same class. At the testing stage, by applying the discriminant model to the pixel pairs generated by a test pixel and each of its neighbors, the local structure is estimated and represented as a customized convolutional kernel. The spectral–spatial feature is generated by a convolutional operation between the estimated kernel and the corresponding spectral features within a local region. The final label is determined by classifying the resulting spectral–spatial feature. Without increasing the number of training samples or involving pixel patches at the training stage, the CSFF framework achieves the state of the art by declining 20%–50% classification failures in experiments on three well-known hyperspectral images. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is a core task in the remote sensing community, and recently, deep learning-based methods have shown their capability of accurate classification of HSIs. Among the deep learning-based methods, deep convolutional neural networks (CNNs) have been widely used for the HSI classification. In order to obtain a good classification performance, substantial efforts are required to design a proper deep learning architecture. Furthermore, the manually designed architecture may not fit a specific data set very well. In this paper, the idea of automatic CNN for the HSI classification is proposed for the first time. First, a number of operations, including convolution, pooling, identity, and batch normalization, are selected. Then, a gradient descent-based search algorithm is used to effectively find the optimal deep architecture that is evaluated on the validation data set. After that, the best CNN architecture is selected as the model for the HSI classification. Specifically, the automatic 1-D Auto-CNN and 3-D Auto-CNN are used as spectral and spectral–spatial HSI classifiers, respectively. Furthermore, the cutout is introduced as a regularization technique for the HSI spectral–spatial classification to further improve the classification accuracy. The experiments on four widely used hyperspectral data sets (i.e., Salinas, Pavia University, Kennedy Space Center, and Indiana Pines) show that the automatically designed data-dependent CNNs obtain competitive classification accuracy compared with the state-of-the-art methods. In addition, the automatic design of the deep learning architecture opens a new window for future research, showing the huge potential of using neural architectures’ optimization capabilities for the accurate HSI classification. [ABSTRACT FROM AUTHOR]

Supervised deep neural networks (DNNs) have been extensively used in diverse tasks. Generally, training such DNNs with superior performance requires a large amount of labeled data. However, it is time-consuming and expensive to manually label the data, especially for tasks in remote sensing, e.g., change detection. The situation motivates us to resort to the existing related images with labels, from which the concept of change can be adapted to new images. However, the distributions of the related labeled images (source domain) and unlabeled new images (target domain) are similar but not identical. It impedes a change detection model learned from source domains being well applied to the target domain. In this paper, we propose a transferred deep learning-based change detection framework to solve this problem. It consists of pretraining and fine-tuning stages. In the pretraining process, we propose two tasks to be learned simultaneously, namely, change detection for the source domain with labels and reconstruction of the unlabeled target data. The auxiliary task aims to reconstruct the difference image (DI) for the target domain. DI is an effective feature, such that the auxiliary task is of much relevance to change detection. The lower layers are shared between these two tasks in the training process. It mitigates the distribution discrepancy between the source and target domains and makes the concept of change from the source domain adapt to the target domain. In addition, we evaluate three modes of the U-net architecture to merge the information for a pair of patches. To fine-tune the change detection network (CDN) for the target domain, two strategies are exploited to select the pixels that have a high possibility of being correctly classified by an unsupervised approach. The proposed method demonstrates an excellent capacity for adapting the concept of change from the source domain to the target domain. It outperforms the state-of-the-art change detection methods via experimental results on real remote sensing data sets. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework that divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral–spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of training a deep neural network for satellite image segmentation so that it can be deployed over images whose statistics differ from those used for training. For example, in postdisaster damage assessment, the tight time constraints make it impractical to train a network from scratch for each image to be segmented. We propose a convolutional encoder–decoder network able to learn visual representations of increasing semantic level as its depth increases, allowing it to generalize over a wider range of satellite images. Then, we propose two additional methods to improve the network performance over each specific image to be segmented. First, we observe that updating the batch normalization layers’ statistics over the target image improves the network performance without human intervention. Second, we show that refining a trained network over a few samples of the image boosts the network performance with minimal human intervention. We evaluate our architecture over three data sets of satellite images, showing the state-of-the-art performance in binary segmentation of previously unseen images and competitive performance with respect to more complex techniques in a multiclass segmentation task. [ABSTRACT FROM AUTHOR]

Recently, hyperspectral image (HSI) classification approaches based on deep learning (DL) models have been proposed and shown promising performance. However, because of very limited available training samples and massive model parameters, DL methods may suffer from overfitting. In this paper, we propose an end-to-end 3-D lightweight convolutional neural network (CNN) (abbreviated as 3-D-LWNet) for limited samples-based HSI classification. Compared with conventional 3-D-CNN models, the proposed 3-D-LWNet has a deeper network structure, less parameters, and lower computation cost, resulting in better classification performance. To further alleviate the small sample problem, we also propose two transfer learning strategies: 1) cross-sensor strategy, in which we pretrain a 3-D model in the source HSI data sets containing a greater number of labeled samples and then transfer it to the target HSI data sets and 2) cross-modal strategy, in which we pretrain a 3-D model in the 2-D RGB image data sets containing a large number of samples and then transfer it to the target HSI data sets. In contrast to previous approaches, we do not impose restrictions over the source data sets, in which they do not have to be collected by the same sensors as the target data sets. Experiments on three public HSI data sets captured by different sensors demonstrate that our model achieves competitive performance for HSI classification compared to several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In the field of spatial–spectral fusion, the variational model-based methods and the deep learning (DL)-based methods are state-of-the-art approaches. This paper presents a fusion method that combines the deep neural network with a variational model for the most common case of spatial–spectral fusion: panchromatic (PAN)/multispectral (MS) fusion. Specifically, a deep residual convolutional neural network (CNN) is first trained to learn the gradient features of the high spatial resolution multispectral image (HR-MS). The image observation variational models are then formulated to describe the relationships of the ideal fused image, the observed low spatial resolution multispectral image (LR-MS) image, and the gradient priors learned before. Then, fusion result can then be obtained by solving the fusion variational model. Both quantitative and visual assessments on high-quality images from various sources demonstrate that the proposed fusion method is superior to all the mainstream algorithms included in the comparison, in terms of overall fusion accuracy. [ABSTRACT FROM AUTHOR]

The current superresolution (SR) methods based on deep learning have shown remarkable comparative advantages but remain unsatisfactory in recovering the high-frequency edge details of the images in noise-contaminated imaging conditions, e.g., remote sensing satellite imaging. In this paper, we propose a generative adversarial network (GAN)-based edge-enhancement network (EEGAN) for robust satellite image SR reconstruction along with the adversarial learning strategy that is insensitive to noise. In particular, EEGAN consists of two main subnetworks: an ultradense subnetwork (UDSN) and an edge-enhancement subnetwork (EESN). In UDSN, a group of 2-D dense blocks is assembled for feature extraction and to obtain an intermediate high-resolution result that looks sharp but is eroded with artifacts and noises as previous GAN-based methods do. Then, EESN is constructed to extract and enhance the image contours by purifying the noise-contaminated components with mask processing. The recovered intermediate image and enhanced edges can be combined to generate the result that enjoys high credibility and clear contents. Extensive experiments on Kaggle Open Source Data set, Jilin-1 video satellite images, and Digitalglobe show superior reconstruction performance compared to the state-of-the-art SR approaches. [ABSTRACT FROM AUTHOR]

Deep learning, especially a discriminative model for image denoising, has shown great potential in removing complex spectral–spatial noise in hyperspectral images (HSI). For HSI denoising, it is crucial to extract more context information around each pixel and to predict each pixel according to the surrounding context. Therefore, the effective receptive field plays an important role when performing denoising task. Generally, an HSI denoising model can achieve better performance by reserving the correlation of adjacent spectral bands and extracting more pixel features in the spatial domain. In this paper, 3-D atrous denoising convolution neural network (3DADCNN) is proposed for HSI. The model extracts feature maps along both spatial and spectral dimensions and enlarges the receptive field without significantly increasing the number of network parameters. Simultaneously, the multibranch and multiscale structure is utilized to reduce training difficulty, lessen overfitting risk, and preserve details in texture. The proposed model can be applied to the corrupted image with a mixed type of photon and thermal noise. Experimental results of the quantitative and qualitative evaluation show that 3DADCNN outperforms state-of-the-art HSI denoising methods. [ABSTRACT FROM AUTHOR]

Object detection is a basic issue of very high-resolution remote sensing images (RSIs) for automatically labeling objects. At present, deep learning has gradually gained the competitive advantage for remote sensing object detection, especially based on convolutional neural networks (CNNs). Most of the existing methods use the global information in the fully connected feature vector and ignore the local information in the convolutional feature cubes. However, the local information can provide spatial information, which is helpful for accurate localization. In addition, there are variable factors, such as rotation and scaling, which affect the object detection accuracy in RSIs. In order to solve these problems, this paper presents a hierarchical robust CNN. First, multiscale convolutional features are extracted to represent the hierarchical spatial semantic information. Second, multiple fully connected layer features are stacked together so as to improve the rotation and scaling robustness. Experiments on two data sets have shown the effectiveness of our method. In addition, a large-scale high-resolution remote sensing object detection data set is established to make up for the current situation that the existing data set is insufficient or too small. The data set is available at https://github.com/CrazyStoneonRoad/TGRS-HRRSD-Dataset. [ABSTRACT FROM AUTHOR]

Pan-sharpening is an important preprocessing step for remote sensing image processing tasks; it fuses a low-resolution multispectral image and a high-resolution (HR) panchromatic (PAN) image to reconstruct a HR multispectral (MS) image. This paper introduces a new end-to-end bidirectional pyramid network for pan-sharpening. The overall structure of the proposed network is a bidirectional pyramid, which permits the network to process MS and PAN images in two separate branches level by level. At each level of the network, spatial details extracted from the PAN image are injected into the upsampled MS image to reconstruct the pan-sharpened image from coarse resolution to fine resolution. Subpixel convolutional layers and the enhanced residual blocks are used to make the network efficient. Comparison of the results obtained with our proposed method and the results using other widely used state-of-the-art approaches confirms that our proposed method outperforms the others in visual appearance and objective indexes. [ABSTRACT FROM AUTHOR]

With the rapid development of various technologies of satellite sensor, synthetic aperture radar (SAR) image has been an import source of data in the application of change detection. In this paper, a novel method based on a convolutional neural network (CNN) for SAR image change detection is proposed. The main idea of our method is to generate the classification results directly from the original two SAR images through a CNN without any preprocessing operations, which also eliminate the process of generating the difference image (DI), thus reducing the influence of the DI on the final classification result. In CNN, the spatial characteristics of the raw image can be extracted and captured by automatic learning and the results with stronger robustness can be obtained. The basic idea of the proposed method includes three steps: it first produces false labels through unsupervised spatial fuzzy clustering. Then we train the CNN through proper samples that are selected from the samples with false labels. Finally, the final detection results are obtained by the trained convolutional network. Although training the convolutional network is a supervised learning fashion, the whole process of the algorithm is an unsupervised process without priori knowledge. The theoretical analysis and experimental results demonstrate the validity, robustness, and potential of our algorithm in simulated and real data sets. In addition, we try to apply our algorithm to the change detection of heterogeneous images, which also achieves satisfactory results. [ABSTRACT FROM AUTHOR]

Deep learning artificial neural network techniques can be applied for on-orbit calibration of microwave and millimeter-wave radiometer spaceborne instruments, including those for small satellites. The noise-wave model has been employed for noise characterization and validation of the proposed deep learning calibration technique for a synthetically generated Dicke-switching radiometer. The developed deep learning neural network radiometer calibrator produces high accuracy estimates of antenna temperatures from the measurements of radiometer output voltage and thermistor readings. Tests with noise-free and noisy samples of the developed model have shown that the proposed calibration method does not add any significant noise to the radiometer calibration. The performance of the proposed method does not degrade with increased nonlinearity for a radiometer, while nonlinearity is a challenging issue for conventional calibration techniques. The deep learning calibration model learns the radiometer noise characteristics from radiometer prelaunch measurements during thermal vacuum chamber testing. The neural network calibrator proposed in this paper has self-learning capability during the on-orbit operation of a radiometer that can be used to improve the performance of on-orbit calibration. The proposed technique is demonstrated by comparing the residual uncertainty of the deep learning calibration with the theoretical value. No numerical study is presented to compare the performance with conventional calibration techniques. The new method may be solely applied to calibrate the radiometer or applied along with conventional calibration techniques. [ABSTRACT FROM AUTHOR]

By considering the spectral signature as a sequence, recurrent neural networks (RNNs) have been successfully used to learn discriminative features from hyperspectral images (HSIs) recently. However, most of these models only input the whole spectral bands into RNNs directly, which may not fully explore the specific properties of HSIs. In this paper, we propose a cascaded RNN model using gated recurrent units to explore the redundant and complementary information of HSIs. It mainly consists of two RNN layers. The first RNN layer is used to eliminate redundant information between adjacent spectral bands, while the second RNN layer aims to learn the complementary information from nonadjacent spectral bands. To improve the discriminative ability of the learned features, we design two strategies for the proposed model. Besides, considering the rich spatial information contained in HSIs, we further extend the proposed model to its spectral–spatial counterpart by incorporating some convolutional layers. To test the effectiveness of our proposed models, we conduct experiments on two widely used HSIs. The experimental results show that our proposed models can achieve better results than the compared models. [ABSTRACT FROM AUTHOR]

Recently, deep learning-based methods have brought new ideas for ship detection in synthetic aperture radar (SAR) images. However, several challenges still exist: 1) deep models contain millions of parameters, whereas the available annotated samples are not sufficient in number for training. Therefore, most deep detectors have to fine-tune networks pre-trained on ImageNet, which incurs learning bias due to the huge domain mismatch between SAR images and ImageNet images. Furthermore, it has a little flexibility to redesign the network structure; and 2) ships in SAR images are relatively small in size and densely clustered, whereas most deep detectors have poor performance with small objects due to the rough feature map used for detection and the extreme foreground–background imbalance. To address these problems, this paper proposes an effective approach to learn deep ship detector from scratch. First, we design a condensed backbone network, which consists of several dense blocks. Hence, earlier layers can receive additional supervision from the objective function through the dense connections, which makes it easy to train. In addition, feature reuse strategy is adopted to make it highly parameter efficient. Therefore, the backbone network could be freely designed and effectively trained from scratch without using a large amount of annotated samples. Second, we improve the cross-entropy loss to address the foreground–background imbalance and predict multi-scale ship proposals from several intermediate layers to improve the recall rate. Then, position-sensitive score maps are adopted to encode position information into each ship proposal for discrimination. The comparison results on the Sentinel-1 data set show that: 1) learning ship detector from scratch achieved better performance than ImageNet pre-trained model-based detectors and 2) our method is more effective than existing algorithms for detecting the small and densely clustered ships. [ABSTRACT FROM AUTHOR]

Recently, deep learning (DL) is of great interest in hyperspectral image (HSI) classification. Although many effective frameworks exist in the literature, the generally limited availability of training samples poses great challenges in applying DL to HSI classification. In this paper, we present a novel DL framework, namely, semisupervised stacked autoencoders (Semi-SAEs) with cotraining, for HSI classification. First, two SAEs are pretrained based on the hyperspectral features and the spatial features, respectively. Second, fine-tuning is alternatively conducted for the two SAEs in a semisupervised cotraining fashion, where the initial training set is enlarged by designing an effective region growing method. Finally, the classification probabilities obtained by the two SAEs are fused using a Markov random field model solved by iterated conditional modes. Experimental results based on three popular hyperspectral data sets demonstrate that the proposed method outperforms other state-of-the-art DL methods. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has been successfully applied to single image super-resolution (SISR), which aims at reconstructing a high-resolution (HR) image from its low-resolution (LR) counterpart. Different from most current DL-based methods, which perform reconstruction in the spatial domain, we use a scheme based in the frequency domain to reconstruct the HR image at various frequency bands. Further, we propose a method that incorporates the wavelet transform (WT) and the recursive Res-Net. The WT is applied to the LR image to divide it into various frequency components. Then, an elaborately designed network with recursive residual blocks is used to predict high-frequency components. Finally, the reconstructed image is obtained via the inverse WT. This paper has three main contributions: 1) an SISR scheme based on the frequency domain is proposed under a DL framework to fully exploit the potential to depict images at different frequency bands; 2) recursive block and residual learning in global and local manners are adopted to ease the training of the deep network, and the batch normalization layer is removed to increase the flexibility of the network, save memory, and promote speed; and 3) the low-frequency wavelet component is replaced by an LR image with more details to further improve performance. To validate the effectiveness of the proposed method, extensive experiments are performed using the NWPU-RESISC45 data set, and the results demonstrate that the proposed method outperforms several state-of-the-art methods in terms of both objective evaluation and subjective perspective. [ABSTRACT FROM AUTHOR]

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial–spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min–max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen–Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Aiming at improving the classification accuracy with limited numbers of labeled pixels in polarimetric synthetic aperture radar (PolSAR) image classification task, this paper presents a graph-based semisupervised deep learning model for PolSAR image classification. It models the PolSAR image as an undirected graph, where the nodes correspond to the labeled and unlabeled pixels, and the weighted edges represent similarities between the pixels. Upon the graph, we design an energy function incorporating a semisupervision term, a convolutional neural network (CNN) term, and a pairwise smoothness term. The employed CNN extracts abstract and data-driven polarimetric features and outputs class label predictions to the graph model. The semisupervision term enforces the category label constraints on the human-labeled pixels. The pairwise smoothness term encourages class label smoothness and the alignment of class label boundaries with the image edges. Starting from an initialized class label map generated based on $K$ -Wishart distribution hypothesis or superpixel segmentation of PauliRGB images, we iteratively and alternately optimize the defined energy function until it converges. We conducted experiments on two real benchmark PolSAR images, and extensive experiments demonstrated that our approach achieved the state-of-the-art results for PolSAR image classification. [ABSTRACT FROM AUTHOR]

In this paper, we propose a spectral-spatial feature extraction and classification framework based on an artificial neuron network in the context of hyperspectral imagery. With limited labeled samples, only spectral information is exploited for training and spatial context is integrated posteriorly at the testing stage. Taking advantage of recent advances in face recognition, a joint supervision symbol that combines softmax loss and center loss is adopted to train the proposed network, by which intraclass features are gathered while interclass variations are enlarged. Based on the learned architecture, the extracted spectrum-based features are classified by a center classifier. Moreover, to fuse the spectral and spatial information, an adaptive spectral-spatial center classifier is developed, where multiscale neighborhoods are considered simultaneously, and the final label is determined using an adaptive voting strategy. Finally, experimental results on three well-known data sets validate the effectiveness of the proposed methods compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Deep learning has recently attracted significant attention in the field of hyperspectral images (HSIs) classification. However, the construction of an efficient deep neural network mostly relies on a large number of labeled samples being available. To address this problem, this paper proposes a unified deep network, combined with active transfer learning (TL) that can be well-trained for HSIs classification using only minimally labeled training data. More specifically, deep joint spectral–spatial feature is first extracted through hierarchical stacked sparse autoencoder (SSAE) networks. Active TL is then exploited to transfer the pretrained SSAE network and the limited training samples from the source domain to the target domain, where the SSAE network is subsequently fine-tuned using the limited labeled samples selected from both source and target domains by the corresponding active learning (AL) strategies. The advantages of our proposed method are threefold: 1) the network can be effectively trained using only limited labeled samples with the help of novel AL strategies; 2) the network is flexible and scalable enough to function across various transfer situations, including cross data set and intraimage; and 3) the learned deep joint spectral–spatial feature representation is more generic and robust than many joint spectral–spatial feature representations. Extensive comparative evaluations demonstrate that our proposed method significantly outperforms many state-of-the-art approaches, including both traditional and deep network-based methods, on three popular data sets. [ABSTRACT FROM AUTHOR]

Satellite images are widely applied in humanitarian mapping that labels buildings, roads, and so on for humanitarian aid and economic development. However, the labeling now is mostly done by volunteers. In this paper, we utilize deep learning to solve humanitarian mapping tasks of a mobile software named MapSwipe. The current deep learning techniques, e.g., convolutional neural network (CNN), can recognize ground objects from satellite images but rely on numerous labels for training for each specific task. We solve this problem by fusing multiple freely accessible crowdsourced geographic data and propose an active learning-based CNN training framework named MC-CNN to deal with the quality issues of the labels extracted from these data, including incompleteness (e.g., some kinds of object are not labeled) and heterogeneity (e.g., different spatial granularities). The method is evaluated with building mapping in South Malawi and road mapping in Guinea with level-18 satellite images provided by Bing Map and volunteered geographic information from OpenStreetMap, MapSwipe, and OsmAnd. The results based on multiple metrics, including Precision, Recall, F1 Score, and area under the receiver operating characteristic curve, show that MC-CNN can fuse the crowdsourced labels for higher prediction performance and be successfully applied in MapSwipe for humanitarian mapping with 85% labor saved and an overall accuracy of 0.86 achieved. [ABSTRACT FROM AUTHOR]

Image classification is considered to be one of the critical tasks in hyperspectral remote sensing image processing. Recently, a convolutional neural network (CNN) has established itself as a powerful model in classification by demonstrating excellent performances. The use of a graphical model such as a conditional random field (CRF) contributes further in capturing contextual information and thus improving the classification performance. In this paper, we propose a method to classify hyperspectral images by considering both spectral and spatial information via a combined framework consisting of CNN and CRF. We use multiple spectral band groups to learn deep features using CNN, and then formulate deep CRF with CNN-based unary and pairwise potential functions to effectively extract the semantic correlations between patches consisting of 3-D data cubes. Furthermore, we introduce a deep deconvolution network that improves the final classification performance. We also introduced a new data set and experimented our proposed method on it along with several widely adopted benchmark data sets to evaluate the effectiveness of our method. By comparing our results with those from several state-of-the-art models, we show the promising potential of our method. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) denoising is a crucial preprocessing procedure to improve the performance of the subsequent HSI interpretation and applications. In this paper, a novel deep learning-based method for this task is proposed, by learning a nonlinear end-to-end mapping between the noisy and clean HSIs with a combined spatial–spectral deep convolutional neural network (HSID-CNN). Both the spatial and spectral information are simultaneously assigned to the proposed network. In addition, multiscale feature extraction and multilevel feature representation are, respectively, employed to capture both the multiscale spatial–spectral feature and fuse different feature representations for the final restoration. The simulated and real-data experiments demonstrate that the proposed HSID-CNN outperforms many of the mainstream methods in both the quantitative evaluation indexes, visual effects, and HSI classification accuracy. [ABSTRACT FROM AUTHOR]

Object detection and semantic segmentation are two main themes in object retrieval from high-resolution remote sensing images, which have recently achieved remarkable performance by surfing the wave of deep learning and, more notably, convolutional neural networks. In this paper, we are interested in a novel, more challenging problem of vehicle instance segmentation, which entails identifying, at a pixel level, where the vehicles appear as well as associating each pixel with a physical instance of a vehicle. In contrast, vehicle detection and semantic segmentation each only concern one of the two. We propose to tackle this problem with a semantic boundary-aware multitask learning network. More specifically, we utilize the philosophy of residual learning to construct a fully convolutional network that is capable of harnessing multilevel contextual feature representations learned from different residual blocks. We theoretically analyze and discuss why residual networks can produce better probability maps for pixelwise segmentation tasks. Then, based on this network architecture, we propose a unified multitask learning network that can simultaneously learn two complementary tasks, namely, segmenting vehicle regions and detecting semantic boundaries. The latter subproblem is helpful for differentiating “touching” vehicles that are usually not correctly separated into instances. Currently, data sets with a pixelwise annotation for vehicle extraction are the ISPRS data set and the IEEE GRSS DFC2015 data set over Zeebrugge, which specializes in a semantic segmentation. Therefore, we built a new, more challenging data set for vehicle instance segmentation, called the Busy Parking Lot Unmanned Aerial Vehicle Video data set, and we make our data set available at http://www.sipeo.bgu.tum.de/downloads so that it can be used to benchmark future vehicle instance segmentation algorithms. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is an active and important research task driven by many practical applications. To leverage deep learning models especially convolutional neural networks (CNNs) for HSI classification, this paper proposes a simple yet effective method to extract hierarchical deep spatial feature for HSI classification by exploring the power of off-the-shelf CNN models, without any additional retraining or fine-tuning on the target data set. To obtain better classification accuracy, we further propose a unified metric learning-based framework to alternately learn discriminative spectral–spatial features, which have better representation capability and train support vector machine (SVM) classifiers. To this end, we design a new objective function that explicitly embeds a metric learning regularization term into SVM training. The metric learning regularization term is used to learn a powerful spectral–spatial feature representation by fusing spectral feature and deep spatial feature, which has small intraclass scatter but big between class separation. By transforming HSI data into new spectral–spatial feature space through CNN and metric learning, we can pull the pixels from the same class closer, while pushing the different class pixels farther away. In the experiments, we comprehensively evaluate the proposed method on three commonly used HSI benchmark data sets. State-of-the-art results are achieved when compared with the existing HSI classification methods. [ABSTRACT FROM AUTHOR]

Classification of very high-resolution (VHR) satellite images has three major challenges: 1) inherent low intraclass and high interclass spectral similarities; 2) mismatching resolution of available bands; and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and postclassification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this paper, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet, aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet. We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusions, e.g., pansharpening and resampling through interpolation and map regularization such as conditional random fields. We carried out our experiments on a land-cover classification task using a Worldview-03 image of Quezon City, Philippines, and the International Society for Photogrammetry and Remote Sensing 2-D semantic labeling benchmark data set of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both the quantitative and qualitative results. [ABSTRACT FROM AUTHOR]