Search

Deep neural networks (DNNs), including convolutional neural networks (CNNs) and residual networks (ResNets) models, are able to learn abstract representations from the input data by considering a deep hierarchy of layers that perform advanced feature extraction. The combination of these models with visual attention techniques can assist with the identification of the most representative parts of the data from a visual standpoint, obtained through more detailed filtering of the features extracted by the operational layers of the network. This is of significant interest for analyzing remotely sensed hyperspectral images (HSIs), characterized by their very high spectral dimensionality. However, few efforts have been conducted in the literature in order to adapt visual attention methods to remotely sensed HSI data analysis. In this paper, we introduce a new visual attention-driven technique for the HSI classification. Specifically, we incorporate attention mechanisms to a ResNet in order to better characterize the spectral–spatial information contained in the data. Our newly proposed method calculates a mask that is applied to the features obtained by the network in order to identify the most desirable ones for classification purposes. Our experiments, conducted using four widely used HSI data sets, reveal that the proposed deep attention model provides competitive advantages in terms of classification accuracy when compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

This paper proposes a complex-valued convolutional neural network for land form classification and discovery in interferometric synthetic aperture radar (InSAR). Since the amount of satellite-borne SAR data has been increasing drastically, it is necessary to structurize the local features contained in observation data prior to utilization in the so-called big data framework for higher usability. Convolutional neural networks have such potential in general. However, there exists no network that can deal with complex amplitude data obtained in InSAR consistently. In this paper, we propose a complex-valued convolutional neural network to deal with InSAR. We demonstrate that the network classifies slopes and plains adaptively and, moreover, indicates small volcanos similar to a sample volcano (Omuroyama) included in the InSAR data. We also find their characteristic features emerging in the kernels in the convolution layers. These results reveal that the proposed complex-valued convolutional neural network is capable of successfully discovering unidentified lands similar to a prepared sample, which is highly useful for the InSAR data structurization. [ABSTRACT FROM AUTHOR]

Recently, researchers have shown the powerful ability of deep methods with multilayers to extract high-level features and to obtain better performance for hyperspectral image classification. However, a common problem of traditional deep models is that the learned deep models might be suboptimal because of the limited number of training samples, especially for the image with large intraclass variance and low interclass variance. In this paper, novel convolutional neural networks (CNNs) with multiscale convolution (MS-CNNs) are proposed to address this problem by extracting deep multiscale features from the hyperspectral image. Moreover, deep metrics usually accompany with MS-CNNs to improve the representational ability for the hyperspectral image. However, the usual metric learning would make the metric parameters in the learned model tend to behave similarly. This similarity leads to obvious model’s redundancy and, thus, shows negative effects on the description ability of the deep metrics. Traditionally, determinantal point process (DPP) priors, which encourage the learned factors to repulse from one another, can be imposed over these factors to diversify them. Taking advantage of both the MS-CNNs and DPP-based diversity-promoting deep metrics, this paper develops a CNN with multiscale convolution and diversified metric to obtain discriminative features for hyperspectral image classification. Experiments are conducted over four real-world hyperspectral image data sets to show the effectiveness and applicability of the proposed method. Experimental results show that our method is better than original deep models and can produce comparable or even better classification performance in different hyperspectral image data sets with respect to spectral and spectral–spatial features. [ABSTRACT FROM AUTHOR]

This paper is devoted to solving a full-wave inverse scattering problem (ISP), which is aimed at retrieving permittivities of dielectric scatterers from the knowledge of measured scattering data. ISPs are highly nonlinear due to multiple scattering, and iterative algorithms with regularizations are often used to solve such problems. However, they are associated with heavy computational cost, and consequently, they are often time-consuming. This paper proposes the convolutional neural network (CNN) technique to solve full-wave ISPs. We introduce and compare three training schemes based on U-Net CNN, including direct inversion, backpropagation, and dominant current schemes (DCS). Several representative tests are carried out, including both synthetic and experimental data, to evaluate the performances of the proposed methods. It is demonstrated that the proposed DCS outperforms the other two schemes in terms of accuracy and is able to solve typical ISPs quickly within 1 s. The proposed deep-learning inversion scheme is promising in providing quantitative images in real time. [ABSTRACT FROM AUTHOR]

We recently proposed a convolutional neural network (CNN) for remote sensing image pansharpening obtaining a significant performance gain over the state of the art. In this paper, we explore a number of architectural and training variations to this baseline, achieving further performance gains with a lightweight network that trains very fast. Leveraging on this latter property, we propose a target-adaptive usage modality that ensures a very good performance also in the presence of a mismatch with respect to the training set and even across different sensors. The proposed method, published online as an off-the-shelf software tool, allows users to perform fast and high-quality CNN-based pansharpening of their own target images on general-purpose hardware. [ABSTRACT FROM AUTHOR]

Compression of hyperspectral images onboard of spacecrafts is a tradeoff between the limited computational resources and the ever-growing spatial and spectral resolution of the optical instruments. As such, it requires low-complexity algorithms with good rate-distortion performance and high throughput. In recent years, the Consultative Committee for Space Data Systems (CCSDS) has focused on lossless and near-lossless compression approaches based on predictive coding, resulting in the recently published CCSDS 123.0-B-2 recommended standard. While the in-loop reconstruction of quantized prediction residuals provides excellent rate-distortion performance for the near-lossless operating mode, it significantly constrains the achievable throughput due to data dependencies. In this paper, we study the performance of a faster method based on the prequantization of the image followed by a lossless predictive compressor. While this is well known to be suboptimal, one can exploit powerful signal models to reconstruct the image at the ground segment, recovering part of the suboptimality. In particular, we show that convolutional neural networks can be used for this task and that they can recover the whole SNR drop incurred at a bit rate of 2 bits per pixel. [ABSTRACT FROM AUTHOR]

Spatiotemporal image fusion is considered as a promising way to provide Earth observations with both high spatial resolution and frequent coverage, and recently, learning-based solutions have been receiving broad attention. However, these algorithms treating spatiotemporal fusion as a single image super-resolution problem, generally suffers from the significant spatial information loss in coarse images, due to the large upscaling factors in real applications. To address this issue, in this paper, we exploit temporal information in fine image sequences and solve the spatiotemporal fusion problem with a two-stream convolutional neural network called StfNet. The novelty of this paper is twofold. First, considering the temporal dependence among image sequences, we incorporate the fine image acquired at the neighboring date to super-resolve the coarse image at the prediction date. In this way, our network predicts a fine image not only from the structural similarity between coarse and fine image pairs but also by exploiting abundant texture information in the available neighboring fine images. Second, instead of estimating each output fine image independently, we consider the temporal relations among time-series images and formulate a temporal constraint. This temporal constraint aiming to guarantee the uniqueness of the fusion result and encourages temporal consistent predictions in learning and thus leads to more realistic final results. We evaluate the performance of the StfNet using two actual data sets of Landsat-Moderate Resolution Imaging Spectroradiometer (MODIS) acquisitions, and both visual and quantitative evaluations demonstrate that our algorithm achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

The accurate reconstruction of areas obscured by clouds is among the most challenging topics for the remote sensing community since a significant percentage of images archived throughout the world are affected by cloud covers which make them not fully exploitable. The purpose of this paper is to propose new methods to recover missing data in multispectral images due to the presence of clouds by relying on a formulation based on an autoencoder (AE) neural network. We suppose that clouds are opaque and their detection is performed by dedicated algorithms. The AE in our methods aims at modeling the relationship between a given cloud-free image (source image) and a cloud-contaminated image (target image). In particular, two strategies are developed: the first one performs the mapping at a pixel level while the second one at a patch level to take profit from spatial contextual information. Moreover, in order to fix the problem of the hidden layer size, a new solution combining the minimum descriptive length criterion and a Pareto-like selection procedure is introduced. The results of experiments conducted on three different data sets are reported and discussed together with a comparison with reference techniques. [ABSTRACT FROM AUTHOR]

Supervised approaches classify input data using a set of representative samples for each class, known as training samples. The collection of such samples is expensive and time demanding. Hence, unsupervised feature learning, which has a quick access to arbitrary amounts of unlabeled data, is conceptually of high interest. In this paper, we propose a novel network architecture, fully Conv-Deconv network, for unsupervised spectral-spatial feature learning of hyperspectral images, which is able to be trained in an end-to-end manner. Specifically, our network is based on the so-called encoder-decoder paradigm, i.e., the input 3-D hyperspectral patch is first transformed into a typically lower dimensional space via a convolutional subnetwork (encoder), and then expanded to reproduce the initial data by a deconvolutional subnetwork (decoder). However, during the experiment, we found that such a network is not easy to be optimized. To address this problem, we refine the proposed network architecture by incorporating: 1) residual learning and 2) a new unpooling operation that can use memorized max-pooling indexes. Moreover, to understand the "black box," we make an in-depth study of the learned feature maps in the experimental analysis. A very interesting discovery is that some specific "neurons" in the first residual block of the proposed network own good description power for semantic visual patterns in the object level, which provide an opportunity to achieve "free" object detection. This paper, for the first time in the remote sensing community, proposes an end-to-end fully Conv-Deconv network for unsupervised spectral-spatial feature learning. Moreover, this paper also introduces an in-depth investigation of learned features. Experimental results on two widely used hyperspectral data, Indian Pines and Pavia University, demonstrate competitive performance obtained by the proposed methodology compared with other studied approaches. [ABSTRACT FROM AUTHOR]

Interferograms obtained by interferometric synthetic aperture radar generally include many singular points (SPs) originating from interference distortion and noise in the measurement. The filtering process is one of the key techniques in the generation of an accurate digital elevation model (DEM). In this paper, we propose a filter to remove SPs and related distortion using a complex-valued neural network in the spectral domain. It removes SPs nonlinearly and adaptively by referring to the statistics in neighboring windows in the 2-D frequency domain, where the textural features are represented in a more continuous manner than in the real space domain. Experiments demonstrate that the proposed method removes SPs and the distortion in SP-constructing four pixels, namely, the singular unit, more effectively than the conventional filters, resulting in the generation of a more accurate DEM. [ABSTRACT FROM AUTHOR]

Interferograms obtained by synthetic aperture radar often include many singular points (SPs), which makes it difficult to generate an accurate digital elevation model. This paper proposes a filtering method to compensate SPs adaptively by using polarization and phase information around the SPs. Phase value is essentially related to polarization changes in scattering as well as propagation. In order to handle the polarization and phase information simultaneously in a consistent manner, we define a new number, phasor quaternion (PQ), by combining quaternion and complex amplitude, with which we construct the theory of PQ neural networks (PQNNs). Experiments demonstrate that the proposed PQNN filter compensates SPs very effectively. Even in the situations where the conventional methods deteriorate in their performance, it realizes accurate compensation, thanks to its good generalization characteristics in integrated Poincare-sphere polarization space and the complex-amplitude space. We find that PQNN is an excellent framework to deal with the polarization and phase of electromagnetic wave adaptively and consistently. [ABSTRACT FROM AUTHOR]

This paper addresses the highly challenging problem of automatically detecting man-made structures especially buildings in very high-resolution (VHR) synthetic aperture radar (SAR) images. In this context, this paper has two major contributions. First, it presents a novel and generic workflow that initially classifies the spaceborne SAR tomography (TomoSAR) point clouds—generated by processing VHR SAR image stacks using advanced interferometric techniques known as TomoSAR—into buildings and nonbuildings with the aid of auxiliary information (i.e., either using openly available 2-D building footprints or adopting an optical image classification scheme) and later back project the extracted building points onto the SAR imaging coordinates to produce automatic large-scale benchmark labeled (buildings/nonbuildings) SAR data sets. Second, these labeled data sets (i.e., building masks) have been utilized to construct and train the state-of-the-art deep fully convolution neural networks with an additional conditional random field represented as a recurrent neural network to detect building regions in a single VHR SAR image. Such a cascaded formation has been successfully employed in computer vision and remote sensing fields for optical image classification but, to our knowledge, has not been applied to SAR images. The results of the building detection are illustrated and validated over a TerraSAR-X VHR spotlight SAR image covering approximately 39 km 2—almost the whole city of Berlin— with the mean pixel accuracies of around 93.84%. [ABSTRACT FROM AUTHOR]

Change detection is one of the central problems in earth observation and was extensively investigated over recent decades. In this paper, we propose a novel recurrent convolutional neural network (ReCNN) architecture, which is trained to learn a joint spectral–spatial–temporal feature representation in a unified framework for change detection in multispectral images. To this end, we bring together a convolutional neural network and a recurrent neural network into one end-to-end network. The former is able to generate rich spectral-spatial feature representations, while the latter effectively analyzes temporal dependence in bitemporal images. In comparison with previous approaches to change detection, the proposed network architecture possesses three distinctive properties: 1) it is end-to-end trainable, in contrast to most existing methods whose components are separately trained or computed; 2) it naturally harnesses spatial information that has been proven to be beneficial to change detection task; and 3) it is capable of adaptively learning the temporal dependence between multitemporal images, unlike most of the algorithms that use fairly simple operation like image differencing or stacking. As far as we know, this is the first time that a recurrent convolutional network architecture has been proposed for multitemporal remote sensing image analysis. The proposed network is validated on real multispectral data sets. Both visual and quantitative analyses of the experimental results demonstrate competitive performance in the proposed mode. [ABSTRACT FROM AUTHOR]

The classification of hyperspectral images (HSIs) using convolutional neural networks (CNNs) has recently drawn significant attention. However, it is important to address the potential overfitting problems that CNN-based methods suffer when dealing with HSIs. Unlike common natural images, HSIs are essentially three-order tensors which contain two spatial dimensions and one spectral dimension. As a result, exploiting both spatial and spectral information is very important for HSI classification. This paper proposes a new hand-crafted feature extraction method, based on multiscale covariance maps (MCMs), that is specifically aimed at improving the classification of HSIs using CNNs. The proposed method has the following distinctive advantages. First, with the use of covariance maps, the spatial and spectral information of the HSI can be jointly exploited. Each entry in the covariance map stands for the covariance between two different spectral bands within a local spatial window, which can absorb and integrate the two kinds of information (spatial and spectral) in a natural way. Second, by means of our multiscale strategy, each sample can be enhanced with spatial information from different scales, increasing the information conveyed by training samples significantly. To verify the effectiveness of our proposed method, we conduct comprehensive experiments on three widely used hyperspectral data sets, using a classical 2-D CNN (2DCNN) model. Our experimental results demonstrate that the proposed method can indeed increase the robustness of the CNN model. Moreover, the proposed MCMs+2DCNN method exhibits better classification performance than other CNN-based classification strategies and several standard techniques for spectral-spatial classification of HSIs. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel weakly supervised semantic segmentation (WSSS) method that uses image tags as supervision to achieve joint pixel-level localization of the key local structure (KLS) and image-level classification of the aurora images captured by the ground-based optical all-sky imager. First, a patch-scale model (PSM) based on the small-scale structure of aurora is designed to identify the type-specific regions for each training image. Second, a region-scale model is trained with the identified type-specific regions to coarsely localize the KLS from multiple sizes of field of view, based on which the aurora image is classified. Finally, given the predicted image type, the PSM further refines the KLS in a pixel level. By localizing KLS from coarse to fine, the proposed method captures both overall shape with a bottom–up processing and local structure details of aurora in a top–down manner. Extensive experiments on the expert labeled data sets have demonstrated the efficacy of the proposed method in benchmarking with the state-of-the-art WSSS methods. [ABSTRACT FROM AUTHOR]

This paper presents a new neural network (NN) algorithm for real-time retrievals of low amounts of precipitable water vapor (PWV) and integrated liquid water from millimeter-wave ground-based observations. Measurements are collected by the 183.3-GHz G-band vapor radiometer (GVR) operating at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility, Barrow, AK. The NN provides the means to explore the nonlinear regime of the measurements and investigate the physical boundaries of the operability of the instrument. A methodology to compute individual error bars associated with the NN output is developed, and a detailed error analysis of the network output is provided. Through the error analysis, it is possible to isolate several components contributing to the overall retrieval errors and to analyze the dependence of the errors on the inputs. The network outputs and associated errors are then compared with results from a physical retrieval and with the ARM two-channel microwave radiometer (MWR) statistical retrieval. When the NN is trained with a seasonal training data set, the retrievals of water vapor yield results that are comparable to those obtained from a traditional physical retrieval, with a retrieval error percentage of ~5% when the PWV is between 2 and l0 mm, but with the advantages that the NN algorithm does not require vertical profiles of temperature and humidity as input and is significantly faster computationally. Liquid water path (LWP) retrievals from the NN have a significantly improved clear-sky bias (mean of ~2.4 g/m²) and a retrieval error varying from 1 to about 10 g/ms when the PWV amount is between 1 and 10 mm. As an independent validation of the LWP retrieval, the Iongwave downwelling surface flux was computed and compared with observations. The comparison shows a significant improvement with respect to the MWR statistical retrievals, particularly for LWP amounts of less than 60 g/m². This paper shows that the GVR alone can provide overall improved PWV and LWP retrievals when the PWV amount is less than 10 mm, and, when combined with the MWR, can provide improved retrievals over the whole water-vapor range. [ABSTRACT FROM AUTHOR]

Contemporary surveillance systems mainly use video cameras as their primary sensor. However, video cameras possess fundamental deficiencies, such as the inability to handle low-light environments, poor weather conditions, and concealing clothing. In contrast, radar devices are able to sense in pitch-dark environments and to see through walls. In this paper, we investigate the use of micro-Doppler (MD) signatures retrieved from a low-power radar device to identify a set of persons based on their gait characteristics. To that end, we propose a robust feature learning approach based on deep convolutional neural networks. Given that we aim at providing a solution for a real-world problem, people are allowed to walk around freely in two different rooms. In this setting, the IDentification with Radar data data set is constructed and published, consisting of 150 min of annotated MD data equally spread over five targets. Through experiments, we investigate the effectiveness of both the Doppler and time dimension, showing that our approach achieves a classification error rate of 24.70% on the validation set and 21.54% on the test set for the five targets used. When experimenting with larger time windows, we are able to further lower the error rate. [ABSTRACT FROM AUTHOR]

The reconstruction of urban buildings from large-scale airborne laser scanning point clouds is an important research topic in the geoscience field. Large-scale urban scenes usually contain a large number of object categories and many overlapped or closely neighboring objects, which poses great challenges for classifying and modeling buildings from these data sets. In this paper, we propose a deep reinforcement learning framework that integrates a 3-D convolutional neural network, a deep Q-network, and a residual recurrent neural network for the efficient semantic parsing of large-scale 3-D point clouds. The proposed framework provides an end-to-end automatic processing method that maps the raw point cloud to the classification results of the given categories. After obtaining the building classes, we utilize an edge-aware resampling algorithm to consolidate the point set with noise-free normals and clean preservation of sharp features. Finally, 2.5-D dual contouring, which is a data-driven approach, is introduced to generate urban building models from the consolidated point clouds. Our method can generate lightweight building models with arbitrarily shaped roofs while preserving the verticality of connecting walls. [ABSTRACT FROM AUTHOR]

Global Navigation Satellite System (GNSS) radio occultation (RO) has been widely used in the prediction of weather, climate, and space weather, particularly in the area of tropospheric analyses. However, one of the issues with GNSS RO measurements is that they are interfered with by the signals reflected from the earth's surface. Many RO events are subject to such interfered GNSS measurements, which are considerably difficult to extract from the GNSS RO measurements. To precisely identify interfered RO events, an improved machine learning approach--a gradient descent artificial neural network (ANN)-aided radio-holography method--is proposed in this paper. Since this method is more complex than most other machine learning methods, for improving its efficiency through the reduction in computational time for near-real-time applications, a scale factor and a regularization factor are also adjusted in the ANN approach. This approach was validated using Constellation Observing System for Meteorology, Ionosphere, and Climate/FC-3 atmPhs (level 1b) data during the period of day of year 172-202, 2015, and its detection results were compared with the flag data set provided by Radio Occultation Meteorology Satellite Application Facilities for the performance assessment and validation of the new approach. The results were also compared with those of the support vector machine method for improvement assessment. The comparison results showed that the proposed method can considerably improve both the success rate of GNSS RO reflection detection and the computational efficiency. [ABSTRACT FROM AUTHOR]

Deep models with multiple layers have demonstrated their potential in learning abstract and invariant features for better representation and classification of remote sensing images. Moreover, metric learning (ML) is usually introduced into the deep models to further increase the discrimination of deep representations. However, the usual deep ML methods treat the training samples in each training batch in the stochastic gradient descent-based learning procedure independently, and thus, they neglect the important contextual (structural) information in the training samples. In this paper, we first introduce deep structural ML (DSML) into the literature of remote sensing scene classification and specifically capture and use the structural information during the training on the remote sensing images. Further analysis demonstrates that DSML usually makes many learned metric parameters similar. This similarity leads to obvious model redundancy and thus decreases the representational ability of the model. To address this problem, this paper proposes a new diversity-promoting DSML (D-DSML) method by regularizing the learning procedure by a diversity-promoting prior over the parameter factors. The proposed D-DSML encourages the parameter factors to be uncorrelated, such that each factor can model unique information, and thus, the model's description ability and classification performance would be significantly improved. Experiments over six real-world remote sensing scene data sets demonstrate that the proposed method obtains much better results than those obtained by the original deep models and has comparable or even better performances when compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper proposes a multi-scale rotation-invariant haar-like (MSRI-HL) feature integrated convolutional neural network (MSRIHL-CNN)-based ship detection algorithm of the multiple-target environment in synthetic aperture radar (SAR) imagery. Usually, ship detection includes preprocessing, prescreening, discrimination, and classification. Among them, prescreening and discrimination are the most two important stages so that they catch great intention. Based on our previous work, we propose a truncated-clutter-statistics-based joint, constant false alarm rate (CFAR) detector (TCS-JCFAR) for ship target prescreening in the multiple-target environment. TCS-JCFAR greatly enhances the prescreening rate in the multiple-target environment while achieving a low observed FAR. In the discrimination stage, conventional CNN extracts the deep features (high-level features); however, it will lose the local texture and edge information (low-level features) which are of great significance for target discrimination. Hence, the MSRI-HL features are used to represent the multi-scale, rotation-invariant texture, and edge information that conventional CNN fails to capture. The extracted low-level MSRI-HL features and the high-level deep features are optimally fused to a multi-layered feature vector. Finally, the multi-layered feature vector is fed into a typical support vector machine (SVM) classifier for ship target discrimination. The proposed MSRIHL-CNN combines the low-level texture and edge features and the high-level deep features; moreover, they are optimally fused to fully represent the ship targets. Undoubtedly, MSRIHL-CNN has better discrimination performance. The superiority of the proposed TCS-JCFAR-based prescreener and MSRIHL-CNN-based discriminator is validated on the Chinese Gaofen-3 SAR imagery. [ABSTRACT FROM AUTHOR]

Single- and dual-polarimetric synthetic aperture radar (SAR) images provide very limited capabilities to interpret physical radar signatures. For generality and simplicity, we call single-polarimetric, dual-polarimetric, and fully polarimetric SAR (PolSAR) images flexible PolSAR images. In order to sufficiently extract physical scattering signatures from this kind of data and explore the potentials of different polarization modes on this task, this paper proposes a contrastive-regulated convolutional neural network (CNN) in the complex domain, attempting to learn a physically interpretable deep learning model directly from the original backscattered data. To achieve a better deep model containing physically interpretable parameters, the objective cost is compared to and selected from several commonly used loss functions in the complex form. The required ground-truth labels are generated automatically according to Cloude and Pottier’s H-alpha division plane, which significantly reduces intensive labor cost and transfers this method to an unsupervised learning mechanism. The boundaries between different scattering signatures, however, sometimes show an erroneous separation. With the aim of aggregating intra-class instances and alienating inter-class instances, meanwhile, a complex-valued contrastive regularization term is computed mathematically and is added to the objective cost by a tradeoff factor. Moreover, data augmentation is applied to relieve the side effects caused by data imbalance. Finally, we performed experiments on German Aerospace Center’s (DLR)’s L-band, high-resolution (HR), and airborne F-SAR data. Our results demonstrate the possibility of extracting physical scattering signatures from flexible PolSAR images. Physically interpretable potentials of SAR images with different polarization modes are analyzed, and we conclude with physical signature identification. [ABSTRACT FROM AUTHOR]

Frequency filtering is widely used in routine processing of seismic data to improve the signal-to-noise ratio (SNR) of recorded signals and by doing so to improve subsequent analyses. In this paper, we develop a new denoising/decomposition method, DeepDenoiser, based on a deep neural network. This network is able to simultaneously learn a sparse representation of data in the time–frequency domain and a non-linear function that maps this representation into masks that decompose input data into a signal of interest and noise (defined as any non-seismic signal). We show that DeepDenoiser achieves impressive denoising of seismic signals even when the signal and noise share a common frequency band. Because the noise statistics are automatically learned from data and require no assumptions, our method properly handles white noise, a variety of colored noise, and non-earthquake signals. DeepDenoiser can significantly improve the SNR with minimal changes in the waveform shape of interest, even in the presence of high noise levels. We demonstrate the effect of our method on improving earthquake detection. There are clear applications of DeepDenoiser to seismic imaging, micro-seismic monitoring, and preprocessing of ambient noise data. We also note that the potential applications of our approach are not limited to these applications or even to earthquake data and that our approach can be adapted to diverse signals and applications in other settings. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN)-based methods have been successfully applied to SAR target detection. Different from prevalently used detection approaches with rectangle bounding box, rotatable bounding box (RBox)-based methods, such as DRBox-v1, can effectively reduce the interference of background pixels and locate the targets more finely for geospatial object detection. Although DRBox-v1 has achieved impressive detected performance, there still exist some remaining problems and room for improvement. In this paper, an improved RBox-based target detection framework is proposed to boost precision and recall rates of detection, and we refer to the method as DRBox-v2 and apply it to target detection in SAR images. The main improvements of DRBox-v2 as well as the contributions of this paper are fourfold. First, a multi-layer prior box generation strategy is designed for detecting small-scale targets. Since shallow layers lack strong sematic information, the feature pyramid network (FPN) module is applied. Second, a modified encoding scheme for RBox is proposed for more precisely estimating the position of RBox and orientation of targets. Third, a focal loss (FL) combined with hard negative mining (HNM) technique is proposed to mitigate the issue of the imbalance between positive and negative samples, which produces better results than solely employing either one. Fourth, comprehensive ablation studies are conducted to reveal the effect of each improvement on detected results. The results of the target detection on three data sets are illustrated and our method obtains 0.135, 0.081, 0.115 gains in average precision compared with three state-of-the-art methods, respectively. [ABSTRACT FROM AUTHOR]

This paper proposes a novel framework for fusing multi-temporal, multispectral satellite images and OpenStreetMap (OSM) data for the classification of local climate zones (LCZs). Feature stacking is the most commonly used method of data fusion but does not consider the heterogeneity of multimodal optical images and OSM data, which becomes its main drawback. The proposed framework processes two data sources separately and then combines them at the model level through two fusion models (the landuse fusion model and building fusion model) that aim to fuse optical images with landuse and buildings layers of OSM data, respectively. In addition, a new approach to detecting building incompleteness of OSM data is proposed. The proposed framework was trained and tested using the data from the 2017 IEEE GRSS Data Fusion Contest and further validated on one additional test (AT) set containing test samples that are manually labeled in Munich and New York. The experimental results have indicated that compared with the feature stacking-based baseline framework, the proposed framework is effective in fusing optical images with OSM data for the classification of LCZs with high generalization capability on a large scale. The classification accuracy of the proposed framework outperforms the baseline framework by more than 6% and 2% while testing on the test set of 2017 IEEE GRSS Data Fusion Contest and the AT set, respectively. In addition, the proposed framework is less sensitive to spectral diversities of optical satellite images and thus achieves more stable classification performance than the state-of-the-art frameworks. [ABSTRACT FROM AUTHOR]

Point cloud classification is one of the most important and time-consuming stages of airborne LiDAR (Light Detection and Ranging) data processing, playing a key role in the generation of cartographic products. This paper describes an innovative algorithm to perform LiDAR point-cloud classification, which relies on Convolutional Neural Networks (CNNs) and takes advantage of full-waveform data registered by modern laser scanners. The proposed method consists of two steps. First, a simple CNN is used to preprocess each waveform, providing a compact representation of the data. By exploiting the coordinates of the points associated with the waveforms, output vectors generated by the first CNN are then mapped into an image that is subsequently segmented by a Fully Convolutional Network (FCN): a label is assigned to each pixel and, consequently, to the point falling in the pixel. In this way, spatial positions and geometrical relationships between neighboring data are taken into account. These particular architectures allow to accurately identify even challenging classes such as power line and transmission tower. [ABSTRACT FROM AUTHOR]

Remote sensing image scene classification is a fundamental problem, which aims to label an image with a specific semantic category automatically. Recent progress on remote sensing image scene classification is substantial, benefitting mostly from the powerful feature extraction capability of convolutional neural networks (CNNs). Even though these CNN-based methods have achieved competitive performances, they only construct the representation of the image in location-sensitive space-domain. As a result, their representations are not robust to rotation-variant remote sensing images, which influence the classification accuracy. In this paper, we propose a novel feature representation method by introducing a frequency-domain branch to the traditional only-space-domain architecture. Our framework takes full advantages of discriminative features from space domain and location-robust features from the frequency domain, providing more advanced representations through an additional joint learning module, a property that is critically needed to perform remote sensing image scene classification. Additionally, our method produces satisfactory performances on four public and challenging remote sensing image scene data sets, Sydney, UC-Merced, WHU-RS19, and AID. [ABSTRACT FROM AUTHOR]

In this paper, we propose a spatial–spectral feature fusion model with a predictive feature weighting mechanism and demonstrate its applications to the problems of hyperspectral image classification and segmentation. To address these problems, we learn a set of 1-D convolutional local spectral filters and 2-D spatial–spectral filters that feed features into a fusion module, in an end-to-end fashion. We propose a lightweight predictive feature weighting component embedded in the fusion model and consider four design fusion options, i.e., by adding or concatenating features with equal or predicted weights. For the pixel classification task, the training input consists of image patches with labeled central pixels, whereas for the spatial segmentation task, it includes the label maps of image regions. The proposed networks have been evaluated on the Indian Pines, Pavia University, and Houston University for the classification problem and the SpaceNet data set for the spatial segmentation problem. The quantitative results favor the proposed approach over the state-of-the-art methods across all the four data sets. [ABSTRACT FROM AUTHOR]

In this paper, we study the effectiveness of learning temporal features to improve detection performance in videos captured by small aircraft. To implement this learning process, we use a convolutional long short-term memory (LSTM) associated with a pretrained convolutional neural network (CNN). To improve the training process, we incorporate domain-specific knowledge about the expected size and number of boats. We carry out three tests. The first searches the best sequence length and subsampling rate for training and the second compares the proposed method with a traditional CNN, a traditional LSTM, and a gated recurrent unit (GRU). The final test evaluates our method with the already published detectors in two data sets. Results show that in favorable conditions, our method’s performance is comparable to other detectors but, on more challenging environments, it stands out from other techniques. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of training a deep neural network for satellite image segmentation so that it can be deployed over images whose statistics differ from those used for training. For example, in postdisaster damage assessment, the tight time constraints make it impractical to train a network from scratch for each image to be segmented. We propose a convolutional encoder–decoder network able to learn visual representations of increasing semantic level as its depth increases, allowing it to generalize over a wider range of satellite images. Then, we propose two additional methods to improve the network performance over each specific image to be segmented. First, we observe that updating the batch normalization layers’ statistics over the target image improves the network performance without human intervention. Second, we show that refining a trained network over a few samples of the image boosts the network performance with minimal human intervention. We evaluate our architecture over three data sets of satellite images, showing the state-of-the-art performance in binary segmentation of previously unseen images and competitive performance with respect to more complex techniques in a multiclass segmentation task. [ABSTRACT FROM AUTHOR]

In the field of spatial–spectral fusion, the variational model-based methods and the deep learning (DL)-based methods are state-of-the-art approaches. This paper presents a fusion method that combines the deep neural network with a variational model for the most common case of spatial–spectral fusion: panchromatic (PAN)/multispectral (MS) fusion. Specifically, a deep residual convolutional neural network (CNN) is first trained to learn the gradient features of the high spatial resolution multispectral image (HR-MS). The image observation variational models are then formulated to describe the relationships of the ideal fused image, the observed low spatial resolution multispectral image (LR-MS) image, and the gradient priors learned before. Then, fusion result can then be obtained by solving the fusion variational model. Both quantitative and visual assessments on high-quality images from various sources demonstrate that the proposed fusion method is superior to all the mainstream algorithms included in the comparison, in terms of overall fusion accuracy. [ABSTRACT FROM AUTHOR]

Deep learning artificial neural network techniques can be applied for on-orbit calibration of microwave and millimeter-wave radiometer spaceborne instruments, including those for small satellites. The noise-wave model has been employed for noise characterization and validation of the proposed deep learning calibration technique for a synthetically generated Dicke-switching radiometer. The developed deep learning neural network radiometer calibrator produces high accuracy estimates of antenna temperatures from the measurements of radiometer output voltage and thermistor readings. Tests with noise-free and noisy samples of the developed model have shown that the proposed calibration method does not add any significant noise to the radiometer calibration. The performance of the proposed method does not degrade with increased nonlinearity for a radiometer, while nonlinearity is a challenging issue for conventional calibration techniques. The deep learning calibration model learns the radiometer noise characteristics from radiometer prelaunch measurements during thermal vacuum chamber testing. The neural network calibrator proposed in this paper has self-learning capability during the on-orbit operation of a radiometer that can be used to improve the performance of on-orbit calibration. The proposed technique is demonstrated by comparing the residual uncertainty of the deep learning calibration with the theoretical value. No numerical study is presented to compare the performance with conventional calibration techniques. The new method may be solely applied to calibrate the radiometer or applied along with conventional calibration techniques. [ABSTRACT FROM AUTHOR]

This paper presents spiking neural networks (SNNs) for remote sensing spatiotemporal analysis of image time series, which make use of the highly parallel and low-power-consuming neuromorphic hardware platforms possible. This paper illustrates this concept with the introduction of the first SNN computational model for crop yield estimation from normalized difference vegetation index image time series. It presents the development and testing of a methodological framework which utilizes the spatial accumulation of time series of Moderate Resolution Imaging Spectroradiometer 250-m resolution data and historical crop yield data to train an SNN to make timely prediction of crop yield. The research work also includes an analysis on the optimum number of features needed to optimize the results from our experimental data set. The proposed approach was applied to estimate the winter wheat (Triticum aestivum L.) yield in Shandong province, one of the main winter-wheat-growing regions of China. Our method was able to predict the yield around six weeks before harvest with a very high accuracy. Our methodology provided an average accuracy of 95.64%, with an average error of prediction of 0.236 t/ha and correlation coefficient of 0.801 based on a nine-feature model. [ABSTRACT FROM PUBLISHER]

Turbulence has been one of the major concerns for aviation safety. This paper applies evolutionary machine learning (ML) technology to turbulence level classification for civil aviation. An artificial neural network ML approach based on radar observation is developed for classifying the cubed root of the Eddy Dissipation Rate (EDR)1/3, an accepted measure of turbulence intensity. The approach is validated using Typhoon weather data collected by Hong Kong Observatory’s Terminal Doppler Weather Radar (TDWR) located near Hong Kong International Airport and comparing TDWR EDR1/3 detections and predictions with in situ EDR1/3 measured by commercial aircraft. The testing results verified that the ML approach performs reasonably well for both detecting and predicting tasks. [ABSTRACT FROM AUTHOR]

This paper implements a novel approach to automatically detect and classify rockfalls in Lunar Reconnaissance Orbiter narrow angle camera (NAC) images using a single-stage dense object detector (RetinaNet). The convolutional neural network has been trained with a data set of 2932 original rockfall images. In order to avoid overfitting, the initial training data set has been augmented during training using random image rotation, scaling, and flipping. Testing images have been labelled by human operators and have been used for RetinaNet performance evaluation. Testing shows that RetinaNet is capable to reach recall values between 0.98 and 0.39, precision values between 1 and 0.25, and average precisions ranging from 0.89 to 0.69, depending on the used confidence threshold and intersection-over-union values. Mean processing time of a single NAC image in RetinaNet is around 10 s using a GeForce GTX 1080 Ti and GeForce Titan Xp, which is in orders of magnitudes faster than a human operator. The processing speed allows to efficiently exploit the currently available NAC data stack with more than 1 million images in a reasonable timeframe. The combination of speed and detection performance can be used to produce lunar rockfall distribution maps on large spatial scales for utilization by the scientific and engineering community. [ABSTRACT FROM AUTHOR]

The knowledge about the placement and appearance of lane markings is a prerequisite for the creation of maps with high precision, necessary for autonomous driving, infrastructure monitoring, lanewise traffic management, and urban planning. Lane markings are one of the important components of such maps. Lane markings convey the rules of roads to drivers. While these rules are learned by humans, an autonomous driving vehicle should be taught to learn them to localize itself. Therefore, accurate and reliable lane-marking semantic segmentation in the imagery of roads and highways is needed to achieve such goals. We use airborne imagery that can capture a large area in a short period of time by introducing an aerial lane marking data set. In this paper, we propose a symmetric fully convolutional neural network enhanced by wavelet transform in order to automatically carry out lane-marking segmentation in aerial imagery. Due to a heavily unbalanced problem in terms of a number of lane-marking pixels compared with background pixels, we use a customized loss function as well as a new type of data augmentation step. We achieve a high accuracy in pixelwise localization of lane markings compared with the state-of-the-art methods without using the third-party information. In this paper, we introduce the first high-quality data set used within our experiments, which contains a broad range of situations and classes of lane markings representative of today’s transportation systems. This data set will be publicly available, and hence, it can be used as the benchmark data set for future algorithms within this domain. [ABSTRACT FROM AUTHOR]

In multiclass classification, convolutional neural network (CNN) is generally coupled with the cross-entropy (CE) loss, which only penalizes the predicted probability corresponding to a ground truth class and ignores the interclass relationship. We argue that CNN can be improved by using a better loss function. On the other hand, the Wasserstein distance (WD) is a well-known metric used to measure the distance between two distributions. Directly solving the WD problem requires a prohibitively large amount of computation time, whereas the cheaper iterative algorithms have a variety of shortcomings such as computational instability and difficulty in selecting parameters. In this paper, we address these issues by giving an analytical solution to the WD problem—for the first time, we find that for two distributions in hierarchically organized data space, WD has a closed-form solution, which we call “hierarchical WD (HWD).” We use this theory to construct novel loss functions that overcome the shortcomings of CE loss. To this end, multi-CNN information fusion that provides the basis for building category hierarchies is carried out first. Then, the semantic relationship among classes is modeled as a binary tree. Then, CNN coupled with an HWD-based loss, i.e., hierarchical Wasserstein CNN (HW-CNN), is trained to learn deep features. In this way, prior knowledge about the interclass relationship is embedded into HW-CNN, and information from several CNNs provides guidance in the process of training individual HW-CNNs. We conducted extensive experiments over two publicly available remote sensing data sets and achieved a state-of-the-art performance in scene classification tasks. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNN-based network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral–spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral–spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton’s capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral–spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral–spatial capsules, which are able to learn HSI spectral–spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted using five well-known HSI data sets and several state-of-the-art classification methods, reveal that our HSI classification approach based on spectral–spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time. [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) image classification plays an important role in remote sensing image processing. In recent years, stacked auto-encoder (SAE) has obtained a series of excellent results in PolSAR image classification. The recently proposed projective dictionary pair learning (DPL) model takes both accuracy and time consumption into consideration, and another recently proposed semicoupled dictionary learning (SCDL) model gives a new way to fit different features. Based on the SAE, DPL, and SCDL models, we propose a novel semicoupled projective DPL method with SAE (SAE-SDPL) for PolSAR image classification. Our method can get the classification result efficiently and correctly and meanwhile giving a new method to fit different features. In this paper, three PolSAR images are used to test the performance of SAE-SDPL. Compared with some state-of-the-art methods, our method obtains excellent results in PolSAR image classification. [ABSTRACT FROM AUTHOR]

Deep learning methods have recently been successfully explored for hyperspectral image (HSI) classification. However, training a deep-learning classifier notoriously requires hundreds or thousands of labeled samples. In this paper, a deep few-shot learning method is proposed to address the small sample size problem of HSI classification. There are three novel strategies in the proposed algorithm. First, spectral–spatial features are extracted to reduce the labeling uncertainty via a deep residual 3-D convolutional neural network. Second, the network is trained by episodes to learn a metric space where samples from the same class are close and those from different classes are far. Finally, the testing samples are classified by a nearest neighbor classifier in the learned metric space. The key idea is that the designed network learns a metric space from the training data set. Furthermore, such metric space could generalize to the classes of the testing data set. Note that the classes of the testing data set are not seen in the training data set. Four widely used HSI data sets were used to assess the performance of the proposed algorithm. The experimental results indicate that the proposed method can achieve better classification accuracy than the conventional semisupervised methods with only a few labeled samples. [ABSTRACT FROM AUTHOR]

Aiming at improving the classification accuracy with limited numbers of labeled pixels in polarimetric synthetic aperture radar (PolSAR) image classification task, this paper presents a graph-based semisupervised deep learning model for PolSAR image classification. It models the PolSAR image as an undirected graph, where the nodes correspond to the labeled and unlabeled pixels, and the weighted edges represent similarities between the pixels. Upon the graph, we design an energy function incorporating a semisupervision term, a convolutional neural network (CNN) term, and a pairwise smoothness term. The employed CNN extracts abstract and data-driven polarimetric features and outputs class label predictions to the graph model. The semisupervision term enforces the category label constraints on the human-labeled pixels. The pairwise smoothness term encourages class label smoothness and the alignment of class label boundaries with the image edges. Starting from an initialized class label map generated based on $K$ -Wishart distribution hypothesis or superpixel segmentation of PauliRGB images, we iteratively and alternately optimize the defined energy function until it converges. We conducted experiments on two real benchmark PolSAR images, and extensive experiments demonstrated that our approach achieved the state-of-the-art results for PolSAR image classification. [ABSTRACT FROM AUTHOR]

Deep learning has recently attracted significant attention in the field of hyperspectral images (HSIs) classification. However, the construction of an efficient deep neural network mostly relies on a large number of labeled samples being available. To address this problem, this paper proposes a unified deep network, combined with active transfer learning (TL) that can be well-trained for HSIs classification using only minimally labeled training data. More specifically, deep joint spectral–spatial feature is first extracted through hierarchical stacked sparse autoencoder (SSAE) networks. Active TL is then exploited to transfer the pretrained SSAE network and the limited training samples from the source domain to the target domain, where the SSAE network is subsequently fine-tuned using the limited labeled samples selected from both source and target domains by the corresponding active learning (AL) strategies. The advantages of our proposed method are threefold: 1) the network can be effectively trained using only limited labeled samples with the help of novel AL strategies; 2) the network is flexible and scalable enough to function across various transfer situations, including cross data set and intraimage; and 3) the learned deep joint spectral–spatial feature representation is more generic and robust than many joint spectral–spatial feature representations. Extensive comparative evaluations demonstrate that our proposed method significantly outperforms many state-of-the-art approaches, including both traditional and deep network-based methods, on three popular data sets. [ABSTRACT FROM AUTHOR]

Remote sensing image scene classification is a fundamental problem, which aims to label an image with a specific semantic category automatically. Recently, deep learning methods have achieved competitive performance for remote sensing image scene classification, especially the methods based on a convolutional neural network (CNN). However, most of the existing CNN methods only use feature vectors of the last fully connected layer. They give more importance to global information and ignore local information of images. It is common that some images belong to different categories, although they own similar global features. The reason is that the category of an image may be highly related to local features, other than the global feature. To address this problem, a method based on rearranged local features is proposed in this paper. First, outputs of the last convolutional layer and the last fully connected layer are employed to depict the local and global information, respectively. After that, the remote sensing images are clustered to several collections using their global features. For each collection, local features of an image are rearranged according to their similarities with local features of the cluster center. In addition, a fusion strategy is proposed to combine global and local features for enhancing the image representation. The proposed method surpasses the state of the arts on four public and challenging data sets: UC-Merced, WHU-RS19, Sydney, and AID. [ABSTRACT FROM AUTHOR]

Satellite images are widely applied in humanitarian mapping that labels buildings, roads, and so on for humanitarian aid and economic development. However, the labeling now is mostly done by volunteers. In this paper, we utilize deep learning to solve humanitarian mapping tasks of a mobile software named MapSwipe. The current deep learning techniques, e.g., convolutional neural network (CNN), can recognize ground objects from satellite images but rely on numerous labels for training for each specific task. We solve this problem by fusing multiple freely accessible crowdsourced geographic data and propose an active learning-based CNN training framework named MC-CNN to deal with the quality issues of the labels extracted from these data, including incompleteness (e.g., some kinds of object are not labeled) and heterogeneity (e.g., different spatial granularities). The method is evaluated with building mapping in South Malawi and road mapping in Guinea with level-18 satellite images provided by Bing Map and volunteered geographic information from OpenStreetMap, MapSwipe, and OsmAnd. The results based on multiple metrics, including Precision, Recall, F1 Score, and area under the receiver operating characteristic curve, show that MC-CNN can fuse the crowdsourced labels for higher prediction performance and be successfully applied in MapSwipe for humanitarian mapping with 85% labor saved and an overall accuracy of 0.86 achieved. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) denoising is a crucial preprocessing procedure to improve the performance of the subsequent HSI interpretation and applications. In this paper, a novel deep learning-based method for this task is proposed, by learning a nonlinear end-to-end mapping between the noisy and clean HSIs with a combined spatial–spectral deep convolutional neural network (HSID-CNN). Both the spatial and spectral information are simultaneously assigned to the proposed network. In addition, multiscale feature extraction and multilevel feature representation are, respectively, employed to capture both the multiscale spatial–spectral feature and fuse different feature representations for the final restoration. The simulated and real-data experiments demonstrate that the proposed HSID-CNN outperforms many of the mainstream methods in both the quantitative evaluation indexes, visual effects, and HSI classification accuracy. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) exhibit good performance in image processing tasks, pointing themselves as the current state-of-the-art of deep learning methods. However, the intrinsic complexity of remotely sensed hyperspectral images still limits the performance of many CNN models. The high dimensionality of the HSI data, together with the underlying redundancy and noise, often makes the standard CNN approaches unable to generalize discriminative spectral–spatial features. Moreover, deeper CNN architectures also find challenges when additional layers are added, which hampers the network convergence and produces low classification accuracies. In order to mitigate these issues, this paper presents a new deep CNN architecture specially designed for the HSI data. Our new model pursues to improve the spectral–spatial features uncovered by the convolutional filters of the network. Specifically, the proposed residual-based approach gradually increases the feature map dimension at all convolutional layers, grouped in pyramidal bottleneck residual blocks, in order to involve more locations as the network depth increases while balancing the workload among all units, preserving the time complexity per layer. It can be seen as a pyramid, where the deeper the blocks, the more feature maps can be extracted. Therefore, the diversity of high-level spectral–spatial attributes can be gradually increased across layers to enhance the performance of the proposed network with the HSI data. Our experiments, conducted using four well-known HSI data sets and 10 different classification techniques, reveal that our newly developed HSI pyramidal residual model is able to provide competitive advantages (in terms of both classification accuracy and computational time) over the state-of-the-art HSI classification methods [ABSTRACT FROM AUTHOR]

Object-based image analysis is quickly gaining acceptance among remote sensing community, and object-based image classification methods are increasingly being used for classification of land use/cover units from high-resolution satellite images with results closer to human interpretation compared to per-pixel classifiers. The problem of nonlinear separability of classes in a feature space consisting of spectral/spatial/textural features is addressed by kernel-based nonlinear mapping of the feature vectors. This facilitates use of linear discriminant functions for classification as used in artificial neural networks (ANNs). In this paper, performance of a recently introduced kernel called cloud basis function (CBF) is investigated with some modification for classification. The CBF has demonstrated superior performance to the tune of about 4% higher classification accuracy compared to conventional radial basis function used in ANN. The results are further improved by using probabilistic relaxation labeling as a postprocessing step. This paper has potential applications in urban planning and urban studies. [ABSTRACT FROM PUBLISHER]

Ship detection is an important and challenging task in remote sensing applications. Most methods utilize specially designed hand-crafted features to detect ships, and they usually work well only on one scale, which lack generalization and impractical to identify ships with various scales from multiresolution images. In this paper, we propose a novel deep feature-based method to detect ships in very high-resolution optical remote sensing images. In our method, a regional proposal network is used to generate ship candidates from feature maps produced by a deep convolutional neural network. To efficiently detect ships with various scales, a hierarchical selective filtering layer is proposed to map features in different scales to the same scale space. The proposed method is an end-to-end network that can detect both inshore and offshore ships ranging from dozens of pixels to thousands. We test our network on a large ship data set which will be released in the future, consisting of Google Earth images, GaoFen-2 images, and unmanned aerial vehicle data. Experiments demonstrate high precision and robustness of our method. Further experiments on aerial images show its good generalization to unseen scenes. [ABSTRACT FROM AUTHOR]

This paper proposes a new method, called multilayer stacked covariance pooling (MSCP), for remote sensing scene classification. The innovative contribution of the proposed method is that it is able to naturally combine multilayer feature maps, obtained by pretrained convolutional neural network (CNN) models. Specifically, the proposed MSCP-based classification framework consists of the following three steps. First, a pretrained CNN model is used to extract multilayer feature maps. Then, the feature maps are stacked together, and a covariance matrix is calculated for the stacked features. Each entry of the resulting covariance matrix stands for the covariance of two different feature maps, which provides a natural and innovative way to exploit the complementary information provided by feature maps coming from different layers. Finally, the extracted covariance matrices are used as features for classification by a support vector machine. The experimental results, conducted on three challenging data sets, demonstrate that the proposed MSCP method can not only consistently outperform the corresponding single-layer model but also achieve better classification performance than other pretrained CNN-based scene classification methods. [ABSTRACT FROM AUTHOR]