Search

In recent years, numerous deep learning (DL)-based frameworks have been proposed for hyperspectral image classification (HSIC). Considering a large number of spectral bands of hyperspectral images (HSIs), it is still challenging to effectively utilize the spectral information and achieve accurate classification when few training samples are available. To make full use of the spectral–spatial information in HSIs with few training samples, in this letter, we propose a lightweight end-to-end attention-enhanced feature fusion network (AeF2N). The proposed AeF2N consists of four sequential stages, i.e., spectral feature augmentation, spatial contextual feature interaction, spectral feature augmentation, and classification. The first and third stages are used to capture and augment the discriminative spectral features, while the second stage is used to capture spatial information. Notably, two novel attention blocks, spectral augmentation attention (SAA) and spatial integration attention (SIA) are interactively introduced to capture significant spectral and spatial information, respectively. Based on the proposed spectral and spatial feature discrimination stages, the AeF2N effectively identifies both spectrally significant (e.g., irregular small objects) and spatially significant (e.g., specific-shaped objects) land objects with high accuracy. Experimental results obtained on three benchmark hyperspectral datasets demonstrate the superiority of the proposed approach compared with six state-of-the-art DL-based methods in terms of higher classification accuracy and efficiency. [ABSTRACT FROM AUTHOR]

With more detailed spatial information being represented in very-high-resolution (VHR) remote sensing images, stringent requirements are imposed on accurate image classification. Due to the diverse land objects with intraclass variation and interclass similarity, efficient and fine classification of VHR images especially in complex scenes are challenging. Even for some popular deep learning (DL) frameworks, geometric details of land objects may be lost in deep feature levels, so it is difficult to maintain the highly detailed spatial information (e.g., edges, small objects) only relying on the last high-level layer. Moreover, many of the newly developed DL methods require massive well-labeled samples, which inevitably deteriorates the model generalization ability under the few-shot learning. Therefore, in this article, a lightweight shallow-to-deep feature fusion network (SDF2N) is proposed for VHR image classification, where the traditional machine learning (ML) and DL schemes are integrated to learn rich and representative information to improve the classification accuracy. In particular, the shallow spectral–spatial features are first extracted and then a novel triple-stage fusion (TSF) module is designed to learn the saliency and discriminative information at different levels for classification. The TSF module includes three feature fusion stages, that is, low-level spectral–spatial feature fusion, middle-level multiscale feature fusion, and high-level multilayer feature fusion. The proposed SDF2N takes the advantage of the shallow-to-deep features, which can extract representative and complementary information from crossing layers. It is important to note that even with limited training samples, the SDF2N still can achieve satisfying classification performance. Experimental results obtained on three real VHR remote sensing datasets including two multispectral and one airborne hyperspectral images covering complex urban scenarios confirm the effectiveness of the proposed approach compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the increasing availability and resolution of satellite sensor data, multispectral (MS) and panchromatic (PAN) images are the most popular data that are used in remote sensing among applications. This article proposes a novel cross-resolution hidden layer feature fusion (CRHFF) approach for joint classification of multiresolution MS and PAN images. In particular, shallow spectral and spatial features at a global scale are first extracted from an MS image. Then, deep cross-resolution hidden layer features extracted from MS and PAN are fused from patches at a local scale according to an autoencoder (AE)-like deep network. Finally, the selected multiresolution hidden layer features are classified in a supervised manner. By taking advantage of integrated shallow-to-deep and global-to-local features from the high-resolution MS and PAN images, the cross-resolution latent information can be extracted and fused in order to better model imaged objects from the multimodal representation and finally increase the classification accuracy. Experimental results obtained on three real multiresolution datasets covering complex urban scenarios confirm the effectiveness of the proposed approach in terms of higher accuracy and robustness with respect to literature methods. [ABSTRACT FROM AUTHOR]

Crater detection from planetary images is a challenging issue due to the complicated variations in geometry shape, illumination, and scale. An automatic crater detection algorithm (CDA) that is robust to these factors is, therefore, necessary. In this article, a novel automatic CDA that is robust to these factors is proposed to detect the multiscale craters of the Moon. The proposed method consists of two main steps: 1) in the hypothesis generation (HG) step, a novel feature operator called the path-profile, which is constructed based on the self-defined adjacency graph and a path descriptor, is presented to derive the highlight-shadow feature of craters for detecting candidate craters. 2) In the hypothesis verification (HV) step, based on the idea of anomaly detection, the isolation forest algorithm which is an unsupervised learning anomaly detection method is applied to eliminate falsely detected craters. Lunar Reconnaissance Orbiter Camera Wide Angle Camera and Narrow Angle Camera images and Chang’E-4 landing camera images were used to test the accuracy and robustness of the proposed method. The experimental results indicate that: on average, the accuracy of the detection result of the HG step is about 90%, and the HV step can further improve this by 3%–4%. The proposed method is a reliable way to detect multiscale lunar craters for various resolutions images with diameters ranging from five pixels to hundreds of pixels, and it is robust to the different terrains and illumination conditions on the Moon. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have achieved great success when characterizing remote sensing (RS) images. However, the lack of sufficient annotated data (together with the high complexity of the RS image domain) often makes supervised and transfer learning schemes limited from an operational perspective. Despite the fact that unsupervised methods can potentially relieve these limitations, they are frequently unable to effectively exploit relevant prior knowledge about the RS domain, which may eventually constrain their final performance. In order to address these challenges, this article presents a new unsupervised deep metric learning model, called spatially augmented momentum contrast (SauMoCo), which has been specially designed to characterize unlabeled RS scenes. Based on the first law of geography, the proposed approach defines spatial augmentation criteria to uncover semantic relationships among land cover tiles. Then, a queue of deep embeddings is constructed to enhance the semantic variety of RS tiles within the considered contrastive learning process, where an auxiliary CNN model serves as an updating mechanism. Our experimental comparison, including different state-of-the-art techniques and benchmark RS image archives, reveals that the proposed approach obtains remarkable performance gains when characterizing unlabeled scenes since it is able to substantially enhance the discrimination ability among complex land cover categories. The source codes of this article will be made available to the RS community for reproducible research. [ABSTRACT FROM AUTHOR]

Fourier-based image correlation is a powerful area-based image registration technique, which involves aligning images based on a translation model or similarity model by means of the image information and operation in the frequency domain. In recent years, Fourier-based image correlation has made significant progress and attracted extensive research interest in a variety of applications, especially in the field of photogrammetry and remote sensing, leading to the development of a number of subpixel methods that have improved the accuracy and robustness. However, to date, a detailed review of the literature related to Fourier-based image correlation is still lacking. In this review, we aim at providing a comprehensive overview of the fundamentals, developments, and applications of image registration with Fourier-based image correlation methods. Specifically, this review introduces the principal laws underlying these methods, presents a survey of the existing subpixel methods calculated both in the spatial domain and in the frequency domain, summarizes the major applications from three aspects, and discusses the challenges and possible directions of future research. This review is expected to be beneficial for researchers working in the relevant fields to obtain an insight into the current state of the art, to develop new variants, to explore potential applications, and to suggest promising future trends of image registration with Fourier-based image correlation. [ABSTRACT FROM AUTHOR]

Due to the limited number of spectral channels in multispectral remote sensing images, change information, especially the multiclass changes, may be insufficiently represented, resulting in inaccurate detection of changes. In this paper, we propose to use unsupervised band expansion techniques to generate artificial spectral and spatial bands to enhance the change representation and discrimination for change detection (CD) from multispectral images. In particular, in the proposed approach, two simple nonlinear functions, i.e., multiplication and division, are applied for spectral expansion. Multiscale morphological reconstruction is used to extend the band spatial information. The expanded band sets are then used and validated in three popular unsupervised CD techniques for solving a multiclass CD problem. Experimental results obtained on three real bitemporal multispectral remote sensing datasets confirm the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Seed point-based road extraction methods are vital for extracting road networks from satellite images. Despite its effectiveness, roads in very high-resolution (VHR) satellite images are complicated, such as road occlusion and material change. To tackle this issue, this paper proposes to use the colour space transformation and geodesic method. First, the test image is converted from Red-Green-Blue colour space to Hue-Saturation-Value colour space to reduce the material change influence. The geodesic method is subsequently applied to extract initial road segments that link road seed points provided by users. At last, the initial result is adjusted by a kernel density estimation method to produce centred roads. The presented method is quantitatively evaluated on three test images. Experiments show that the proposed method yields a substantial improvement over cutting-edge technologies. The findings in this study shine new light on a practical solution for road extraction from satellite images. [ABSTRACT FROM AUTHOR]

Pixel-based contextual classification methods, including morphological profiles (MPs), extended MPs, attribute profiles (APs), and MPs with partial reconstruction (MPPR), have shown the benefits of using geometrical features extracted from very-high resolution (VHR) images. However, the structural element sequence or the attribute filters that are necessarily adopted in the above solutions always result in computationally inefficient and redundant high-dimensional features. To solve the second problem, we introduce maximally stable extremal regions (MSER) guided MPs (MSER_MPs) and MSER_MPs(M), which contains mean pixel values within regions, to foster effective and efficient spatial feature extraction. In addition, the extremely randomized decision tree (ERDT) and its ensemble version, ExtraTrees, are introduced and investigated. An extremely randomized rotation forest (ERRF) is proposed by simply replacing the conventional C4.5 decision tree in a rotation forest (RoF) with an ERDT. Finally, the proposed spatial feature extractors, ERDT, ExtraTrees, and ERRF are evaluated for their ability to classify three VHR multispectral images acquired over urban areas, and compared against C4.5, Bagging(C4.5), random forest, support vector machine, and RoF in terms of classification accuracy and computational efficiency. The experimental results confirm the superior performance of MSER_MPs(M) and MSER_MPs compared to MPPR and MPs, respectively, and ExtraTrees is better for spectral-spatial classification of VHR multispectral images using the original spectra stacked with MSER_MPs(M) features. [ABSTRACT FROM AUTHOR]

A novel multiscale morphological compressed change vector analysis (M2C2VA) method is proposed to address the multiple-change detection problem (i.e., identifying different classes of changes) in bitemporal remote sensing images. The proposed approach contributes to extend the state-of-the-art spectrum-based compressed change vector analysis (C2VA) method by jointly analyzing the spectral-spatial change information. In greater details, reconstructed spectral change vector features are built according to a morphological analysis. Thus more geometrical details of change classes are preserved while exploiting the interaction of a pixel with its adjacent regions. Two multiscale ensemble strategies, i.e., data level and decision level fusion, are designed to integrate the change information represented at different scales of features or to combine the change detection results obtained by the detector at different scales, respectively. A detailed scale sensitivity analysis is carried out to investigate its impacts on the performance of the proposed method. The proposed method is designed in an unsupervised fashion without requiring any ground reference data. The proposed M2C2VA is tested on one simulated and three real bitemporal remote sensing images showing its properties in terms of different image size and spatial resolution. Experimental results confirm its effectiveness. [ABSTRACT FROM PUBLISHER]

As a result of urbanization, land use/land cover classes in urban areas are changing rapidly, and this trend increased in the recent years. Change information detected from multi-temporal remote sensing images can thus help to understand urban development and to effectively support urban planning. Differences in reflectance spectra, easily obtained by multi-temporal remote sensing images, are important indicators to characterize these changes. Although many algorithms were proposed to generate difference images, the results are usually greatly inconsistent. In order to integrate the merits of different algorithms to recognize spectral changes, fusion techniques merging multiple difference images are proposed and implemented in this paper. Feature and decision level fusion are used to combine simple change detectors, and to build an automatic change detection procedure. The proposed approach is tested with multi-temporal CBERS and HJ-1 images, and experimental results demonstrate its effectiveness and reliability. By integrating different change information, the appropriate fusion method can be selected according to the specific application in order to minimize the omission or the commission errors. [ABSTRACT FROM PUBLISHER]

With the latest popularization of remote desktop software and online work, screen content images (SCIs) are increasingly being contacted by people. Many image quality assessment (IQA) methods based on deep learning have been proposed for SCIs. However, most IQA methods for SCIs pay less attention to regional characteristics. An asymmetric two-stream network based on region features is proposed for SCI objective quality assessment, in which features are derived from a channel separation feature extraction (CSFE) sub-network and a spatial analysis feature extraction (SAFE) sub-network. In the CSFE sub-network, three channel separation (CS) modules are utilized to strengthen the ability for capturing deepthwise features in the pictorial regions. In the SAFE sub-network, two spatial analysis (SA) module are applied to focus on the structural information in the spatial domain for the textual regions. Two asymmetric sub-networks can efficiently extract the corresponding region features by separately considering the pictorial and textual regions of SCIs. In addition, an adaptive global weighting strategy is introduced to estimate the quality for SCIs. The experimental results on the screen image quality assessment database show that the proposed model is superior to the other state-of-the-art SCI quality assessment methods. [ABSTRACT FROM AUTHOR]

Pansharpening aims at fusing the rich spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images to generate a fused image with both high resolutions. In general, the existing pansharpening methods suffer from the problems of spectral distortion and lack of spatial detail information, which might prevent the accuracy computation for ground object identification. To alleviate these problems, we propose a Hybrid Attention mechanism-based Residual Neural Network (HARNN). In the proposed network, we develop an encoder attention module in the feature extraction part to better utilize the spectral and spatial features of MS and PAN images. Furthermore, the fusion attention module is designed to alleviate spectral distortion and improve contour details of the fused image. A series of ablation and contrast experiments are conducted on GF-1 and GF-2 datasets. The fusion results with less distorted pixels and more spatial details demonstrate that HARNN can implement the pansharpening task effectively, which outperforms the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

In this article, a novel feature selection-based multi-scale superpixel-based guided filter (FS-MSGF) method for classification of very-high-resolution (VHR) remotely sensed imagery is proposed. Improved from the original guided filter (GF) algorithm used in the classification, the guidance image in the proposed approach is constructed based on the superpixel-level segmentation. By taking into account the object boundaries and the inner-homogeneity, the superpixel-level guidance image leads to the geometrical information of land-cover objects in VHR images being better depicted. High-dimensional multi-scale guided filter (MSGF) features are then generated, where the multi-scale information of those land-cover classes is better modelled. In addition, for improving the computational efficiency without the loss of accuracy, a subset of those MSGF features is then automatically selected by using an unsupervised feature selection method, which contains the most distinctive information in all constructed MSGF features. Quantitative and qualitative classification results obtained on two QuickBird remotely sensed imagery datasets covering the Zurich urban scene are provided and analyzed, which demonstrate that the proposed methods outperform the state-of-the-art reference techniques in terms of higher classification accuracies and higher computational efficiency. [ABSTRACT FROM AUTHOR]