Search

Due to the inconsistent imaging environment, the styles of multitemporal multispectral images (MSIs) are quite different, such as image brightness and transparency. For multitemporal MSIs with different styles, the “same object with different spectra” problem is one of the biggest challenges in change detection. To overcome the challenge, a novel unsupervised spatial–spectral feature learning (FL) framework based on style transformation (ST) (called STFL-CD) is proposed for MSI change detection in this article. For dual-temporal MSIs, the proposed STFl-CD algorithm consists of two phases: ST and spatial–spectral FL. Since the image styles are inconsistent under different imaging environments, the first innovation is to transform the image styles through unmixing and reconstruction. Through ST, the challenge of the “same object with different spectra” problem will be reduced fundamentally. By introducing the attention mechanism, the other innovation is to extract the joint spectral–spatial change features based on a 3-D convolutional neural network with spatial and channel attention. In addition, for multitemporal MSIs, a multitemporal version STFL-CD (MT-STFL-CD) framework is designed based on a recurrent neural network to learn the correlation features between multitemporal remote sensing images. Both of the visual and quantitative results on the real MSI datasets indicate that the proposed unsupervised STFL-CD frameworks have significant advantages on multitemporal MSI change detection. In particular, the performance of the proposed unsupervised STFL-CD algorithm is even comparable to that of the state-of-the-art supervised or semisupervised methods. [ABSTRACT FROM AUTHOR]

Remote sensing image captioning (RSIC), which aims at generating a well-formed sentence for a remote sensing image, has attracted more attention in recent years. The general framework for RSIC is the encoder–decoder architecture containing two submodels of encoder and decoder. Although the significant performance is obtained, the encoder–decoder architecture is a black-box model with a lack of explainability. To overcome this drawback, in this article, we propose a new explainable word–sentence framework for RSIC. The proposed word–sentence framework consists of two parts: word extractor and sentence generator, where the former extracts the valuable words in the given remote sensing image, while the latter organizes these words into a well-formed sentence. The proposed framework decomposes RSIC into a word classification task and a word sorting task, which is more in line with human intuitive understanding. On the basis of the word–sentence framework, some ablation experiments are conducted on the three public RSIC data sets of Sydney-captions, UCM-captions, and RSICD to explore the specific and effective network structures. In order to evaluate the proposed word–sentence framework objectively, we further conduct some comparative experiments on these three data sets and achieve comparable results in comparison with the encoder–decoder-based methods. [ABSTRACT FROM AUTHOR]

Recently, remote sensing image captioning (RSIC) has drawn an increasing attention. In this field, the encoder–decoder-based methods have become the mainstream due to their excellent performance. In the encoder–decoder framework, the convolutional neural network (CNN) is used to encode a remote sensing image into a semantic feature vector, and a sequence model such as long short-term memory (LSTM) is subsequently adopted to generate a content-related caption based on the feature vector. During the traditional training stage, the probability of the target word at each time step is forcibly optimized to 1 by the cross entropy (CE) loss. However, because of the variability and ambiguity of possible image captions, the target word could be replaced by other words like its synonyms, and therefore, such an optimization strategy would result in the overfitting of the network. In this article, we explore the overfitting phenomenon in the RSIC caused by CE loss and correspondingly propose a new truncation cross entropy (TCE) loss, aiming to alleviate the overfitting problem. In order to verify the effectiveness of the proposed approach, extensive comparison experiments are performed on three public RSIC data sets, including UCM-captions, Sydney-captions, and RSICD. The state-of-the-art result of Sydney-captions and RSICD and the competitive results of UCM-captions achieved by TCE loss demonstrate that the proposed method is beneficial to RSIC. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) clustering is very important in remote sensing applications. However, most graph-based clustering models are not suitable for dealing with large HSI due to their computational bottlenecks: the construction of the similarity matrix $\boldsymbol {W}$ , the eigenvalue decomposition of the graph Laplacian matrix $\boldsymbol {L}$ , and $k$ -means or other discretization procedures. To solve this problem, we propose a novel approach, scalable graph-based clustering with nonnegative relaxation (SGCNR), to cluster the large HSI. The proposed SGCNR algorithm first constructs an anchor graph and then adds the nonnegative relaxation term. With this, the computational complexity can be reduced to $O(nd\log m+nK^{2}+nKc+K^{3})$ , compared with traditional graph-based clustering algorithms that need at least $O(n^{2}d+n^{2}K)$ or $O(n^{2}d+n^{3})$ , where $n$ , $d$ , $m$ , $K$ , and $c$ are, respectively, the number of samples, features, anchors, classes, and nearest neighbors. In addition, the SGCNR algorithm can directly obtain the clustering indicators, without resort to $k$ -means or other discretization procedures as traditional graph-based clustering algorithms have to do. Experimental results on several HSI data sets have demonstrated the efficiency and effectiveness of the proposed SGCNR algorithm. [ABSTRACT FROM AUTHOR]

Scene classification of remote sensing images has drawn great attention because of its wide applications. In this paper, with the guidance of the human visual system (HVS), we explore the attention mechanism and propose a novel end-to-end attention recurrent convolutional network (ARCNet) for scene classification. It can learn to focus selectively on some key regions or locations and just process them at high-level features, thereby discarding the noncritical information and promoting the classification performance. The contributions of this paper are threefold. First, we design a novel recurrent attention structure to squeeze high-level semantic and spatial features into several simplex vectors for the reduction of learning parameters. Second, an end-to-end network named ARCNet is proposed to adaptively select a series of attention regions and then to generate powerful predictions by learning to process them sequentially. Third, we construct a new data set named OPTIMAL-31, which contains more categories than popular data sets and gives researchers an extra platform to validate their algorithms. The experimental results demonstrate that our model makes great promotion in comparison with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification, which aims to assign an accurate label for hyperspectral pixels, has drawn great interest in recent years. Although low-rank representation (LRR) has been used to classify HSI, its ability to segment each class from the whole HSI data has not been exploited fully yet. LRR has a good capacity to capture the underlying low-dimensional subspaces embedded in original data. However, there are still two drawbacks for LRR. First, the LRR does not consider the local geometric structure within data, which makes the local correlation among neighboring data easily ignored. Second, the representation obtained by solving LRR is not discriminative enough to separate different data. In this paper, a novel locality- and structure-regularized LRR (LSLRR) model is proposed for HSI classification. To overcome the above-mentioned limitations, we present locality constraint criterion and structure preserving strategy to improve the classical LRR. Specifically, we introduce a new distance metric, which combines both spatial and spectral features, to explore the local similarity of pixels. Thus, the global and local structures of HSI data can be exploited sufficiently. In addition, we propose a structural constraint to make the representation have a near-block-diagonal structure. This helps to determine the final classification labels directly. Extensive experiments have been conducted on three popular HSI data sets. And the experimental results demonstrate that the proposed LSLRR outperforms other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Inspired by recent development of artificial satellite, remote sensing images have attracted extensive attention. Recently, notable progress has been made in scene classification and target detection. However, it is still not clear how to describe the remote sensing image content with accurate and concise sentences. In this paper, we investigate to describe the remote sensing images with accurate and flexible sentences. First, some annotated instructions are presented to better describe the remote sensing images considering the special characteristics of remote sensing images. Second, in order to exhaustively exploit the contents of remote sensing images, a large-scale aerial image data set is constructed for remote sensing image caption. Finally, a comprehensive review is presented on the proposed data set to fully advance the task of remote sensing caption. Extensive experiments on the proposed data set demonstrate that the content of the remote sensing image can be completely described by generating language descriptions. The data set is available at https://github.com/201528014227051/RSICD_optimal. [ABSTRACT FROM AUTHOR]