Search

Water area segmentation in remote sensing is of great importance for flood monitoring. To overcome some challenges in this task, we construct the Water Index and Polarization Information (WIPI) multi-modality dataset and propose a multi-Modality Fusion and Gated multi-Filter U-Net (MFGF-UNet) convolutional neural network. The WIPI dataset can enhance the water information while reducing the data dimensionality: specifically, the Cloud-Free Label provided in the dataset can effectively alleviate the problem of labeled sample scarcity. Since a single form or uniform kernel size cannot handle the variety of sizes and shapes of water bodies, we propose the Gated Multi-Filter Inception (GMF-Inception) module in our MFGF-UNet. Moreover, we utilize an attention mechanism by introducing a Gated Channel Transform (GCT) skip connection and integrating GCT into GMF-Inception to further improve model performance. Extensive experiments on three benchmarks, including the WIPI, Chengdu and GF2020 datasets, demonstrate that our method achieves favorable performance with lower complexity and better robustness against six competing approaches. For example, on the WIPI, Chengdu and GF2020 datasets, the proposed MFGF-UNet model achieves F1 scores of 0.9191, 0.7410 and 0.8421, respectively, with the average F1 score on the three datasets 0.0045 higher than that of the U-Net model; likewise, GFLOPS were reduced by 62% on average. The new WIPI dataset, the code and the trained models have been released on GitHub. [ABSTRACT FROM AUTHOR]

The aim of remote sensing image captioning (RSIC) is to describe a given remote sensing image (RSI) using coherent sentences. Most existing attention-based methods model the coherence through an LSTM-based decoder, which dynamically infers a word vector from preceding sentences. However, these methods are indirectly guided through the confusion of attentive regions, as (1) the weighted average in the attention mechanism distracts the word vector from capturing pertinent visual regions and (2) there are few constraints or rewards for learning long-range transitions. In this paper, we propose a multi-source interactive stair attention mechanism that separately models the semantics of preceding sentences and visual regions of interest. Specifically, the multi-source interaction takes previous semantic vectors as queries and applies an attention mechanism on regional features to acquire the next word vector, which reduces immediate hesitation by considering linguistics. The stair attention divides the attentive weights into three levels—that is, the core region, the surrounding region, and other regions—and all regions in the search scope are focused on differently. Then, a CIDEr-based reward reinforcement learning is devised, in order to enhance the quality of the generated sentences. Comprehensive experiments on widely used benchmarks (i.e., the Sydney-Captions, UCM-Captions, and RSICD data sets) demonstrate the superiority of the proposed model over state-of-the-art models, in terms of its coherence, while maintaining high accuracy. [ABSTRACT FROM AUTHOR]

Deep descriptors have advantages over handcrafted descriptors on local image patch matching. However, due to the complex imaging mechanism of remote sensing images and the significant differences in appearance between multi-modal images, existing deep learning descriptors are unsuitable for multi-modal remote sensing image registration directly. To solve this problem, this article proposes a deep feature correlation learning network (Cnet) for multi-modal remote sensing image registration. First, Cnet builds a feature learning network based on the deep convolutional network with the attention learning module, to enhance feature representation by focusing on meaningful features. Second, this article designs a novel feature correlation loss function for Cnet optimization. It focuses on the relative feature correlation between matching and nonmatching samples, which can improve the stability of network training and decrease the risk of overfitting. In addition, the proposed feature correlation loss with a scale factor can further enhance network training and accelerate network convergence. Extensive experimental results on image patch matching (Brown, HPatches), cross-spectral image registration (VIS-NIR), multi-modal remote sensing image registration, and single-modal remote sensing image registration have demonstrated the effectiveness and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Remote sensing object tracking (RSOT) is a novel and challenging problem due to the negative effects of weak features and background noise. In this article, from the perspective of attention-focus deep learning, we propose a joint Siamese attention-aware network (JSANet) for efficient remote sensing tracking which contains both the self-attention and cross-attention modules. First, the self-attention modules we propose emphasize the interdependent channel-wise coefficient via channel attention and conduct corresponding space transformation of spatial domain information with spatial attention. Second, the cross-attention is designed to aggregate rich contextual interdependencies between the Siamese branches via channel attention and excavate association produces reliable correspondence with spatial attention. In addition, a composite feature combine strategy is designed to fuse multiple attention features. Experimental results on the Jilin-1 satellite video datasets demonstrate that the proposed JSANet achieves state-of-the-art performance in terms of precision and success rate, demonstrating the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Hyperspectral images (HSIs) have gained high spectral resolution due to recent advances in spectral imaging technologies. This incurs problems, such as an increased data scale and an increased number of bands for HSIs, which results in a complex correlation between different bands. In the applications of remote sensing and earth observation, ground objects represented by each HSI pixel are composed of physical and chemical non-Euclidean structures, and HSI classification (HIC) is becoming a more challenging task. To solve the above problems, we propose a framework based on a deep attention graph convolutional network (DAGCN). Specifically, we first integrate an attention mechanism into the spectral similarity measurement to aggregate similar spectra. Therefore, we propose a new similarity measurement method, i.e., the mixed measurement of a kernel spectral angle mapper and spectral information divergence (KSAM-SID), to aggregate similar spectra. Considering the non-Euclidean structural characteristics of HSIs, we design deep graph convolutional networks (DeepGCNs) as a feature extraction method to extract deep abstract features and explore the internal relationship between HSI data. Finally, we dynamically update the attention graph adjacency matrix to adapt to the changes in each feature graph. Experiments on three standard HSI data sets, namely, the Indian Pines, Pavia University, and Salinas data sets, demonstrate that the DAGCN outperforms the baselines in terms of various evaluation criteria. For example, on the Indian Pines data set, the overall accuracy of the proposed method achieves 98.61% when the training sample is 10%. [ABSTRACT FROM AUTHOR]

Change detection for remote sensing images is an indispensable procedure for many remote sensing applications, such as geological disaster assessment, environmental monitoring, and urban development monitoring. Through this technique, the difference in certain areas after some emergencies can be determined to estimate their influence. Additionally, by analyzing the sequential difference maps, the change tendency can be found to help to predict future changes, such as urban development and environmental pollution. The complex variety of changes and interferential changes caused by imaging processing, such as season, weather and sensors, are critical factors that affect the effectiveness of change detection methods. Recently, there have been many research achievements surrounding this topic, but a perfect solution to all the problems in change detection has not yet been achieved. In this paper, we mainly focus on reducing the influence of imaging processing through the deep neural network technique with limited labeled samples. The attention-guided Siamese fusion network is constructed based on one basic Siamese network for change detection. In contrast to common processing, besides high-level feature fusion, feature fusion is operated during the whole feature extraction process by using an attention information fusion module. This module can not only realize the information fusion of two feature extraction network branches, but also guide the feature learning network to focus on feature channels with high importance. Finally, extensive experiments were performed on three public datasets, which could verify the significance of information fusion and the guidance of the attention mechanism during feature learning in comparison with related methods. [ABSTRACT FROM AUTHOR]

Recently, with the popularity of space-borne earth satellites, the resolution of high-resolution panchromatic (PAN) and multispectral (MS) remote sensing images is also increasing year by year, multiresolution remote sensing classification has become a research hotspot. In this paper, from the perspective of deep learning, we design a dual-branch interactive spatial-channel collaborative attention enhancement network (SCCA-net) for multiresolution classification. It aims to combine sample enhancement and feature enhancement to improve classification accuracy. In the part of sample enhancement, we propose an adaptive neighbourhood transfer sampling strategy (ANTSS). Different from the traditional pixel-centric sampling strategy with orthogonal sampling angle, our algorithm allows each patch to adaptively transfer the neighbourhood range by finding the homogeneous region of the pixel to be classified. And it also adaptively adjust the sampling angle according to the texture distribution of the homogeneous region to capture neighbourhood information that is more conducive for classification. Moreover, in the part of feature enhancement part, we design a local spatial attention module (LSA-module) for PAN data to highlight the spatial resolution advantages and a global channel attention module (GCA-module) for MS data to improve the multi-channel representation. It not only highlights the spatial resolution advantage of PAN data and the multi-channel advantage of MS data, but also improves the difference between features through the interaction between the two modules. Quantitative and qualitative experimental results verify the robustness and effectiveness of the method. [ABSTRACT FROM AUTHOR]

Object detection has made significant progress in many real-world scenes. Despite this remarkable progress, the common use case of detection in remote sensing images remains challenging even for leading object detectors, due to the complex background, objects with arbitrary orientation, and large difference in scale of objects. In this paper, we propose a novel rotation detector for remote sensing images, mainly inspired by Mask R-CNN, namely RADet. RADet can obtain the rotation bounding box of objects with shape mask predicted by the mask branch, which is a novel, simple and effective way to get the rotation bounding box of objects. Specifically, a refine feature pyramid network is devised with an improved building block constructing top-down feature maps, to solve the problem of large difference in scales. Meanwhile, the position attention network and the channel attention network are jointly explored by modeling the spatial position dependence between global pixels and highlighting the object feature, for detecting small object surrounded by complex background. Extensive experiments on two remote sensing public datasets, DOTA and NWPUVHR -10, show our method to outperform existing leading object detectors in remote sensing field. [ABSTRACT FROM AUTHOR]

• A dual branch attention fusion deep network for classification is proposed. • An adaptive center offset sampling strategy for each patch is proposed. • Two attention based modules to highlight the advantages of two different data. • The feature level fusion is achieved by the proposed network. In recent years, with the diversification of acquisition methods of very high resolution panchromatic (PAN) and multispectral (MS) remote sensing images, multiresolution remote sensing classification has become a research hotspot. In this paper, from the perspective of data–driven deep learning, we design a dual–branch attention fusion deep network (DBAF–Net) for the multiresolution classification. It aims to integrate the feature–level fusion and classification into an end–to–end network model. In the process of establishing a training sample library, unlike the traditional pixel–centric sampling strategy with fixed patch size, we propose an adaptive center–offset sampling strategy (ACO–SS), which allows each patch to adaptively determine the neighborhood range by finding the texture structure of the pixel to be classified. And the neighborhood range is not symmetrical with this pixel, we expect to capture the neighborhood information that is more conducive to its classification. In network structure, based on the captured patches by ACO–SS, we design a spatial attention module (SA–module) for PAN data and a channel attention module (CA–module) for MS data, thus highlighting the spatial resolution advantages of PAN data and the multi–channel advantages of MS data, respectively. Then these two features are interfused to improve and strengthen the fusion features in both spatial and channel. The quantitative and qualitative experimental results verify the robustness and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]