Search

The multispectral (MS) and the panchromatic (PAN) images represent complementary and synergistic spatial spectral information, how to make optimal use of the advantages of them has become a hot research topic. This paper proposes a selectable Transformer and Gist CNN network (STGC-Net). It designs a subspace similar recombination module (SSR-Module) based on non-negative matrix factorization (NMF) and the self-attention mechanism for feature decomposition. This can alleviate the redundant information of multi-modal data and extract their own singular and common features. Considering that the MS and the PAN images exhibit different advantageous properties, a selectable self-attention spectral feature extraction module (S 3 FE-Module) and a multi-stream Gist spatial feature extraction module (MGSFE-Module) are proposed for the different singular features. The former can refine the Transformer's input and simultaneously characterize the sequence information between channels for the MS image. The latter introduces the positional relationship between local features while extracting spatial features for the PAN image, thereby improving the accuracy of scene classification. Experimental results indicate that the proposed method performs better than the other methods. The relevant code of this paper is provided at: https://github.com/ru-willow/ST-GC-Net. • A subspace similar recombination module is proposed. • The selectable self-attention spectral feature extraction module is designed. • The multi-stream Gist spatial feature extraction module is introduced. [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) image classification is a vital application in remote sensing image processing. In recent years, deep learning models like stacked autoencoder and its variants have been utilized to handle this problem and perform well. But their performances highly depend on proper hyper-parameter configuration. In this paper, we propose a multiobjective evolutionary algorithm assisted stacked autoencoder (SAE_MOEA/D) for PolSAR image classification, which could adaptively optimize its parameters and hyper-parameters such as weights, activation functions and the balance factor in the loss function of stacked autoencoder, and decide how many layers of the network should be used according to datasets. Its performance has been tested on five PolSAR images. Compared with commonly used methods, our method obtains competitive results and could save manpower. [ABSTRACT FROM AUTHOR]

Recently, a massive number of deep learning-based approaches have been successfully applied to various remote sensing image (RSI) recognition tasks. However, most existing advances of deep learning methods in the RSI field heavily rely on the features extracted by the manually designed backbone network, which severely hinders the potential of deep learning models due to the complexity of RSI and the limitation of prior knowledge. In this paper, we research a new design paradigm for the backbone architecture in RSI recognition tasks, including scene classification, land-cover classification and object detection. A novel one-shot architecture search framework based on a weight-sharing strategy and an evolutionary algorithm is proposed, called RSBNet, which consists of three stages: Firstly, a supernet constructed in a layer-wise search space is pretrained on a self-assembled large-scale RSI dataset based on an ensemble single-path training strategy. Next, the pre-trained supernet is equipped with different recognition heads through the switchable recognition module and respectively fine-tuned on the target dataset to obtain task-specific supernet. Finally, we search for the optimal backbone architecture for different recognition tasks based on the evolutionary algorithm without any network training. Extensive experiments have been conducted on five benchmark datasets for different recognition tasks, the results show the effectiveness of the proposed search paradigm and demonstrate that the searched backbone is able to flexibly adapt different RSI recognition tasks and achieve impressive performance. [ABSTRACT FROM AUTHOR]

• A dual branch attention fusion deep network for classification is proposed. • An adaptive center offset sampling strategy for each patch is proposed. • Two attention based modules to highlight the advantages of two different data. • The feature level fusion is achieved by the proposed network. In recent years, with the diversification of acquisition methods of very high resolution panchromatic (PAN) and multispectral (MS) remote sensing images, multiresolution remote sensing classification has become a research hotspot. In this paper, from the perspective of data–driven deep learning, we design a dual–branch attention fusion deep network (DBAF–Net) for the multiresolution classification. It aims to integrate the feature–level fusion and classification into an end–to–end network model. In the process of establishing a training sample library, unlike the traditional pixel–centric sampling strategy with fixed patch size, we propose an adaptive center–offset sampling strategy (ACO–SS), which allows each patch to adaptively determine the neighborhood range by finding the texture structure of the pixel to be classified. And the neighborhood range is not symmetrical with this pixel, we expect to capture the neighborhood information that is more conducive to its classification. In network structure, based on the captured patches by ACO–SS, we design a spatial attention module (SA–module) for PAN data and a channel attention module (CA–module) for MS data, thus highlighting the spatial resolution advantages of PAN data and the multi–channel advantages of MS data, respectively. Then these two features are interfused to improve and strengthen the fusion features in both spatial and channel. The quantitative and qualitative experimental results verify the robustness and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Hyperspectral pansharpening (HP) can fuse different kinds of remote sensing images for the downstream classification or detection task. Deep learning methods have made rapid development in HP. However, most deep learning-based HP methods design the network architecture without considering the specific downstream task. When the pansharpening image is transferred to the downstream tasks, it may lead to poor performance on the downstream tasks. To solve this problem, a novel classification-oriented multi-task network (CMNet) is proposed for HP. CMNet combines the classification and HP networks in a multi-task learning way. In the classification network, class-specific and spatial-difference contrastive constraints are defined to enforce the extraction module to extract discriminative features for classification. In the HP network, a fusion module with an identical structure to the extraction module is constructed to extract fused features for reconstruction. Then, multi-gradient learning is designed to optimize the HP network under the guidance of the classification network. It adopts gradient mapping to consider the conflicting or collaborative situations between these different tasks. In this way, the fusion module in the HP network learns discriminative feature extraction ability from the classification network, which makes the HP network realize classification-oriented HP. Experimental results demonstrate that the proposed method markedly improves the pansharpening performance and the pansharpening is also able to elevate the classification performance. [ABSTRACT FROM AUTHOR]