Search

Recently, convolutional neural network (CNN) and Transformer have been successfully applied in hyperspectral image (HSI) classification. Despite their success, some challenges persist. Popular HSI classification methods typically employ only a single patch as the network input, which may not provide robust information for various datasets or within different scenarios of a specific dataset. Large-scale patches are usually conducive to classifying large-scale scenes, while small-scale patches excel at classifying boundary locations. In addition, the majority of methods do not distinguish between the central pixel and its adjacent pixels, although the information from the central position is crucial. In this paper, a novel double-branch hierarchical partial convolution network (DBHP-Net) is proposed to address the aforementioned challenges. Concretely, DBHP-Net employs a new convolution, named TConv, to emphasize mining the spatial-channel features of the central position. Then, the TConvFormer Block is built by adopting the advanced engineering techniques of Transformer. Furthermore, we construct our model in a hierarchical style. The double-branch structure with large-scale and small-scale patches provides complementary information from the dataset. Experimental results based on randomly selected samples and spatially disjoint samples of two benchmark datasets demonstrate that our method outperforms other state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]

Vision Transformers (ViTs) have become one of the main architectures in deep learning with the self-attention mechanism, and are becoming an alternative to Convolutional Neural Networks (CNNs) for remote sensing scene classification tasks. However, the earlier self-attention layer of ViTs focuses on local features rather than global features, and the deeper self-attention layer focuses on global features but ignores the different impact of different frequency information. This will greatly increase the training and computational cost due to the quadratic complexity of the self-attention mechanism on the long sequence representation. In this paper, we propose a hybrid CNN – vision transformer structure (HCVNet), which uses convolutional layers to replace the earlier self-attention layers, and a novel Frequency Multi-head Self Attention (F-MSA) mechanism to replace the deeper self-attention layers. Specifically, F-MSA is a dual-stream structure that reduces the computational cost and improves the classification performance by encoding the high/low frequency information separately. In addition, a Semantic-aware Localization (SaL) module is introduced, which can guide the selection of crop by learning prior knowledge, avoiding the issue of pure background sampling. Our method performed an accuracy value of 97.20 ± 0.02% on the Aerial Image Dataset and 93.89 ± 0.03% on the NWPU-RESISC45 Dataset, with low complexity costs. [ABSTRACT FROM AUTHOR]