Search

Remote sensing image classification (RSIC) is a classical and fundamental task in the intelligent interpretation of remote sensing imagery, which can provide unique labeling information for each acquired remote sensing image. Thanks to the potent global context information extraction ability of the multi-head self-attention (MSA) mechanism, visual transformer (ViT)-based architectures have shown excellent capability in natural scene image classification. However, in order to achieve powerful RSIC performance, it is insufficient to capture global spatial information alone. Specifically, for fine-grained target recognition tasks with high inter-class similarity, discriminative and effective local feature representations are key to correct classification. In addition, due to the lack of inductive biases, the powerful global spatial context representation capability of ViT requires lengthy training procedures and large-scale pre-training data volume. To solve the above problems, a hybrid architecture of convolution neural network (CNN) and ViT is proposed to improve the RSIC ability, called P 2 FEViT, which integrates plug-and-play CNN features with ViT. In this paper, the feature representation capabilities of CNN and ViT applying for RSIC are first analyzed. Second, aiming to integrate the advantages of CNN and ViT, a novel approach embedding CNN features into the ViT architecture is proposed, which can make the model synchronously capture and fuse global context and local multimodal information to further improve the classification capability of ViT. Third, based on the hybrid structure, only a simple cross-entropy loss is employed for model training. The model can also have rapid and comfortable convergence with relatively less training data than the original ViT. Finally, extensive experiments are conducted on the public and challenging remote sensing scene classification dataset of NWPU-RESISC45 (NWPU-R45) and the self-built fine-grained target classification dataset called BIT-AFGR50. The experimental results demonstrate that the proposed P 2 FEViT can effectively improve the feature description capability and obtain outstanding image classification performance, while significantly reducing the high dependence of ViT on large-scale pre-training data volume and accelerating the convergence speed. The code and self-built dataset will be released at our webpages. [ABSTRACT FROM AUTHOR]

Currently, under supervised learning, a model pre-trained by a large-scale nature scene dataset and then fine-tuned on a few specific task labeling data is the paradigm that has dominated knowledge transfer learning. Unfortunately, due to different categories of imaging data and stiff challenges of data annotation, there is not a large enough and uniform remote sensing dataset to support large-scale pre-training in the remote sensing domain (RSD). Moreover, pre-training models on large-scale nature scene datasets by supervised learning and then directly fine-tuning on diverse downstream tasks seems to be a crude method, which is easily affected by inevitable incorrect labeling, severe domain gaps and task-aware discrepancies. Thus, in this paper, considering the self-supervised pre-training and powerful vision transformer (ViT) architecture, a concise and effective knowledge transfer learning strategy called ConSecutive Pre-Training (CSPT) is proposed based on the idea of not stopping pre-training in natural language processing (NLP), which can gradually bridge the domain gap and transfer large-scale data knowledge to any specific domain (e.g., from nature scene domain to RSD) In addition, the proposed CSPT also can release the huge potential of unlabeled data for task-aware model training. Finally, extensive experiments were carried out on twelve remote sensing datasets involving three types of downstream tasks (e.g., scene classification, object detection and land cover classification) and two types of imaging data (e.g., optical and synthetic aperture radar (SAR)). The results show that by utilizing the proposed CSPT for task-aware model training, almost all downstream tasks in the RSD can outperform the previous knowledge transfer learning strategies based on model pre-training without any expensive manually labeling and even surpass the state-of-the-art (SOTA) performance without any careful network architecture designing. [ABSTRACT FROM AUTHOR]

Due to the proliferation of precision agriculture, the obstacle of estimating evapotranspiration (ET) and its components from shadow pixels acquired from remote sensing technology should not be neglected. To accurately detect shaded soil and leaf pixels and quantify the implications of shadow pixels on ET inversion, a two-year field-scale observation was carried out in the growing season for a pinot noir vineyard. Based on high-resolution remote sensing sensors covering visible light, thermal infrared, and multispectral light, the supervised classification was applied to detect shadow pixels. Then, we innovatively combined the normalized difference vegetation index with the three-temperature model to quantify the proportion of plant transpiration (T) and soil evaporation (E) in the vineyard ecosystem. Finally, evaluated with the eddy covariance system, we clarified the implications of the shadow pixels on the ET estimation and the spatiotemporal patterns of ET in a vineyard system by considering where shadow pixels were presented. Results indicated that the shadow detection process significantly improved reliable assessment of ET and its components. (1) The shaded soil pixels misled the land cover classification, with the mean canopy cover ignoring shadows 1.68–1.70 times more often than that of shaded area removal; the estimation accuracy of ET can be improved by 4.59–6.82% after considering the effect of shaded soil pixels; and the accuracy can be improved by 0.28–0.89% after multispectral correction. (2) There was a 2 °C canopy temperature discrepancy between sunlit leaves and shaded leaves, meaning that the estimation accuracy of T can be improved by 1.38–7.16% after considering the effect of shaded canopy pixels. (3) Simultaneously, the characteristics showed that there was heterogeneity of ET in the vineyard spatially and that E and T fluxes accounted for 238.05 and 208.79 W·m−2, respectively; the diurnal variation represented a single-peak curve, with a mean of 0.26 mm/h. Our findings provide a better understanding of the influences of shadow pixels on ET estimation using remote sensing techniques. [ABSTRACT FROM AUTHOR]

Remote sensing images contain complex backgrounds and multi-scale objects, which pose a challenging task for scene classification. The performance is highly dependent on the capacity of the scene representation as well as the discriminability of the classifier. Although multiple models possess better properties than a single model on these aspects, the fusion strategy for these models is a key component to maximize the final accuracy. In this paper, we construct a novel dual-model architecture with a grouping-attention-fusion strategy to improve the performance of scene classification. Specifically, the model employs two different convolutional neural networks (CNNs) for feature extraction, where the grouping-attention-fusion strategy is used to fuse the features of the CNNs in a fine and multi-scale manner. In this way, the resultant feature representation of the scene is enhanced. Moreover, to address the issue of similar appearances between different scenes, we develop a loss function which encourages small intra-class diversities and large inter-class distances. Extensive experiments are conducted on four scene classification datasets include the UCM land-use dataset, the WHU-RS19 dataset, the AID dataset, and the OPTIMAL-31 dataset. The experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-arts. [ABSTRACT FROM AUTHOR]