Search

The classification of remote sensing images is inherently challenging due to the complexity, diversity, and sparsity of the data across different image samples. Existing advanced methods often require substantial modifications to model architectures to achieve optimal performance, resulting in complex frameworks that are difficult to adapt. To overcome these limitations, we propose a lightweight ensemble method, enhanced by pure data correction, called the Exceptionally Straightforward Ensemble. This approach eliminates the need for extensive structural modifications to models. A key innovation in our method is the introduction of a novel strategy, quantitative augmentation, implemented through a plug-and-play module. This strategy effectively corrects feature distributions across remote sensing data, significantly improving the performance of Convolutional Neural Networks and Vision Transformers beyond traditional data augmentation techniques. Furthermore, we propose a straightforward algorithm to generate an ensemble network composed of two components, serving as the proposed lightweight classifier. We evaluate our method on three well-known datasets, with results demonstrating that our ensemble models outperform 48 state-of-the-art methods published since 2020, excelling in accuracy, inference speed, and model compactness. Specifically, our models achieve an overall accuracy of up to 96.8%, representing a 1.1% improvement on the challenging NWPU45 dataset. Moreover, the smallest model in our ensemble reduces parameters by up to 90% and inference time by 74%. Notably, our approach significantly enhances the performance of Convolutional Neural Networks and Vision Transformers, even with limited training data, thus alleviating the performance dependence on large-scale datasets. In summary, our data-driven approach offers an efficient, accessible solution for remote sensing image classification, providing an elegant alternative for researchers in geoscience fields who may have limited time or resources for model optimization. [ABSTRACT FROM AUTHOR]

This paper introduces an end-to-end approach for land cover classification utilizing high-resolution remote sensing images (HRRSI), leveraging an Interval Type-2 Fuzzy Convolutional Neural Network (IT2FCNN). This method employs fuzzy logic for nonlinear pixel mapping and adaptively identifies the bounds of Interval Type-2 Fuzzy Sets through fuzzy convolution operations. By incorporating multivariate Type-1 membership functions into conventional convolutional kernels, we have engineered fuzzy convolutional kernels. These kernels, along with fuzzy rule libraries, activate features derived from fuzzy convolutions, facilitating the iterative refinement of the model’s fuzzy sets. This hierarchical process culminates in the development of the IT2FCNN model. When applied to the Wuhan dense labeling dataset (WHDLD), our proposed method outperformed the latest Interval Type-2 Fuzzy Neural Network by 5.27% in accuracy across nine land cover categories. Furthermore, it demonstrated a 17.52% increase in accuracy on the UC Merced Land Use Dataset (UCM Dataset), particularly in dense residential areas, and an 18.3% improvement in sparse residential areas across eleven land cover categories. These results highlight the approach’s effectiveness in mitigating the impact of regional noise on land cover classification, showcasing its strong generalization capability and superior classification accuracy. [ABSTRACT FROM AUTHOR]

In the past few years, the swift advancement of remote sensing technology has greatly promoted its widespread application in the agricultural field. For example, remote sensing technology is used to monitor the planting area and growth status of crops, classify crops, and detect agricultural disasters. In these applications, the accuracy of image classification is of great significance in improving the efficiency and sustainability of agricultural production. However, many of the existing studies primarily rely on contrastive self‐supervised learning methods, which come with certain limitations such as complex data construction and a bias towards invariant features. To address these issues, additional techniques like knowledge distillation are often employed to optimize the learned features. In this article, we propose a novel approach to enhance feature acquisition specific to remote sensing images by introducing a classification‐based self‐supervised auxiliary task. This auxiliary task involves performing image transformation self‐supervised learning tasks directly on the remote sensing images, thereby improving the overall capacity for feature representation. In this work, we design a texture fading reinforcement auxiliary task to reinforce texture features and color features that are useful for distinguishing similar classes of remote sensing. Different auxiliary tasks are fused to form a multi‐view self‐supervised auxiliary task and integrated with the main task to optimize the model training in an end‐to‐end manner. The experimental results on several popular few‐shot remote sensing image datasets validate the effectiveness of the proposed method. The performance better than many advanced algorithms is achieved with a more concise structure. [ABSTRACT FROM AUTHOR]

Vision Transformers (ViTs) have demonstrated exceptional accuracy in classifying remote sensing images (RSIs). However, existing knowledge distillation (KD) methods for transferring representations from a large ViT to a more compact Convolutional Neural Network (CNN) have proven ineffective. This limitation significantly hampers the remarkable generalization capability of ViTs during deployment due to their substantial size. Contrary to common beliefs, we argue that domain discrepancies along with the RSI inherent natures constrain the effectiveness and efficiency of cross-modal knowledge transfer. Consequently, we propose a novel Variance Consistency Learning (VCL) strategy to enhance the efficiency of the cross-modal KD process, implemented through a plug-and-plug module within a ViTteachingCNN pipeline. We evaluated our student model, termed VCL-Net, on three RSI datasets. The results reveal that VCL-Net exhibits superior accuracy and a more compact size compared to 33 other state-of-the-art methods published in the past three years. Specifically, VCL-Net surpasses other KD-based methods with a maximum improvement in accuracy of 22% across different datasets. Furthermore, the visualization analysis of model activations reveals that VCL-Net has learned long-range dependencies of features from the ViT teacher. Moreover, the ablation experiments suggest that our method has reduced the time costs in the KD process by at least 75%. Therefore, our study offers a more effective and efficient approach for cross-modal knowledge transfer when addressing domain discrepancies. [ABSTRACT FROM AUTHOR]

Classification of remote sensing image scenes is an important and challenging task in understanding the Earth's surface and its changes. At the same time, classification is a complex task because of the high variability, multifaceted composition, and dimensionality present in data. The Attention U-Net was originally designed for image segmentation rather than classification. In the proposed framework Attention U-Net is modified, and modifications are made by incorporating a classification layer to AttentionUNet. The full connection layer of AttentionUNet is utilized as the base learner for XGBoost, to develop an efficient framework for remote sensing image classification. However, adapting AttentionUnet for classification presents several advantages such as harnessing attention mechanisms to emphasize pertinent image regions and potentially increasing classification accuracy. Additionally, it enables the exploitation of the U-Net's multi-scale contextual capabilities, aiding in the classification tasks. The proposed approach was evaluated on a dataset of high-resolution remote sensing images from the NWPU-RESISC45 and RSI-CB256 datasets. The results show that the proposed approach has outperformed other baseline models by achieving an overall accuracy of 92.67% in remote sensing image classification. The proposed architecture demonstrates the potential of combining Attention U-Net and XGBoost and highlights the importance of considering both the spatial and contextual information present in remote sensing images. [ABSTRACT FROM AUTHOR]

The classification of remote sensing images (RSIs) poses a significant challenge due to the presence of clustered ground objects and noisy backgrounds. While many approaches rely on scaling models to enhance accuracy, the deployment of RSI classifiers often requires substantial computational resources, thus necessitating the use of lightweight algorithms. In this paper, we present an efficient and robust knowledge transfer network named ERKT-Net, which is designed to provide a lightweight yet accurate convolutional neural network (CNN) classifier. This method utilizes innovative yet straightforward concepts to better accommodate the inherent nature of RSIs, thereby significantly improving the efficiency and robustness of traditional knowledge distillation (KD) techniques developed on ImageNet-1K. We evaluate ERKT-Net on three benchmark RSI datasets. The results demonstrate that our model presents superior accuracy and a very compact size compared to 40 other advanced methods published between 2020 and 2023. On the most challenging NWPU45 dataset, ERKT-Net outperformed other KD-based methods with a maximum overall accuracy (OA) value of 22.4%. Using the same criterion, it also surpassed the first-ranked multi-model method with a minimum OA value of 0.6 but presented at least a 95% reduction in parameters. Furthermore, ablation experiments indicated that our training approach has significantly improved the efficiency and robustness of classic DA techniques. Notably, it can reduce the time expenditure in the distillation phase by at least 80%, with a slight sacrifice in accuracy. This study confirmed that a logit-based KD technique can be more efficient and effective in developing lightweight yet accurate classifiers, especially when the method is tailored to the inherent characteristics of RSIs. [ABSTRACT FROM AUTHOR]

Vision Transformers (ViTs) are exceptional at vision tasks. However, when applied to remote sensing images (RSIs), existing methods often necessitate extensive modifications of ViTs to rival convolutional neural networks (CNNs). This requirement significantly impedes the application of ViTs in geosciences, particularly for researchers who lack the time for comprehensive model redesign. To address this issue, we introduce the concept of quantitative regularization (QR), designed to enhance the performance of ViTs in RSI classification. QR represents an effective algorithm that adeptly manages domain discrepancies in RSIs and can be integrated with any ViTs in transfer learning. We evaluated the effectiveness of QR using three ViT architectures: vanilla ViT, Swin‐ViT and Next‐ViT, on four datasets: AID30, NWPU45, AFGR50 and UCM21. The results reveal that our Next‐ViT model surpasses 39 other advanced methods published in the past 3 years, maintaining robust performance even with a limited number of training samples. We also discovered that our ViT and Swin‐ViT achieve significantly higher accuracy and robustness compared to other methods using the same backbone. Our findings confirm that ViTs can be as effective as CNNs for RSI classification, regardless of the dataset size. Our approach exclusively employs open‐source ViTs and easily accessible training strategies. Consequently, we believe that our method can significantly lower the barriers for geoscience researchers intending to use ViT for RSI applications. [ABSTRACT FROM AUTHOR]

Abstract In various fields, knowledge distillation (KD) techniques that combine vision transformers (ViTs) and convolutional neural networks (CNNs) as a hybrid teacher have shown remarkable results in classification. However, in the realm of remote sensing images (RSIs), existing KD research studies are not only scarce but also lack competitiveness. This issue significantly impedes the deployment of the notable advantages of ViTs and CNNs. To tackle this, the authors introduce a novel hybrid‐model KD approach named HMKD‐Net, which comprises a CNN‐ViT ensemble teacher and a CNN student. Contrary to popular opinion, the authors posit that the sparsity in RSI data distribution limits the effectiveness and efficiency of hybrid‐model knowledge transfer. As a solution, a simple yet innovative method to handle variances during the KD phase is suggested, leading to substantial enhancements in the effectiveness and efficiency of hybrid knowledge transfer. The authors assessed the performance of HMKD‐Net on three RSI datasets. The findings indicate that HMKD‐Net significantly outperforms other cutting‐edge methods while maintaining a significantly smaller size. Specifically, HMKD‐Net exceeds other KD‐based methods with a maximum accuracy improvement of 22.8% across various datasets. As ablation experiments indicated, HMKD‐Net has cut down on time expenses by about 80% in the KD process. This research study validates that the hybrid‐model KD technique can be more effective and efficient if the data distribution sparsity in RSIs is well handled.

Deep convolutional neural networks (DCNNs) have emerged as powerful tools in diverse remote sensing domains, but their optimization remains challenging due to their complex nature and the large number of parameters involved. Researchers have been exploring more sophisticated methodologies to improve image classification accuracy. In this paper, we introduce a multi-granularity feature encoding ensemble network (MGFEEN) that is designed to fine-tune features at different levels of granularity. The network is trained in a two-step process: First, the output of granularity level i is used as the input for the next level; then, a fully connected layer is added to the pre-trained network to advance to the next level. The effectiveness of the MGFEEN's feature extraction is evaluated by feeding the globally extracted features to a softmax classifier for classification. By applying ensemble learning principles, our proposed MGFEEN achieves more accurate final predictions. We evaluate our model on three widely recognized benchmark datasets: UC-Merced, SIRIWHU, and EAC-Dataset. Notably, on the EAC-Dataset, our results show a significant 0.54% improvement in accuracy over a single-training-network setup, resulting in an impressive 98.70% accuracy level. [ABSTRACT FROM AUTHOR]

Superpixel segmentation is an essential step of object-oriented remote sensing image classification; the accuracy of the superpixel segmentation boundary will directly affect the classification result. Most of the traditional superpixel segmentation algorithms rely on spectral similarity and spatial connectivity to construct superpixels. They cannot find the accurate boundary in the complex scenes, such as the spatial distribution of ground features being relatively broken, and large differences in the size and shape, especially long-thin shape and circular shape. Aiming at this problem, a superpixel segmentation algorithm based on an anisotropic diffusion model named ADS is proposed and applied to image classification. The anisotropic diffusion model originated in thermodynamics has excellent properties in which the diffusion is continuous and smooth and its diffusion speed depends on the medium, which provides convenience for smoothing homogeneous regions and establishing boundary constraints for different ground objects. With this advantage, the diffusion flux model is established to consider the influence of boundary factors and used to simulate the dissimilarity measure with boundary constraints between pixels and seed points by combining the traditional spectral and spatial distance. Then, the seed points of superpixel are optimized under the K-means framework. The effectiveness of the proposed algorithm is tested and verified with different spatial resolutions, such as Landsat 8 with 30 m, Sentinel-2 with 10 m, and SkySat with 0.5 m. A large number of experiments show that the proposed algorithm can better correct the superpixel boundary-fitting deviation problem in complex scenes and effectively promote the improvement of image classification accuracy.

In the realm of urban development, the precise classification and identification of land types are crucial for improving land use efficiency. This article proposes a land recognition and classification method based on data sparsity and improved Soft Hirschfeld-Gebelein-Rényi (Soft-HGR) under multimodal conditions. First, a sparse information processing module is designed to enhance information accuracy and quickly obtain data sample features. Then, to solve the problem of information independence in single mode and lack of fusion in multimodal mode, an improved SoftHGR module is developed. This module incorporates covariance and trace constraints, enhances machine learning efficiency by stabilizing output and addressing dimensionality and variance issues in HGR, and speeds up land classification by cross-fusing multimodal features to deepen the understanding of diverse information interconnections. Based on this, a multimodal MI-SoftHGR fusion network is constructed, which can achieve cross-correlation sharing and collaborative extraction of feature information, thereby realizing accurate remote sensing image recognition and classification under multimodal conditions. Finally, empirical evaluations were conducted on Berlin, Augsburg, and MUUFL datasets, and the proposed method was compared with state-of-the-art algorithms. The results fully validate the efficacy and significant superiority of the proposed method.

Remote sensing image classification (RSIC) is designed to assign specific semantic labels to aerial images, which is significant and fundamental in many applications. In recent years, substantial work has been conducted on RSIC with the help of deep learning models. Even though these models have greatly enhanced the performance of RSIC, the issues of diversity in the same class and similarity between different classes in remote sensing images remain huge challenges for RSIC. To solve these problems, a duplex-hierarchy representation learning (DHRL) method is proposed. The proposed DHRL method aims to explore duplex-hierarchy spaces, including a common space and a label space, to learn discriminative representations for RSIC. The proposed DHRL method consists of three main steps: First, paired images are fed to a pretrained ResNet network for extracting the corresponding features. Second, the extracted features are further explored and mapped into a common space for reducing the intra-class scatter and enlarging the inter-class separation. Third, the obtained representations are used to predict the categories of the input images, and the discrimination loss in the label space is minimized to further promote the learning of discriminative representations. Meanwhile, a confusion score is computed and added to the classification loss for guiding the discriminative representation learning via backpropagation. The comprehensive experimental results show that the proposed method is superior to the existing state-of-the-art methods on two challenging remote sensing image scene datasets, demonstrating that the proposed method is significantly effective. [ABSTRACT FROM AUTHOR]

The study of few-shot remote sensing image classification has received significant attention. Although meta-learning-based algorithms have been the primary focus of recent examination, feature fusion methods stress feature extraction and representation. Nonetheless, current feature fusion methods, like the multi-head mechanism, are restricted by their complicated network structure and challenging training process. This manuscript presents a simplified multi-head mechanism for obtaining multiple feature representations from a single sample. Furthermore, we perform specific fundamental transformations on remote-sensing images to obtain more suitable features for information representation. Specifically, we reduce multiple feature extractors of the multi-head mechanism to a single one and add an image transformation module before the feature extractor. After transforming the image, the features are extracted resulting in multiple features for each sample. The feature fusion stage is integrated with the classification prediction stage, and multiple linear classifiers are combined for multi-decision fusion to complete feature fusion and classification. By combining image transformation with feature decision fusion, we compare our results with other methods through validation tests and demonstrate that our algorithm simplifies the multi-head mechanism while maintaining or improving classification performance. [ABSTRACT FROM AUTHOR]

In the study of remote sensing image classification, feature extraction and selection is an effective method to distinguish different classification targets. Constructing a high-quality spectral-spatial feature and feature combination has been a worthwhile topic for improving classification accuracy. In this context, this study constructed a spectral-spatial feature, namely the Pixel Neighbourhood Similarity (PNS) index. Meanwhile, the PNS index and 19 spectral, textural and terrain features were involved in the Correlation-based Feature Selection (CFS) algorithm for feature selection to generate a feature combination (PNS-CFS). To explore how PNS and PNS-CFS improve the classification accuracy of land types. The results show that: (1) The PNS index exhibited clear boundaries between different land types. The performance quality of PNS was relatively highest compared to other spectral-spatial features, namely the Vector Similarity (VS) index, the Change Vector Intensity (CVI) index and the Correlation (COR) index. (2) The Overall Accuracy (OA) of the PNS-CFS was 94.66% and 93.59% in study areas 1 and 2, respectively. These were 7.48% and 6.02% higher than the original image data (ORI) and 7.27% and 2.39% higher than the single-dimensional feature combination (SIN-CFS). Compared to the feature combinations of VS, CVI, and COR indices (VS-CFS, CVI-COM, COR-COM), PNS-CFS had the relatively highest performance and classification accuracy. The study demonstrated that the PNS index and PNS-CFS have a high potential for image classification.

This paper proposes an automatic method for land cover change detection in very-high-spatial-resolution optical remote sensing images based on the automatic selection of training samples using expectation-maximization (EM) and an extreme learning machine classifier, which has two key characteristics: (1) combining the advantages of supervised and unsupervised methods with progressive mask classification. (2) training samples of three classes (changed, unchanged, and unknown) were automatically selected by K-Means and EM, and then further refined by the likelihood function. The method was validated by one dataset of SuperView-1 imagery with a spatial resolution of 2.0 m, two datasets of TripleSat-2 imagery with a spatial resolution of 3.2 m, and one open dataset of Zi-Yuan-3 imagery with a spatial resolution of 5.8 m, and the results were compared with that of three unsupervised methods (iterative slow feature analysis, multivariate alteration detection, and adaptive object-oriented spatial-contextual extraction algorithm), two deep learning methods (convolutional-wavelet neural networks and dual-domain networks), and three supervised classifiers (support vector machine, random forest, and Naive Bayes), showing the effectiveness of this method in decrease of false-positive rates and increase of change detection accuracy. The average and maximum of the accuracy metric F1 score of our method are 0.6842 and 0.8708, respectively; the average F1 score of the unsupervised, deep learning, and supervised methods are 0.5049, 0.4943, and 0.633, respectively. [ABSTRACT FROM AUTHOR]

With the development of deep learning, image recognition based on deep learning is now widely used in remote sensing. As we know, the effectiveness of deep learning models significantly benefits from the size and quality of the dataset. However, remote sensing data are often distributed in different parts. They cannot be shared directly for privacy and security reasons, and this has motivated some scholars to apply federated learning (FL) to remote sensing. However, research has found that federated learning is usually vulnerable to white-box membership inference attacks (MIAs), which aim to infer whether a piece of data was participating in model training. In remote sensing, the MIA can lead to the disclosure of sensitive information about the model trainers, such as their location and type, as well as time information about the remote sensing equipment. To solve this issue, we consider embedding local differential privacy (LDP) into FL and propose LDP-Fed. LDP-Fed performs local differential privacy perturbation after properly pruning the uploaded parameters, preventing the central server from obtaining the original local models from the participants. To achieve a trade-off between privacy and model performance, LDP-Fed adds different noise levels to the parameters for various layers of the local models. This paper conducted comprehensive experiments to evaluate the framework's effectiveness on two remote sensing image datasets and two machine learning benchmark datasets. The results demonstrate that remote sensing image classification models are susceptible to MIAs, and our framework can successfully defend against white-box MIA while achieving an excellent global model. [ABSTRACT FROM AUTHOR]

Few-shot image classification is currently valuable research in the field of remote sensing image applications. The difficulty in obtaining training data for a large number of labelled remote sensing images leads to difficulties in applying traditional deep learning-based remote sensing image classification methods. Existing few-shot remote sensing image classification methods usually use a large number of base datasets to train network models in the pre-training phase and perform few-shot classification tasks after model fine-tuning on new datasets in the meta-testing phase. However, an insufficient number of remote sensing images lead to inaccurate features being extracted by the model, and the large amount of new class data for fine-tuning is also not easily accessible. In addition, using a pre-trained model to extract features from the new dataset will create a"negative transfer" problem. In this paper, we aim to address the above challenges in two ways. First, we use a subset of ImageNet, a readily available few-shot natural image dataset, for model pre-training, and eliminate the fine-tuning operation to simulate a real-world application scenario where remote sensing data is extremely scarce. Second, due to the significant differences in scale and style between the ImageNet dataset and the remote sensing dataset, which lead to a serious negative transfer problem in the model, we design a multi-scale feature fusion module to comprehensively decide the labels of query samples considering all scales to compensate for the scale differences and alleviate the"negative transfer" problem. We conducted experiments on four benchmark remote sensing datasets and achieved satisfactory performance. [ABSTRACT FROM AUTHOR]

Recently, researchers have proposed a lot of methods to boost the performance of convolutional neural networks (CNNs) for classifying remote sensing images (RSI). However, the methods' performance improvements were insignificant, while time and hardware costs increased dramatically due to re-modeling. To tackle this problem, this study sought a simple, lightweight, yet more accurate solution for RSI semantic classification (RSI-SC). At first, we proposed a set of mathematical derivations to analyze and identify the best way among different technical roadmaps. Afterward, we selected a simple route that can significantly boost a single CNN's performance while maintaining simplicity and reducing costs in time and hardware. The proposed method, called RE-EfficientNet, only consists of a lightweight EfficientNet-B3 and a concise training algorithm named RE-CNN. The novelty of RE-EfficientNet and RE-CNN includes the following: First, EfficientNet-B3 employs transfer learning from ImageNet-1K and excludes any complicated re-modeling. It can adequately utilize the easily accessible pre-trained weights for time savings and avoid the pre-training effect being weakened due to re-modeling. Second, RE-CNN includes an effective combination of data augmentation (DA) transformations and two modified training tricks (TTs). It can alleviate the data distribution shift from DA-processed training sets and make the TTs more effective through modification according to the inherent nature of RSI. Extensive experimental results on two RSI sets prove that RE-EfficientNet can surpass all 30 cutting-edge methods published before 2023. It gives a remarkable improvement of 0.50% to 0.75% in overall accuracy (OA) and a 75% or more reduction in parameters. The ablation experiment also reveals that RE-CNN can improve CNN OA by 0.55% to 1.10%. All the results indicate that RE-EfficientNet is a simple, lightweight and more accurate solution for RSI-SC. In addition, we argue that the ideas proposed in this work about how to choose an appropriate model and training algorithm can help us find more efficient approaches in the future. [ABSTRACT FROM AUTHOR]

With the improvement of remote sensing image resolution, remote sensing image scene classification has become a major difficulty in the research of remote sensing Urban green space spatial layout and site selection. Complex data and network structure affect the processing effect of the traditional Convolutional neural network model, so it is particularly important to design a more efficient Convolutional neural network. This research will first expand the convolution design of the Convolutional neural network to improve the scope of model recognition, then select two methods of structural pruning and separable knowledge distillation for lightweight processing of the model, and finally introduce relevant models for comparative experiments to verify the lightweight effect of the model. The experimental results show that the global average accuracy of the lightweight model based on structural pruning is 95.5%, and the Kappa coefficient value is 0.947; The global classification accuracy of the lightweight model based on knowledge distillation reaches 95.60%, with a Kappa coefficient value of 0.939. It only uses 38.419MB of storage space to recognize a remote sensing image, 4698352 model parameters, and 1397527639 floating-point operations per second. The results show that the expansion convolution and network pruning methods improve the classification performance of the Convolutional neural network, improve the accuracy of the model, and the knowledge distillation method has a better effect in reducing the complexity and making up for the loss of the classification performance of the network model.

Accurate land use and land cover (LULC) maps are effective tools to help achieve sound urban planning and precision agriculture. As an intelligent optimization technology, genetic algorithm (GA) has been successfully applied to various image classification tasks in recent years. However, simple GA faces challenges, such as complex calculation, poor noise immunity, and slow convergence. This research proposes a two-step ensemble protocol for LULC classification using a grayscale-spatial-based GA model. The first ensemble framework uses fuzzy c-means to classify pixels into those that are difficult to cluster and those that are easy to cluster, which aids in reducing the search space for evolutionary computation. The second ensemble framework uses neighborhood windows as heuristic information to adaptively modify the objective function and mutation probability of the GA, which brings valuable benefits to the discrimination and decision of GA. In this study, three research areas in Dangyang, China, are utilized to validate the effectiveness of the proposed method. The experiments show that the proposed method can effectively maintain the image details, restrain noise, and achieve rapid algorithm convergence. Compared with the reference methods, the best overall accuracy obtained by the proposed algorithm is 88.72%.

With the development of remote sensing (RS) technology, single-modal data alone has gradually become difficult to meet the requirement for high accuracy of RS image classification. As a result, multi-modal RS image classification has become a hot research topic. However, multi-modal RS image classification faces challenges in effectively utilizing advanced semantic information regarding the relationships between land cover classes and extracting discriminative features from the data. Based on this, a multi-modal RS image classification method based on knowledge graph (KG) is proposed. Firstly, graph topology is used to align the hyperspectral image (HSI) and multispectral image (MSI) features extracted by graph convolution networks. Constraint alignment is performed on different feature graphs to reduce the difficulty of fusion and the false recognition rate. Then, we use self-attention and cross-attention to purposefully fuse HSI and MSI to obtain discriminative features rich in two modal information and achieve feature weighted fusion. Finally, a KG based on object spatial relationships is constructed to obtain spatial relationships between different classes to assist in multi-modal RS image classification. The experimental results on the Houston and Ausburg datasets demonstrate that the proposed method achieves overall accuracy of 90.40% and 90.85%, respectively, both of which are more than 3% higher than existing classification methods. The results indicate that our method has better classification performance and can provide a useful reference for RS image classification research. [ABSTRACT FROM AUTHOR]

Generating learning-friendly representations for points in space is a fundamental and long-standing problem in machine learning. Recently, multi-scale encoding schemes (such as Space2Vec and NeRF) were proposed to directly encode any point in 2D or 3D Euclidean space as a high-dimensional vector, and has been successfully applied to various (geo)spatial prediction and generative tasks. However, all current 2D and 3D location encoders are designed to model point distances in Euclidean space. So when applied to large-scale real-world GPS coordinate datasets (e.g., species or satellite images taken all over the world), which require distance metric learning on the spherical surface, both types of models can fail due to the map projection distortion problem (2D) and the spherical-to-Euclidean distance approximation error (3D). To solve these problems, we propose a multi-scale location encoder called Sphere2Vec which can preserve spherical distances when encoding point coordinates on a spherical surface. We developed a unified view of distance-reserving encoding on spheres based on the Double Fourier Sphere (DFS). We also provide theoretical proof that the Sphere2Vec encoding preserves the spherical surface distance between any two points, while existing encoding schemes such as Space2Vec and NeRF do not. Experiments on 20 synthetic datasets show that Sphere2Vec can outperform all baseline models including the state-of-the-art (SOTA) 2D location encoder (i.e., Space2Vec) and 3D encoder NeRF on all these datasets with up to 30.8% error rate reduction. We then apply Sphere2Vec to three geo-aware image classification tasks - fine-grained species recognition, Flickr image recognition, and remote sensing image classification. Results on 7 real-world datasets show the superiority of Sphere2Vec over multiple 2D and 3D location encoders on all three tasks. Further analysis shows that Sphere2Vec outperforms other location encoder models, especially in the polar regions and data-sparse areas because of its nature for spherical surface distance preservation. Code and data of this work are available at https://gengchenmai.github.io/sphere2vec-website/. • We propose Sphere2Vec , the first location encoder can preserve spherical distance. • We provide a theoretical proof about the spherical-distance-kept nature of Sphere2Vec. • We provide theoretical proof showing that Space2Vec and NeRF cannot correctly model spherical distance. • Sphere2Vec can outperform all baselines on 20 synthetic datasets and 7 real-world datasets. • Further analysis shows that Sphere2Vec excels in the polar regions and data-sparse areas. [ABSTRACT FROM AUTHOR]

Network pruning has been widely used in model compression techniques, and offers a promising prospect for deploying models on devices with limited resources. Nevertheless, existing pruning methods merely consider the importance of feature maps and filters in the spatial domain. In this paper, we re-consider the model characteristics and propose a novel filter pruning method that corresponds to the human visual system, termed Low Frequency Preference (LFP), in the frequency domain. It is essentially an indicator that determines the importance of a filter based on the relative low-frequency components across channels, which can be intuitively understood as a measurement of the "low-frequency components". When the feature map of a filter has more low-frequency components than the other feature maps, it is considered more crucial and should be preserved during the pruning process. We conduct the proposed LFP on three different scales of datasets through several models and achieve superior performances. The experimental results obtained on the CIFAR datasets and ImageNet dataset demonstrate that our method significantly reduces the model size and FLOPs. The results on the UC Merced dataset show that our approach is also significant for remote sensing image classification. [ABSTRACT FROM AUTHOR]

The application of deep learning in remote sensing image classification has been paid more and more attention by industry and academia. However, manually designed remote sensing image classification models based on convolutional neural networks usually require sophisticated expert knowledge. Moreover, it is notoriously difficult to design a model with both high classification accuracy and few parameters. Recently, neural architecture search (NAS) has emerged as an effective method that can greatly reduce the heavy burden of manually designing models. However, it remains a challenge to search for a classification model with high classification accuracy and few parameters in the huge search space. To tackle this challenge, we propose TPENAS, a two-phase evolutionary neural architecture search framework, which optimizes the model using computational intelligence techniques in two search phases. In the first search phase, TPENAS searches for the optimal depth of the model. In the second search phase, TPENAS searches for the structure of the model from the perspective of the whole model. Experiments on three open benchmark datasets demonstrate that our proposed TPENAS outperforms the state-of-the-art baselines in both classification accuracy and reducing parameters. [ABSTRACT FROM AUTHOR]

Remote sensing image classification (RSIC) is a classical and fundamental task in the intelligent interpretation of remote sensing imagery, which can provide unique labeling information for each acquired remote sensing image. Thanks to the potent global context information extraction ability of the multi-head self-attention (MSA) mechanism, visual transformer (ViT)-based architectures have shown excellent capability in natural scene image classification. However, in order to achieve powerful RSIC performance, it is insufficient to capture global spatial information alone. Specifically, for fine-grained target recognition tasks with high inter-class similarity, discriminative and effective local feature representations are key to correct classification. In addition, due to the lack of inductive biases, the powerful global spatial context representation capability of ViT requires lengthy training procedures and large-scale pre-training data volume. To solve the above problems, a hybrid architecture of convolution neural network (CNN) and ViT is proposed to improve the RSIC ability, called P 2 FEViT, which integrates plug-and-play CNN features with ViT. In this paper, the feature representation capabilities of CNN and ViT applying for RSIC are first analyzed. Second, aiming to integrate the advantages of CNN and ViT, a novel approach embedding CNN features into the ViT architecture is proposed, which can make the model synchronously capture and fuse global context and local multimodal information to further improve the classification capability of ViT. Third, based on the hybrid structure, only a simple cross-entropy loss is employed for model training. The model can also have rapid and comfortable convergence with relatively less training data than the original ViT. Finally, extensive experiments are conducted on the public and challenging remote sensing scene classification dataset of NWPU-RESISC45 (NWPU-R45) and the self-built fine-grained target classification dataset called BIT-AFGR50. The experimental results demonstrate that the proposed P 2 FEViT can effectively improve the feature description capability and obtain outstanding image classification performance, while significantly reducing the high dependence of ViT on large-scale pre-training data volume and accelerating the convergence speed. The code and self-built dataset will be released at our webpages. [ABSTRACT FROM AUTHOR]

For the past few years, the barrier of explainability accompanying by deep neural networks (DNNs) has been increasingly studied. The methods based on class activation map (CAM) which interpret the model decision by mapping the output back to the input space, have achieved a notable momentum among the research. However, the CAM-based methods cannot stably produce effective explanation results on remote sensing images (RSIs), owing to the coarse location map generated by high-level features, whereas, the RSIs contain abundant detailed spatial information and multi-scale objects. To address this issue, this article proposes class activation map weighted with channel saliency and gradient (CSG-CAM) to enhance the low-level features in saliency map. To do this, we firstly introduce the idea of dynamic channel pruning and propose the channel saliency to describe the channel importance of specific layer. Then the channel saliency, instead of gradient, is considered as the neuron importance weights to calculate the saliency map on shallow layer. Furthermore, the channel saliency also participates in the neuron importance weights of final layer, jointly with a gradient weighted combination of the positive partial derivatives. Finally, the saliency map of proposed CSG-CAM is fused by the explanation heat maps from shallow and final layer of networks. The sufficient experimental results on two publicly available datasets designed for RSI scene classification and three practical networks demonstrate the effectiveness of the CSG-CAM in terms of both faithfulness and explainability.

Although many machine learning methods have been successfully applied for the object-based classification of high resolution (HR) remote sensing imagery, current methods are highly dependent on the spectral similarity between segmented objects and have disappointingly poor performance when dealing with different segmented objects that have similar spectra. To overcome this limitation, this study exploited a knowledge graph (KG) that preserved the spatial relationships between segmented objects and has a reasoning capability that can assist in improving the probability of correctly classifying different segmented objects with similar spectra. In addition, to assist the knowledge graph classifications, an image segmentation method generating segmented objects that closely resemble real ground objects in size was used, which improves the integrity of the object classification results. Therefore, a novel HR remote sensing image classification scheme is proposed that involves a knowledge graph and an optimal segmentation algorithm, which takes full advantage of object-based classification and knowledge inference. This method effectively addresses the problems of object classification integrity and misclassification of objects with the same spectrum. In the evaluation experiments, three QuickBird-2 images and over 15 different land cover classes were utilized. The results showed that the classification accuracy of the proposed method is high, with overall accuracies exceeding 0.85. These accuracies are higher than the K Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Tree (DT), and Random Forest (RF) methods. The evaluated results confirmed that the proposed method offers excellent performance in HR remote sensing image classification. [ABSTRACT FROM AUTHOR]

Accurate land use and land cover (LULC) maps are effective tools to help achieve sound urban planning and precision agriculture. As an intelligent optimization technology, genetic algorithm (GA) has been successfully applied to various image classification tasks in recent years. However, simple GA faces challenges, such as complex calculation, poor noise immunity, and slow convergence. This research proposes a two-step ensemble protocol for LULC classification using a grayscale-spatial-based GA model. The first ensemble framework uses fuzzy c-means to classify pixels into those that are difficult to cluster and those that are easy to cluster, which aids in reducing the search space for evolutionary computation. The second ensemble framework uses neighborhood windows as heuristic information to adaptively modify the objective function and mutation probability of the GA, which brings valuable benefits to the discrimination and decision of GA. In this study, three research areas in Dangyang, China, are utilized to validate the effectiveness of the proposed method. The experiments show that the proposed method can effectively maintain the image details, restrain noise, and achieve rapid algorithm convergence. Compared with the reference methods, the best overall accuracy obtained by the proposed algorithm is 88.72%. [ABSTRACT FROM AUTHOR]

In the study of remote sensing image classification, feature extraction and selection is an effective method to distinguish different classification targets. Constructing a high-quality spectral-spatial feature and feature combination has been a worthwhile topic for improving classification accuracy. In this context, this study constructed a spectral-spatial feature, namely the Pixel Neighbourhood Similarity (PNS) index. Meanwhile, the PNS index and 19 spectral, textural and terrain features were involved in the Correlation-based Feature Selection (CFS) algorithm for feature selection to generate a feature combination (PNS-CFS). To explore how PNS and PNS-CFS improve the classification accuracy of land types. The results show that: (1) The PNS index exhibited clear boundaries between different land types. The performance quality of PNS was relatively highest compared to other spectral-spatial features, namely the Vector Similarity (VS) index, the Change Vector Intensity (CVI) index and the Correlation (COR) index. (2) The Overall Accuracy (OA) of the PNS-CFS was 94.66% and 93.59% in study areas 1 and 2, respectively. These were 7.48% and 6.02% higher than the original image data (ORI) and 7.27% and 2.39% higher than the single-dimensional feature combination (SIN-CFS). Compared to the feature combinations of VS, CVI, and COR indices (VS-CFS, CVI-COM, COR-COM), PNS-CFS had the relatively highest performance and classification accuracy. The study demonstrated that the PNS index and PNS-CFS have a high potential for image classification. [ABSTRACT FROM AUTHOR]

One important method for classifying images from remote sensing is discriminative feature learning. On the other hand, the majority of the methods that are currently in use do not take into account that the image labels will be tainted and that their performance will suffer as a result of noisy supervised information. However, single-view based feature learning is unable to present the image in its entirety. A multi-view robust discriminative feature learning approach for remote sensing images with noisy labels is proposed in this paper to address these issues. First, the local neighborhood division is carried out with the help of the entropy rate superpixel segmentation. Second, a graph-based multi-view propagation model is used to recover noisy labels in these specific locations. Finally, a multi-view discriminative feature learning model for remote sensing image classification is proposed using recovery labels as supervised information. Extensive testing on three remote sensing image datasets shows that our proposed method outperforms seven other excellent methods and has some advantages in terms of robustness and a number of objective indicators. [ABSTRACT FROM AUTHOR]

Realizing accurate perception of urban boundary changes is conducive to the formulation of regional development planning and researches of urban sustainable development. In this paper, an improved fully convolution neural network was provided for perceiving large-scale urban change, by modifying network structure and updating network strategy to extract richer feature information, and to meet the requirement of urban construction land extraction under the background of large-scale low-resolution image. This paper takes the Yangtze River Economic Belt of China as an empirical object to verify the practicability of the network, the results show the extraction results of the improved fully convolutional neural network model reached a precision of kappa coefficient of 0.88, which is better than traditional fully convolutional neural networks, it performs well in the construction land extraction at the scale of small and medium-sized cities. [ABSTRACT FROM AUTHOR]

Land use/land cover classification of remote sensing images provide information to take efficient decisions related to resource monitoring. There exists several algorithms for remote sensing image classification. In the recent years, Deep learning models like convolution neural networks (CNNs) are widely used for remote sensing image classification. The learning and generalization ability of CNN, results in better performance in comparison with similar type of models. The functional behavior of CNNs is unexplainable because of its multiple layers of convolution and pooling operations. This results in black box characteristics of CNNs. Motivated with this factor, a CNN model with functional transparency is proposed in the present study. The model is named as Knowledge Based Morphological Deep Transparent Neural Networks (KBMDTNN) for remote sensing image classification. The architecture of KBMDTNN model provides functional transparency due to application of morphological operators, convolutional and pooling layers, and transparent neural network. In KBMDTNN model, the morphological operator preserve the shape/size information of the objects through efficient image segmentation. Convolution and pooling layers are used to produce minimal number of features from the image. The operational transparency of proposed model is coined based on the mathematical understanding of each layer in the model instead of randomly adding layers to the architecture of model. The transparency of proposed model is also because of assigning the initial weights of NN in output layer of model with computed values instead of random values. The proposed KBMDTNN model outperformed similar type of models as tested with multispectral and hyperspectral remote sensing images. The performance of KBMDTNN model is evaluated with the metrics like overall accuracy (OA), overall accuracy standard deviation ($OA_{\text{STD}}$), producer’s accuracy (PA), user’s accuracy (UA), dispersion score (DS), and kappa coefficient (KC).

Urban expansion is generally accompanied by a series of ecological problems; therefore, it is of great significance to strengthen the research on urban expansion to effectively guide and control urban expansion. In this study, we used a one-class support vector machine (OCSVM) based on Landsat image data to extract the construction land area in Xiaogan City of Hubei Province (China) in 2000, 2005, 2010, 2015, and 2020. We analyzed the characteristics of construction land expansion and explore the driving mechanisms of construction land expansion in Xiaogan City. The results show that 1) the accuracy of the construction land information extracted by OCSVM was 91.46%, 90.02%, 89.31%, 92.23%, and 89.67%, respectively, which met the expected results and could be used for the study of extension and driving mechanism, proving that OCSVM is suitable for the study of remote sensing image classification when only one class of features is extracted; 2) Xiaogan City's overall expansion is of high-speed type in 20 years, which is restricted by the terrain, medium-speed expansion between 2000 and 2005, and high-speed expansion between 2005 and 2020, and the expansion intensity of Xiaogan City in all four time periods from 2000 to 2020 is of slow expansion type; and 3) among the main factors influencing urban expansion in Xiaogan, the increase in population, economic development, development of high-tech zones, and construction of transportation lines increase the demand for construction land; government planning decisions provide the direction and scope for the expansion of construction land, which is the most leading factor among the drivers of construction land expansion.

Copyright of Geography & Geographic Information Science is the property of Geography & Geo-Information Science Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Aiming at the poor classification effect caused by complex background, large intra-class difference and high interclass similarity in remote sensing scene images, this paper proposed an attentional mechanism and residual contraction unit algorithm based on supervised contrast learning. Firstly, the algorithm improved the effective channel attention mechanism (ECA), and optimized the extraction of image features to be recognized. Then, the algorithm proposed a cooperative residual shrinkage unit algorithm, which is used to filter and eliminate redundant information of images. In addition, supervised contrast learning algorithm was used to enhance the generalization ability of the algorithm. Finally, experiments were carried out with remote sensing image data set and compared with the latest algorithms such as enhanced attention algorithm and scale attention mechanism algorithm. Experimental results show that the proposed algorithm improves the classification accuracy by 1.75% and 2.5% in AID Data Set with 20% training ratio. [ABSTRACT FROM AUTHOR]

The selection and representation of classification features in remote sensing image play crucial roles in image classification accuracy. To effectively improve the features classification accuracy, an improved U-Net remote sensing classification algorithm fusing attention and multiscale features is proposed in this paper, called spatial attention-atrous spatial pyramid pooling U-Net (SA-UNet). This framework connects atrous spatial pyramid pooling (ASPP) with the convolutional units of the encoder of the original U-Net in the form of residuals. The ASPP module expands the receptive field, integrates multiscale features in the network, and enhances the ability to express shallow features. Through the fusion residual module, shallow and deep features are deeply fused, and the characteristics of shallow and deep features are further used. The spatial attention mechanism is used to combine spatial with semantic information so that the decoder can recover more spatial information. In this study, the crop distribution in central Guangxi province was analyzed, and experiments were conducted based on Landsat 8 multispectral remote sensing images. The experimental results showed that the improved algorithm increases the classification accuracy, with the accuracy increasing from 93.33% to 96.25%, The segmentation accuracy of sugarcane, rice, and other land increased from 96.42%, 63.37%, and 88.43% to 98.01%, 83.21%, and 95.71%, respectively. The agricultural planting area results obtained by the proposed algorithm can be used as input data for regional ecological models, which is conducive to the development of accurate and real-time crop growth change models. [ABSTRACT FROM AUTHOR]

The successful application of deep learning approaches in remote sensing image classification requires large hyperspectral image (HSI) datasets to learn discriminative spectral–spatial features simultaneously. To date, the HSI datasets available for image classification are relatively small to train deep learning methods. This study proposes a deep 3D/2D genome graph-based network (abbreviated as HybridGBN-SR) that is computationally efficient and not prone to overfitting even with extremely few training sample data. At the feature extraction level, the HybridGBN-SR utilizes the three-dimensional (3D) and two-dimensional (2D) Genoblocks trained using very few samples while improving HSI classification accuracy. The design of a Genoblock is based on a biological genome graph. From the experimental results, the study shows that our model achieves better classification accuracy than the compared state-of-the-art methods over the three publicly available HSI benchmarking datasets such as the Indian Pines (IP), the University of Pavia (UP), and the Salinas Scene (SA). For instance, using only 5% labeled data for training in IP, and 1% in UP and SA, the overall classification accuracy of the proposed HybridGBN-SR is 97.42%, 97.85%, and 99.34%, respectively, which is better than the compared state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN)-based research has been successfully applied in remote sensing image classification due to its powerful feature representation ability. However, these high-capacity networks bring heavy inference costs and are easily overparameterized, especially for the deep CNNs pretrained on natural image datasets. Network pruning is regarded as a prevalent approach for compressing networks, but most existing research ignores model interpretability while formulating pruning criterion. To address these issues, a network pruning method for remote sensing image classification based on interpretable CNNs is proposed. More specifically, an original interpretable CNN with a predefined pruning ratio is trained at first. The filters, namely channels in the high convolutional layer, are able to learn specific semantic meanings in proportion to the predefined pruning ratio. The filters without interpretability are supposed to be removed. As for other convolutional layers, a sensitivity function is designed to assess the risk of pruning channels for each layer, and furthermore, the pruning ratio for each layer is corrected adaptively. The pruning method based on the proposed sensitivity function is effective and requires little computational costs to search abandoned channels without damaging classification performance. To demonstrate the effectiveness, the proposed method is implemented on different scales of modern CNN models, including VGG-VD and AlexNet. The experimental results, obtained on the UC Merced dataset and NWPU-RESISC45 dataset, prove that our method significantly reduces the inference costs and improves the interpretability of networks. [ABSTRACT FROM AUTHOR]

Recently, time series image (TSI) has been reported to be an effective resource to mapping fine land use/land cover (LULC), and deep learning, in particular, has been gaining growing attention in this field. However, deep learning methods using single classifier need further improvement for accurate TSI classification owing to the 1D temporal properties and insufficient dense time series of the remote sensing images. To overcome such disadvantages, we proposed an innovative approach involving construction of TSI and combination of deep learning and multiple classifiers system (MCS). Firstly, we used a normalised difference index (NDI) to establish an NDIs-based TSI and then designed a framework consisting of a deep learning-based feature extractor and multiple classifiers system (MCS) based classification model to classify the TSI. With the new approach, our experiments were conducted on Landsat images located in two counties, Sutter and Kings in California, United States. The experimental results indicate that our proposed method achieves great progress on accuracy improvement and LULC mapping, outperforming classifications using comparative deep learning and non-deep learning methods.