Search

This document, titled "Computational Intelligence in Remote Sensing," discusses the application of computational intelligence (CI) methods in the field of remote sensing. It highlights recent research and progress in this area, categorizing the papers into four sections: computational intelligence methods in hyperspectral remote sensing images, object detection techniques in remote sensing images, deep learning approaches in remote sensing image classification, and intelligent optimization and control in satellite image applications. The document emphasizes the potential of CI in addressing the challenges of remote sensing and encourages further interdisciplinary cooperation to solve real-world problems. The authors express their gratitude to the contributors and highlight the achievements of the research papers in this journal. [Extracted from the article]

The objectives of this paper are to investigate the trade-offs between a physically constrained neural network and a deep, convolutional neural network and to design a combined ML approach ("VarioCNN"). Our solution is provided in the framework of a cyberinfrastructure that includes a newly designed ML software, GEOCLASS-image (v1.0), modern high-resolution satellite image data sets (Maxar WorldView data), and instructions/descriptions that may facilitate solving similar spatial classification problems. Combining the advantages of the physically-driven connectionist-geostatistical classification method with those of an efficient CNN, VarioCNN provides a means for rapid and efficient extraction of complex geophysical information from submeter resolution satellite imagery. A retraining loop overcomes the difficulties of creating a labeled training data set. Computational analyses and developments are centered on a specific, but generalizable, geophysical problem: The classification of crevasse types that form during the surge of a glacier system. A surge is a glacial catastrophe, an acceleration of a glacier to typically 100–200 times its normal velocity. GEOCLASS-image is applied to study the current (2016-2024) surge in the Negribreen Glacier System, Svalbard. The geophysical result is a description of the structural evolution and expansion of the surge, based on crevasse types that capture ice deformation in six simplified classes. [ABSTRACT FROM AUTHOR]

The census and management of hazard-bearing entities, along with the integrity of data quality, form crucial foundations for disaster risk assessment and zoning. By addressing the challenge of feature confusion, prevalent in single remotely sensed image recognition methods, this paper introduces a novel method, Spatially Constrained Deep Learning (SCDL), that combines deep learning with spatial constraint strategies for the extraction of disaster-bearing bodies, focusing on dams as a typical example. The methodology involves the creation of a dam dataset using a database of dams, followed by the training of YOLOv5, Varifocal Net, Faster R-CNN, and Cascade R-CNN models. These models are trained separately, and highly confidential dam location information is extracted through parameter thresholding. Furthermore, three spatial constraint strategies are employed to mitigate the impact of other factors, particularly confusing features, in the background region. To assess the method's applicability and efficiency, Qinghai Province serves as the experimental area, with dam images from the Google Earth Pro database used as validation samples. The experimental results demonstrate that the recognition accuracy of SCDL reaches 94.73%, effectively addressing interference from background factors. Notably, the proposed method identifies six dams not recorded in the GOODD database, while also detecting six dams in the database that were previously unrecorded. Additionally, four dams misdirected in the database are corrected, contributing to the enhancement and supplementation of the global dam geo-reference database and providing robust support for disaster risk assessment. In conclusion, leveraging open geographic data products, the comprehensive framework presented in this paper, encompassing deep learning target detection technology and spatial constraint strategies, enables more efficient and accurate intelligent retrieval of disaster-bearing bodies, specifically dams. The findings offer valuable insights and inspiration for future advancements in related fields. [ABSTRACT FROM AUTHOR]

Earthquake Disaster Assessment (EDA) plays a critical role in earthquake disaster prevention, evacuation, and rescue efforts. Deep learning (DL), which boasts advantages in image processing, signal recognition, and object detection, has facilitated scientific research in EDA. This paper analyses 204 articles through a systematic literature review to investigate the status quo, development, and challenges of DL for EDA. The paper first examines the distribution characteristics and trends of the two categories of EDA assessment objects, including earthquakes and secondary disasters as disaster objects, buildings, infrastructure, and areas as physical objects. Next, this study analyses the application distribution, advantages, and disadvantages of the three types of data (remote sensing data, seismic data, and social media data) mainly involved in these studies. Furthermore, the review identifies the characteristics and application of six commonly used DL models in EDA, including convolutional neural network (CNN), multi-layer perceptron (MLP), recurrent neural network (RNN), generative adversarial network (GAN), transfer learning (TL), and hybrid models. The paper also systematically details the application of DL for EDA at different times (i.e., pre-earthquake stage, during-earthquake stage, post-earthquake stage, and multi-stage). We find that the most extensive research in this field involves using CNNs for image classification to detect and assess building damage resulting from earthquakes. Finally, the paper discusses challenges related to training data and DL models, and identifies opportunities in new data sources, multimodal DL, and new concepts. This review provides valuable references for scholars and practitioners in related fields. [ABSTRACT FROM AUTHOR]

The advancement of multi-source Earth observation technology has led to a substantial body of literature on inland water monitoring. This has resulted in the emergence of a distinct interdisciplinary field encompassing the application of multi-source Earth observation techniques in inland water monitoring. Despite this growth, few systematic reviews of this field exist. Therefore, in this paper, we offer a comprehensive analysis based on 30,212 publications spanning the years 1990 to 2022, providing valuable insights. We collected and analyzed fundamental information such as publication year, country, affiliation, journal, and author details. Through co-occurrence analysis, we identified country and author partnerships, while co-citation analysis revealed the influence of journals, authors, and documents. We employed keywords to explore the evolution of hydrological phenomena and study areas, using burst analysis to predict trends and frontiers. We discovered exponential growth in this field with a closer integration of hydrological phenomena and Earth observation techniques. The research focus has shifted from large glaciers to encompass large river basins and the Tibetan Plateau. Long-term research attention has been dedicated to optical properties, sea level, and satellite gravity. The adoption of automatic image recognition and processing, enabled by deep learning and artificial intelligence, has opened new interdisciplinary avenues. The results of the study emphasize the significance of long-term, stable, and accurate global observation and monitoring of inland water, particularly in the context of cloud computing and big data. [ABSTRACT FROM AUTHOR]

Recently, transformer has gradually attracted interest for its excellence in modeling the long-range dependencies of spatial-spectral features in HSI. However, transformer has the problem of the quadratic computational complexity due to the self-attention mechanism, which is heavier than other models and thus has limited adoption in HSI processing. Fortunately, the recently emerging state space model-based Mamba shows great computational efficiency while achieving the modeling power of transformers. Therefore, in this paper, we first proposed spectral-spatial Mamba (SS-Mamba) for HSI classification. Specifically, SS-Mamba mainly includes a spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks. Firstly, the token generation module converts any given HSI cube to spatial and spectral tokens as sequences. And then these tokens are sent to stacked spectral-spatial mamba blocks (SS-MB). Each SS-MB includes two basic mamba blocks and a spectral-spatial feature enhancement module. The spatial and spectral tokens are processed separately by the two basic mamba blocks, correspondingly. Moreover, the feature enhancement module modulates spatial and spectral tokens using HSI sample's center region information. Therefore, the spectral and spatial tokens cooperate with each other and achieve information fusion within each block. The experimental results conducted on widely used HSI datasets reveal that the proposed SS-Mamba requires less processing time compared with transformer. The Mamba-based method thus opens a new window for HSI classification. [ABSTRACT FROM AUTHOR]

Polarimetric features extracted from polarimetric synthetic aperture radar (PolSAR) images contain abundant back-scattering information about objects. Utilizing this information for PolSAR image classification can improve accuracy and enhance object monitoring. In this paper, a deep learning classification method based on polarimetric channel power features for PolSAR is proposed. The distinctive characteristic of this method is that the polarimetric features input into the deep learning network are the power values of polarimetric channels and contain complete polarimetric information. The other two input data schemes are designed to compare the proposed method. The neural network can utilize the extracted polarimetric features to classify images, and the classification accuracy analysis is employed to compare the strengths and weaknesses of the power-based scheme. It is worth mentioning that the polarized characteristics of the data input scheme mentioned in this article have been derived through rigorous mathematical deduction, and each polarimetric feature has a clear physical meaning. By testing different data input schemes on the Gaofen-3 (GF-3) PolSAR image, the experimental results show that the method proposed in this article outperforms existing methods and can improve the accuracy of classification to a certain extent, validating the effectiveness of this method in large-scale area classification. [ABSTRACT FROM AUTHOR]

Hyperspectral image classification (HSIC) has garnered increasing attention among researchers. While classical networks like convolution neural networks (CNNs) have achieved satisfactory results with the advent of deep learning, they are confined to processing local information. Vision transformers, despite being effective at establishing long-distance dependencies, face challenges in extracting high-representation features for high-dimensional images. In this paper, we present the multiscale efficient attention with enhanced feature transformer (MEA-EFFormer), which is designed for the efficient extraction of spectral–spatial features, leading to effective classification. MEA-EFFormer employs a multiscale efficient attention feature extraction module to initially extract 3D convolution features and applies effective channel attention to refine spectral information. Following this, 2D convolution features are extracted and integrated with local binary pattern (LBP) spatial information to augment their representation. Then, the processed features are fed into a spectral–spatial enhancement attention (SSEA) module that facilitates interactive enhancement of spectral–spatial information across the three dimensions. Finally, these features undergo classification through a transformer encoder. We evaluate MEA-EFFormer against several state-of-the-art methods on three datasets and demonstrate its outstanding HSIC performance. [ABSTRACT FROM AUTHOR]

With the development of deep learning techniques, convolutional neural networks (CNN) are increasingly being used in image recognition for marine surveys and underwater object classification. Automatic recognition of targets on side-scan sonar (SSS) images using CNN can improve recognition accuracy and efficiency. However, the vast selection of CNN models makes it challenging to select models for target recognition in SSS images. Therefore, this paper aims to compare different CNN models' prediction accuracy and computational performance comprehensively. First, four traditional CNN models were applied to train and predict the same submarine SSS dataset using both the original model and models with transfer learning methods. Then, we examined and studied the prediction accuracy and computation performance of four CNN models. Results showed that transfer learning enhances the accuracy of all CNN models, with lesser improvements for AlexNet and VGG-16 and greater improvements for GoogleNet and ResNet101. GoogleNet has the highest prediction of accuracy (100% in the train dataset and 94.27% in the test dataset) and good computational difficulty. The findings of this work are useful for future model selection in target recognition in SSS images. [ABSTRACT FROM AUTHOR]

This paper proposes a network structure called CAMP-Net, which considers the problem that traditional deep learning algorithms are unable to manage the pixel information of different bands, resulting in poor differentiation of feature representations of different categories and causing classification overfitting. CAMP-Net is a parallel network that, firstly, enhances the interaction of local information of bands by grouping the spectral nesting of the band information and then proposes a parallel processing model. One branch is responsible for inputting the features, normalized difference vegetation index (NDVI) and normalized difference water index (NDWI) band information generated by grouped nesting into the ViT framework, and enhancing the interaction and information flow between different channels in the feature map by adding the channel attention mechanism to realize the expressive capability of the feature map. The other branch assists the network's ability to enhance the extraction of different feature channels by designing a multi-layer perceptron network based on the utilization of the feature channels. Finally, the classification results are obtained by fusing the features obtained by the channel attention mechanism with those obtained by the MLP to achieve pixel-level multispectral image classification. In this study, the application of the algorithm was carried out in the feature distribution of South County, Yiyang City, Hunan Province, and the experiments were conducted based on 10 m Sentinel-2 multispectral RS images. The experimental results show that the overall accuracy of the algorithm proposed in this paper is 99.00% and the transformer (ViT) is 95.81%, while the performance of the algorithm in the Sentinel-2 dataset was greatly improved for the transformer. The transformer shows a huge improvement, which provides research value for developing a land cover classification algorithm for remote sensing images. [ABSTRACT FROM AUTHOR]

The rapid advancement of remote sensing technology has significantly enhanced the temporal resolution of remote sensing data. Multitemporal remote sensing image classification can extract richer spatiotemporal features. However, this also presents the challenge of mining massive data features. In response to this challenge, deep learning methods have become prevalent in machine learning and have been widely applied in remote sensing due to their ability to handle large datasets. The combination of remote sensing classification and deep learning has become a trend and has developed rapidly in recent years. However, there is a lack of summary and discussion on the research status and trends in multitemporal images. This review retrieved and screened 170 papers and proposed a research framework for this field. It includes retrieval statistics from existing research, preparation of multitemporal datasets, sample acquisition, an overview of typical models, and a discussion of application status. Finally, this paper discusses current problems and puts forward prospects for the future from three directions: adaptability between deep learning models and multitemporal classification, prospects for high-resolution image applications, and large-scale monitoring and model generalization. The aim is to help readers quickly understand the research process and application status of this field. [ABSTRACT FROM AUTHOR]

In deep learning-based hyperspectral remote sensing image classification tasks, random sampling strategies are typically used to train model parameters for testing and evaluation. However, this approach leads to strong spatial autocorrelation between the training set samples and the surrounding test set samples, and some unlabeled test set data directly participate in the training of the network. This leaked information makes the model overly optimistic. Models trained under these conditions tend to overfit to a single dataset, which limits the range of practical applications. This paper analyzes the causes and effects of information leakage and summarizes the methods from existing models to mitigate the effects of information leakage. Specifically, this paper states the main issues in this area, where the issue of information leakage is addressed in detail. Second, some algorithms and related models used to mitigate information leakage are categorized, including reducing the number of training samples, using spatially disjoint sampling strategies, few-shot learning, and unsupervised learning. These models and methods are classified according to the sample-related phase and the feature extraction phase. Finally, several representative hyperspectral image classification models experiments are conducted on the common datasets and their effectiveness in mitigating information leakage is analyzed. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing images (HSIs) have both spectral and spatial characteristics. The adept exploitation of these attributes is central to enhancing the classification accuracy of HSIs. In order to effectively utilize spatial and spectral features to classify HSIs, this paper proposes a method for the spatial feature extraction of HSIs based on a mutually guided image filter (muGIF) and combined with the band-distance-grouped principal component. Firstly, aiming at the problem that previously guided image filtering cannot effectively deal with the inconsistent information structure between the guided and target information, a method for extracting spatial features using muGIF is proposed. Then, aiming at the problem of the information loss caused by a single principal component as a guided image in the traditional GIF-based spatial–spectral classification, a spatial feature-extraction framework based on the band-distance-grouped principal component is proposed. The method groups the bands according to the band distance and extracts the principal components of each set of band subsets as the guide map of the current band subset to filter the HSIs. A deep convolutional neural network model and a generative adversarial network model for the filtered HSIs are constructed and then trained using samples for HSIs' spatial–spectral classification. Experiments show that compared with the traditional methods and several popular spatial–spectral HSI classification methods based on a filter, the proposed methods based on muGIF can effectively extract the spatial–spectral features and improve the classification accuracy of HSIs. [ABSTRACT FROM AUTHOR]

Deep learning, which is a dominating technique in artificial intelligence, has completely changed image understanding over the past decade. As a consequence, the sea ice extraction (SIE) problem has reached a new era. We present a comprehensive review of four important aspects of SIE, including algorithms, datasets, applications and future trends. Our review focuses on research published from 2016 to the present, with a specific focus on deep-learning-based approaches in the last five years. We divided all related algorithms into three categories, including the conventional image classification approach, the machine learning-based approach and deep-learning-based methods. We reviewed the accessible ice datasets including SAR-based datasets, the optical-based datasets and others. The applications are presented in four aspects including climate research, navigation, geographic information systems (GIS) production and others. This paper also provides insightful observations and inspiring future research directions. [ABSTRACT FROM AUTHOR]

Leveraging multi-view remote sensing images in scene classification tasks significantly enhances the precision of such classifications. This approach, however, poses challenges due to the simultaneous use of multi-view images, which often leads to a misalignment between the visual content and semantic labels, thus complicating the classification process. In addition, as the number of image viewpoints increases, the quality problem for remote sensing images further limits the effectiveness of multi-view image classification. Traditional scene classification methods predominantly employ SoftMax deep learning techniques, which lack the capability to assess the quality of remote sensing images or to provide explicit explanations for the network's predictive outcomes. To address these issues, this paper introduces a novel end-to-end multi-view decision fusion network specifically designed for remote sensing scene classification. The network integrates information from multi-view remote sensing images under the guidance of image credibility and uncertainty, and when the multi-view image fusion process encounters conflicts, it greatly alleviates the conflicts and provides more reasonable and credible predictions for the multi-view scene classification results. Initially, multi-scale features are extracted from the multi-view images using convolutional neural networks (CNNs). Following this, an asymptotic adaptive feature fusion module (AAFFM) is constructed to gradually integrate these multi-scale features. An adaptive spatial fusion method is then applied to assign different spatial weights to the multi-scale feature maps, thereby significantly enhancing the model's feature discrimination capability. Finally, an evidence decision fusion module (EDFM), utilizing evidence theory and the Dirichlet distribution, is developed. This module quantitatively assesses the uncertainty in the multi-perspective image classification process. Through the fusing of multi-perspective remote sensing image information in this module, a rational explanation for the prediction results is provided. The efficacy of the proposed method was validated through experiments conducted on the AiRound and CV-BrCT datasets. The results show that our method not only improves single-view scene classification results but also advances multi-view remote sensing scene classification results by accurately characterizing the scene and mitigating the conflicting nature of the fusion process. [ABSTRACT FROM AUTHOR]

Several deep learning and transformer models have been recommended in previous research to deal with the classification of hyperspectral images (HSIs). Among them, one of the most innovative is the bidirectional encoder representation from transformers (BERT), which applies a distance-independent approach to capture the global dependency among all pixels in a selected region. However, this model does not consider the local spatial-spectral and spectral sequential relations. In this paper, a dual-dimensional (i.e., spatial and spectral) BERT (the so-called D2BERT) is proposed, which improves the existing BERT model by capturing more global and local dependencies between sequential spectral bands regardless of distance. In the proposed model, two BERT branches work in parallel to investigate relations among pixels and spectral bands, respectively. In addition, the layer intermediate information is used for supervision during the training phase to enhance the performance. We used two widely employed datasets for our experimental analysis. The proposed D2BERT shows superior classification accuracy and computational efficiency with respect to some state-of-the-art neural networks and the previously developed BERT model for this task. [ABSTRACT FROM AUTHOR]

Over the past few decades, researchers have shown sustained and robust investment in exploring methods for hyperspectral image classification (HSIC). The utilization of hyperspectral imagery (HSI) for crop classification in agricultural areas has been widely demonstrated for its feasibility, flexibility, and cost-effectiveness. However, numerous coexisting issues in agricultural scenarios, such as limited annotated samples, uneven distribution of crops, and mixed cropping, could not be explored insightfully in the mainstream datasets. The limitations within these impractical datasets have severely restricted the widespread application of HSIC methods in agricultural scenarios. A benchmark dataset named Heilongjiang (HLJ) for HSIC is introduced in this paper, which is designed for large-scale crop classification. For practical applications, the HLJ dataset covers a wide range of genuine agricultural regions in Heilongjiang Province; it provides rich spectral diversity enriched through two images from diverse time periods and vast geographical areas with intercropped multiple crops. Simultaneously, considering the urgent demand of deep learning models, the two images in the HLJ dataset have 319,685 and 318,942 annotated samples, along with 151 and 149 spectral bands, respectively. To validate the suitability of the HLJ dataset as a baseline dataset for HSIC, we employed eight classical classification models in fundamental experiments on the HLJ dataset. Most of the methods achieved an overall accuracy of more than 80% with 10% of the labeled samples used for training. Furthermore, the advantages of the HLJ dataset and the impact of real-world factors on experimental results are comprehensively elucidated. The comprehensive baseline experimental evaluation and analysis affirm the research potential of the HLJ dataset as a large-scale crop classification dataset. [ABSTRACT FROM AUTHOR]

Crop classification of large-scale agricultural land is crucial for crop monitoring and yield estimation. Hyperspectral image classification has proven to be an effective method for this task. Most current popular hyperspectral image classification methods are based on image classification, specifically on convolutional neural networks (CNNs) and recurrent neural networks (RNNs). In contrast, this paper focuses on methods based on semantic segmentation and proposes a new transformer-based approach called HyperSFormer for crop hyperspectral image classification. The key enhancement of the proposed method is the replacement of the encoder in SegFormer with an improved Swin Transformer while keeping the SegFormer decoder. The entire model adopts a simple and uniform transformer architecture. Additionally, the paper introduces the hyper patch embedding (HPE) module to extract spectral and local spatial information from the hyperspectral images, which enhances the effectiveness of the features used as input for the model. To ensure detailed model processing and achieve end-to-end hyperspectral image classification, the transpose padding upsample (TPU) module is proposed for the model's output. In order to address the problem of insufficient and imbalanced samples in hyperspectral image classification, the paper designs an adaptive min log sampling (AMLS) strategy and a loss function that incorporates dice loss and focal loss to assist model training. Experimental results using three public hyperspectral image datasets demonstrate the strong performance of HyperSFormer, particularly in the presence of imbalanced sample data, complex negative samples, and mixed sample classes. HyperSFormer outperforms state-of-the-art methods, including fast patch-free global learning (FPGA), a spectral–spatial-dependent global learning framework (SSDGL), and SegFormer, by at least 2.7% in the mean intersection over union (mIoU). It also improves the overall accuracy and average accuracy values by at least 0.9% and 0.3%, respectively, and the kappa coefficient by at least 0.011. Furthermore, ablation experiments were conducted to determine the optimal hyperparameter and loss function settings for the proposed method, validating the rationality of these settings and the fusion loss function. [ABSTRACT FROM AUTHOR]

For practical maritime SAR image classification tasks with special imaging platforms, scenes to be classified are often different from those in the training sets. The quantity and diversity of the available training data can also be extremely limited. This problem of out-of-distribution (OOD) generalization with limited training samples leads to a sharp drop in the performance of conventional deep learning algorithms. In this paper, a knowledge-guided neural network (KGNN) model is proposed to overcome these challenges. By analyzing the saliency features of various maritime SAR scenes, universal knowledge in descriptive sentences is summarized. A feature integration strategy is designed to assign the descriptive knowledge to the ResNet-18 backbone. Both the individual semantic information and the inherent relations of the entities in SAR images are addressed. The experimental results show that our KGNN method outperforms conventional deep learning models in OOD scenarios with varying training sample sizes and achieves higher robustness in handling distributional shifts caused by weather conditions, terrain type, and sensor characteristics. In addition, the KGNN model converges within many fewer epochs during training. The performance improvement indicates that the KGNN model learns representations guided by beneficial properties for ODD generalization with limited training samples. [ABSTRACT FROM AUTHOR]

In recent years, remote sensing target recognition algorithms based on deep learning technology have gradually become mainstream in the field of remote sensing because of the great improvements that have been made in the accuracy of image target recognition through the use of deep learning. In the research of remote sensing image target recognition based on deep learning, an insufficient number of research samples is often an encountered issue; too small a number of research samples will cause the phenomenon of an overfitting of the model. To solve this problem, data augmentation techniques have also been developed along with the popularity of deep learning, and many methods have been proposed. However, to date, there is no literature aimed at expounding and summarizing the current state of the research applied to data augmentation for remote sensing object recognition, which is the purpose of this article. First, based on the essential principles of data augmentation methods, the existing methods are divided into two categories: data-based data augmentation methods and network-based data augmentation methods. Second, this paper subdivides and compares each method category to show the advantages, disadvantages, and characteristics of each method. Finally, this paper discusses the limitations of the existing methods and points out future research directions for data augmentation methods. [ABSTRACT FROM AUTHOR]

The recognition and monitoring of marine ranching help to determine the type, spatial distribution and dynamic change of marine life, as well as promote the rational utilization of marine resources and marine ecological environment protection, which has important research significance and application value. In the study of marine ranching recognition based on high-resolution remote sensing images, the convolutional neural network (CNN)-based deep learning method, which can adapt to different water environments, can be used to identify multi-scale targets and restore the boundaries of extraction results with high accuracy. However, research based on deep learning still has problems such as rarely considering the feature complexity of marine ranching and inadequate discussion of model generalization for multi-temporal images. In this paper, we construct a multi-temporal dataset that describes multiple features of marine ranching to explore the recognition and generalization ability of the semantic segmentation models DeepLab-v3+ and U-Net (used in large-area marine ranching) based on GF-1 remote sensing images with a 2 m spatial resolution. Through experiments, we find that the F-score of the U-Net extraction results from multi-temporal test images is basically stable at more than 90%, while the F-score of DeepLab-v3+ fluctuates around 80%. The results show that, compared with DeepLab-v3+, U-Net has a stronger recognition and generalization ability for marine ranching. U-Net can identify floating raft aquaculture areas at different growing stages and distinguish between cage aquaculture areas with different colors but similar morphological characteristics. [ABSTRACT FROM AUTHOR]

With the development of deep learning, image recognition based on deep learning is now widely used in remote sensing. As we know, the effectiveness of deep learning models significantly benefits from the size and quality of the dataset. However, remote sensing data are often distributed in different parts. They cannot be shared directly for privacy and security reasons, and this has motivated some scholars to apply federated learning (FL) to remote sensing. However, research has found that federated learning is usually vulnerable to white-box membership inference attacks (MIAs), which aim to infer whether a piece of data was participating in model training. In remote sensing, the MIA can lead to the disclosure of sensitive information about the model trainers, such as their location and type, as well as time information about the remote sensing equipment. To solve this issue, we consider embedding local differential privacy (LDP) into FL and propose LDP-Fed. LDP-Fed performs local differential privacy perturbation after properly pruning the uploaded parameters, preventing the central server from obtaining the original local models from the participants. To achieve a trade-off between privacy and model performance, LDP-Fed adds different noise levels to the parameters for various layers of the local models. This paper conducted comprehensive experiments to evaluate the framework's effectiveness on two remote sensing image datasets and two machine learning benchmark datasets. The results demonstrate that remote sensing image classification models are susceptible to MIAs, and our framework can successfully defend against white-box MIA while achieving an excellent global model. [ABSTRACT FROM AUTHOR]

Remote sensing image scene classification with deep learning (DL) is a rapidly growing field that has gained significant attention in the past few years. While previous review papers in this domain have been confined to 2020, an up-to-date review to show the progression of research extending into the present phase is lacking. In this review, we explore the recent articles, providing a thorough classification of approaches into three main categories: Convolutional Neural Network (CNN)-based, Vision Transformer (ViT)-based, and Generative Adversarial Network (GAN)-based architectures. Notably, within the CNN-based category, we further refine the classification based on specific methodologies and techniques employed. In addition, a novel and rigorous meta-analysis is performed to synthesize and analyze the findings from 50 peer-reviewed journal articles to provide valuable insights in this domain, surpassing the scope of existing review articles. Our meta-analysis shows that the most adopted remote sensing scene datasets are AID (41 articles) and NWPU-RESISC45 (40). A notable paradigm shift is seen towards the use of transformer-based models (6) starting from 2021. Furthermore, we critically discuss the findings from the review and meta-analysis, identifying challenges and future opportunities for improvement in this domain. Our up-to-date study serves as an invaluable resource for researchers seeking to contribute to this growing area of research. [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) image classification has been an important area of research due to its wide range of applications. Traditional machine learning methods were insufficient in achieving satisfactory results before the advent of deep learning. Results have significantly improved with the widespread use of deep learning in PolSAR image classification. However, the challenge of reconciling the complex-valued inputs of PolSAR images with the real-valued models of deep learning remains unsolved. Current complex-valued deep learning models treat complex numbers as two distinct real numbers, providing limited assistance in PolSAR image classification results. This paper proposes a novel, complex-valued deep learning approach for PolSAR image classification to address this issue. The approach includes amplitude-based max pooling, complex-valued nonlinear activation, and a cross-entropy loss function based on complex-valued probability. Amplitude-based max pooling reduces computational effort while preserving the most valuable complex-valued features. Complex-valued nonlinear activation maps feature into a high-dimensional complex-domain space, producing the most discriminative features. The complex-valued cross-entropy loss function computes the classification loss using the complex-valued model output and dataset labels, resulting in more accurate and robust classification results. The proposed method was applied to a shallow CNN, deep CNN, FCN, and SegNet, and its effectiveness was verified on three public datasets. The results showed that the method achieved optimal classification results on any model and dataset. [ABSTRACT FROM AUTHOR]

Deep learning (DL) algorithms have demonstrated important breakthroughs for hyperspectral image (HSI) classification. Despite the remarkable success of DL, the burden of a manually designed DL structure with increased depth and size aroused the difficulty for the application in the mobile and embedded devices in a real application. To tackle this issue, in this paper, we proposed an efficient lightweight attention network architecture search algorithm (EL-NAS) for realizing an efficient automatic design of a lightweight DL structure as well as improving the classification performance of HSI. First, aimed at realizing an efficient search procedure, we construct EL-NAS based on a differentiable network architecture search (NAS), which can greatly accelerate the convergence of the over-parameter supernet in a gradient descent manner. Second, in order to realize lightweight search results with high accuracy, a lightweight attention module search space is designed for EL-NAS. Finally, further for alleviating the problem of higher validation accuracy and worse classification performance, the edge decision strategy is exploited to perform edge decisions through the entropy of distribution estimated over non-skip operations to avoid further performance collapse caused by numerous skip operations. To verify the effectiveness of EL-NAS, we conducted experiments on several real-world hyperspectral images. The results demonstrate that the proposed EL-NAS indicates a more efficient search procedure with smaller parameter sizes and high accuracy performance for HSI classification, even under data-independent and sensor-independent scenarios. [ABSTRACT FROM AUTHOR]

Hyperspectral image classification plays a crucial role in various remote sensing applications. However, existing methods often struggle with the challenge of unknown classes, leading to decreased classification accuracy and limited generalization. In this paper, we propose a novel deep learning framework called IADMRN, which addresses the issue of unknown class handling in hyperspectral image classification. IADMRN combines the strengths of dense connection blocks and attention mechanisms to extract discriminative features from hyperspectral data. Furthermore, it employs a multi-scale deconvolution image reconstruction sub-network to enhance feature reconstruction and provide additional information for classification. To handle unknown classes, IADMRN utilizes an extreme value theory-based model to calculate the probability of unknown class membership. Experimental results on the three public datasets demonstrate that IADMRN outperforms state-of-the-art methods in terms of classification accuracy for both known and unknown classes. Experimental results show that the proposed methods outperform several state-of-the-art methods, which outperformed DCFSL by 8.47%, 6.57%, and 4.25%, and outperformed MDL4OW by 4.35%, 4.08%, and 2.47% on the Salinas, University of Pavia, and Indian Pines datasets, respectively. The proposed framework is computationally efficient and showcases the ability to effectively handle unknown classes in hyperspectral image classification tasks. Overall, IADMRN offers a promising solution for accurate and robust hyperspectral image classification, making it a valuable tool for remote sensing applications. [ABSTRACT FROM AUTHOR]

Hyperspectral sensors acquire spectral responses from objects with a large number of narrow spectral bands. The large volume of data may be costly in terms of storage and computational requirements. In addition, hyperspectral data are often information-wise redundant. Band selection intends to overcome these limitations by selecting a small subset of spectral bands that provide more information or better performance for particular tasks. However, existing band selection techniques do not directly maximize the task-specific performance, but rather utilize hand-crafted metrics as a proxy to the final goal of performance improvement. In this paper, we propose a deep learning (DL) architecture composed of a constrained measurement learning network for band selection, followed by a classification network. The proposed joint DL architecture is trained in a data-driven manner to optimize the classification loss along band selection. In this way, the proposed network directly learns to select bands that enhance the classification performance. Our evaluation results with Indian Pines (IP) and the University of Pavia (UP) datasets show that the proposed constrained measurement learning-based band selection approach provides higher classification accuracy compared to the state-of-the-art supervised band selection methods for the same number of bands selected. The proposed method shows 89.08 % and 97.78 % overall accuracy scores for IP and UP respectively, being 1.34 % and 2.19 % higher than the second-best method. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral–spatial features. FE-HybridSN effectively captures more spectral–spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc. [ABSTRACT FROM AUTHOR]

Integrating Artificial Intelligence (AI) techniques with remote sensing holds great potential for revolutionizing data analysis and applications in many domains of Earth sciences. This review paper synthesizes the existing literature on AI applications in remote sensing, consolidating and analyzing AI methodologies, outcomes, and limitations. The primary objectives are to identify research gaps, assess the effectiveness of AI approaches in practice, and highlight emerging trends and challenges. We explore diverse applications of AI in remote sensing, including image classification, land cover mapping, object detection, change detection, hyperspectral and radar data analysis, and data fusion. We present an overview of the remote sensing technologies, methods employed, and relevant use cases. We further explore challenges associated with practical AI in remote sensing, such as data quality and availability, model uncertainty and interpretability, and integration with domain expertise as well as potential solutions, advancements, and future directions. We provide a comprehensive overview for researchers, practitioners, and decision makers, informing future research and applications at the exciting intersection of AI and remote sensing. [ABSTRACT FROM AUTHOR]

Hyperspectral images are complex images that contain more spectral dimension information than ordinary images. An increasing number of HSI classification methods are using deep learning techniques to process three-dimensional data. The Vision Transformer model is gradually occupying an important position in the field of computer vision and is being used to replace the CNN structure of the network. However, it is still in the preliminary research stage in the field of HSI. In this paper, we propose using a spectral Swin Transformer network for HSI classification, providing a new approach for the HSI field. The Swin Transformer uses group attention to enhance feature representation, and the sliding window attention calculation can take into account the contextual information of different windows, which can retain the global features of HSI and improve classification results. In our experiments, we evaluated our proposed approach on several public hyperspectral datasets and compared it with several methods. The experimental results demonstrate that our proposed model achieved test accuracies of 97.46%, 99.7%, and 99.8% on the IP, SA, and PU public HSI datasets, respectively, when using the AdamW optimizer. Our approach also shows good generalization ability when applied to new datasets. Overall, our proposed approach represents a promising direction for hyperspectral image classification using deep learning techniques. [ABSTRACT FROM AUTHOR]

Currently, researchers commonly use convolutional neural network (CNN) models for landslide remote sensing image recognition. However, with the increase in landslide monitoring data, the available multimodal landslide data contain rich feature information, and existing landslide recognition models have difficulty utilizing such data. A knowledge graph is a linguistic network knowledge base capable of storing and describing various entities and their relationships. A landslide knowledge graph is used to manage multimodal landslide data, and by integrating this graph into a landslide image recognition model, the given multimodal landslide data can be fully utilized for landslide identification. In this paper, we combine knowledge and models, introduce the use of landslide knowledge graphs in landslide identification, and propose a landslide identification method for remote sensing images that fuses knowledge graphs and ResNet (FKGRNet). We take the Loess Plateau of China as the study area and test the effect of the fusion model by comparing the baseline model, the fusion model and other deep learning models. The experimental results show that, first, with ResNet34 as the baseline model, the FKGRNet model achieves 95.08% accuracy in landslide recognition, which is better than that of the baseline model and other deep learning models. Second, the FKGRNet model with different network depths has better landslide recognition accuracy than its corresponding baseline model. Third, the FKGRNet model based on feature splicing outperforms the fused feature classifier in terms of both accuracy and F1-score on the landslide recognition task. Therefore, the FKGRNet model can make fuller use of landslide knowledge to accurately recognize landslides in remote sensing images. [ABSTRACT FROM AUTHOR]

Recently, the rapid development of deep learning has greatly improved the performance of image classification. However, a central problem in hyperspectral image (HSI) classification is spectral uncertainty, where spectral features alone cannot accurately and robustly identify a pixel point in a hyperspectral image. This paper presents a novel HSI classification network called MS-RPNet, i.e., multiscale superpixelwise RPNet, which combines superpixel-based S3-PCA with two-dimensional singular spectrum analysis (2D-SSA) based on the Random Patches Network (RPNet). The proposed frame can not only take advantage of the data-driven method, but can also apply S3-PCA to efficiently consider more global and local spectral knowledge at the super-pixel level. Meanwhile, 2D-SSA is used for noise removal and spatial feature extraction. Then, the final features are obtained by random patch convolution and other steps according to the cascade structure of RPNet. The layered extraction superimposes the different sparial information into multi-scale spatial features, which complements the features of various land covers. Finally, the final fusion features are classified by SVM to obtain the final classification results. The experimental results in several HSI datasets demonstrate the effectiveness and efficiency of MS-RPNet, which outperforms several current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Developing complex hyperspectral image (HSI) sensors that capture high-resolution spatial information and voluminous (hundreds) spectral bands of the earth's surface has made HSI pixel-wise classification a reality. The 3D-CNN has become the preferred HSI pixel-wise classification approach because of its ability to extract discriminative spectral and spatial information while maintaining data integrity. However, HSI datasets are characterized by high nonlinearity, voluminous spectral features, and limited training sample data. Therefore, developing deep HSI classification methods that purely utilize 3D-CNNs in their network structure often results in computationally expensive models prone to overfitting when the model depth increases. In this regard, this paper proposes an integrated deep multi-scale 3D/2D convolutional network block (MiCB) for simultaneous low-level spectral and high-level spatial feature extraction, which can optimally train on limited sample data. The strength of the proposed MiCB model solely lies in the innovative arrangement of convolution layers, giving the network the ability (i) to simultaneously convolve the low-level spectral with high-level spatial features; (ii) to use multiscale kernels to extract abundant contextual information; (iii) to apply residual connections to solve the degradation problem when the model depth increases beyond the threshold; and (iv) to utilize depthwise separable convolutions in its network structure to address the computational cost of the proposed MiCB model. We evaluate the efficacy of our proposed MiCB model using three publicly accessible HSI benchmarking datasets: Salinas Scene (SA), Indian Pines (IP), and the University of Pavia (UP). When trained on small amounts of training sample data, MiCB is better at classifying than the state-of-the-art methods used for comparison. For instance, the MiCB achieves a high overall classification accuracy of 97.35%, 98.29%, and 99.20% when trained on 5% IP, 1% UP, and 1% SA data, respectively. [ABSTRACT FROM AUTHOR]

Graph convolutional networks (GCNs) are neural network frameworks for machine learning on graphs. They can simultaneously perform end-to-end learning on the attribute information and the structure information of graph data. However, most existing GCNs inevitably encounter the limitations of non-robustness and low classification accuracy when labeled nodes are scarce. To address the two issues, the deep graph convolutional generative adversarial network (DGCGAN), a model combining GCN and deep convolutional generative adversarial networks (DCGAN), is proposed in this paper. First, the graph data is mapped to a highly nonlinear space by using the topology and attribute information of the graph for symmetric normalized Laplacian transform. Then, through the feature-structured enhanced module, the node features are expanded into regular structured data, such as images and sequences, which are input to DGCGAN as positive samples, thus expanding the sample capacity. In addition, the feature-enhanced (FE) module is adopted to enhance the typicality and discriminability of node features, and to obtain richer and more representative features, which is helpful for facilitating accurate classification. Finally, additional constraints are added to the network model by introducing DCGAN, thus enhancing the robustness of the model. Through extensive empirical studies on several standard benchmarks, we find that DGCGAN outperforms state-of-the-art baselines on semi-supervised node classification and remote sensing image classification. [ABSTRACT FROM AUTHOR]

In recent years, hyperspectral image classification techniques have attracted a lot of attention from many scholars because they can be used to model the development of different cities and provide a reference for urban planning and construction. However, due to the difficulty in obtaining hyperspectral images, only a limited number of pixels can be used as training samples. Therefore, how to adequately extract and utilize the spatial and spectral information of hyperspectral images with limited training samples has become a difficult problem. To address this issue, we propose a hyperspectral image classification method based on dense pyramidal convolution and multi-feature fusion (DPCMF). In this approach, two branches are designed to extract spatial and spectral features, respectively. In the spatial branch, dense pyramid convolutions and non-local blocks are used to extract multi-scale local and global spatial features in image samples, which are then fused to obtain spatial features. In the spectral branch, dense pyramidal convolution layers are used to extract spectral features in image samples. Finally, the spatial and spectral features are fused and fed into fully connected layers to obtain classification results. The experimental results show that the overall accuracy (OA) of the method proposed in this paper is 96.74%, 98.10%, 98.92% and 96.67% on the four hyperspectral datasets, respectively. Significant improvements are achieved compared to the five methods of SVM, SSRN, FDSSC, DBMA and DBDA for hyperspectral classification. Therefore, the proposed method can better extract and exploit the spatial and spectral information in image samples when the number of training samples is limited. Provide more realistic and intuitive terrain and environmental conditions for urban planning, design, construction and management. [ABSTRACT FROM AUTHOR]

Hyperspectral image classification (HSI) has rich applications in several fields. In the past few years, convolutional neural network (CNN)-based models have demonstrated great performance in HSI classification. However, CNNs are inadequate in capturing long-range dependencies, while it is possible to think of the spectral dimension of HSI as long sequence information. More and more researchers are focusing their attention on transformer which is good at processing sequential data. In this paper, a spectral shifted window self-attention based transformer (SSWT) backbone network is proposed. It is able to improve the extraction of local features compared to the classical transformer. In addition, spatial feature extraction module (SFE) and spatial position encoding (SPE) are designed to enhance the spatial feature extraction of the transformer. The spatial feature extraction module is proposed to address the deficiency of transformer in the capture of spatial features. The loss of spatial structure of HSI data after inputting transformer is supplemented by proposed spatial position encoding. On three public datasets, we ran extensive experiments and contrasted the proposed model with a number of powerful deep learning models. The outcomes demonstrate that our suggested approach is efficient and that the proposed model performs better than other advanced models. [ABSTRACT FROM AUTHOR]

Deep learning-based label noise learning methods provide promising solutions for hyperspectral image (HSI) classification with noisy labels. Currently, label noise learning methods based on deep learning improve their performance by modifying one aspect, such as designing a robust loss function, revamping the network structure, or adding a noise adaptation layer. However, these methods face difficulties in coping with relatively high noise situations. To address this issue, this paper proposes a unified label noise learning framework with a dual-network structure. The goal is to enhance the model's robustness to label noise by utilizing two networks to guide each other. Specifically, to avoid the degeneration of the dual-network training into self-training, the "disagreement" strategy is incorporated with co-learning. Then, the "agreement" strategy is introduced into the model to ensure that the model iterates in the right direction under high noise conditions. To this end, an agreement and disagreement-based co-learning (ADCL) framework is proposed for HSI classification with noisy labels. In addition, a joint loss function consisting of a supervision loss of two networks and a relative loss between two networks is designed for the dual-network structure. Extensive experiments are conducted on three public HSI datasets to demonstrate the robustness of the proposed method to label noise. Specifically, our method obtains the highest overall accuracy of 98.62%, 90.89%, and 99.02% on the three datasets, respectively, which represents an improvement of 2.58%, 2.27%, and 0.86% compared to the second-best method. In future research, the authors suggest using more networks as backbones to implement the ADCL framework. [ABSTRACT FROM AUTHOR]

Economic fruit forest is an important part of Chinese agriculture with high economic value and ecological benefits. Using UAV multi-spectral images to research the classification of economic fruit forests based on deep learning is of great significance for accurately understanding the distribution and scale of fruit forests and the status quo of national economic fruit forest resources. Based on the multi-spectral remote sensing images of UAV, this paper constructed semantic segmentation data of economic fruit forests, conducted a comparative study on the classification and identification of economic fruit forests of FCN, SegNet, and U-Net classic semantic segmentation models, and proposed an improved ISDU-Net model. The recognition accuracies of the ISDU-Net were 87.73%, 70.68%, 78.69%, and 0.84, respectively, in terms of pixel accuracy, average intersection ratio, frequency weight intersection ratio, and Kappa coefficient, which were 3.19%, 8.90%, and 4.51% higher than the original U-Net model. The results showed that the improved ISDU-Net could effectively improve the learning ability of the model, perform better in the prediction of short sample categories, obtain a higher classification accuracy of fruit forest crops, and provide a new idea for the research on accurate fruit forest identification. [ABSTRACT FROM AUTHOR]

The deep learning method has achieved great success in hyperspectral image classification, but the lack of labeled training samples still restricts the development and application of deep learning methods. In order to deal with the problem of small samples in hyperspectral image classification, a novel small sample classification method based on rotation-invariant uniform local binary pattern (RULBP) features and a graph-based masked autoencoder is proposed in this paper. Firstly, the RULBP features of hyperspectral images are extracted, and then the k-nearest neighbor method is utilized to construct the graph. Furthermore, self-supervised learning is conducted on the constructed graph so that the model can learn to extract features more suitable for small sample classification. Since the self-supervised training mainly adopts the masked autoencoder method, only unlabeled samples are needed to complete the training. After training, only a small number of samples are used to fine-tune the graph convolutional network, so as to complete the classification of all nodes in the graph. A large number of classification experiments on three commonly used hyperspectral image datasets show that the proposed method could achieve higher classification accuracy with fewer labeled samples. [ABSTRACT FROM AUTHOR]

With the development of remote sensing technology, classification has become a meaningful way to explore the rich information in hyperspectral images (HSIs). However, various environmental factors may cause noise and shadow areas in HSIs, resulting in weak signals and difficulties in fully utilizing information. In addition, classification methods based on deep learning have made considerable progress, but features extracted from most networks have much redundancy. Therefore, a method based on two-dimensional dynamic stochastic resonance (2D DSR) shadow enhancement and convolutional neural network (CNN) classification combined with an attention mechanism (AM) for HSIs is proposed in this paper. Firstly, to protect the spatial correlation of HSIs, an iterative equation of 2D DSR based on the pixel neighborhood relationship was derived, which made it possible to perform matrix SR in the spatial dimension of the image, instead of one-dimensional vector resonance. Secondly, by using the noise in the shadow area to generate resonance, 2D DSR can help increase the signals in the shadow regions by preserving the spatial characteristics, and enhanced HSIs can be obtained. Then, a 3DCNN embedded with two efficient channel attention (ECA) modules and one convolutional block attention module (CBAM) was designed to make the most of critical features that significantly affect the classification accuracy by giving different weights. Finally, the performance of the proposed method was evaluated on a real-world HSI, and comparative studies were carried out. The experimental results showed that the proposed approach has promising prospects in HSIs' shadow enhancement and information mining. [ABSTRACT FROM AUTHOR]

Highly accurate supervised deep learning-based classifiers for polarimetric synthetic aperture radar (PolSAR) images require large amounts of data with manual annotations. Unfortunately, the complex echo imaging mechanism results in a high labeling cost for PolSAR images. Extracting and transferring knowledge to utilize the existing labeled data to the fullest extent is a viable approach in such circumstances. To this end, we are introducing unsupervised deep adversarial domain adaptation (ADA) into PolSAR image classification for the first time. In contrast to the standard learning paradigm, in this study, the deep learning model is trained on labeled data from a source domain and unlabeled data from a related but distinct target domain. The purpose of this is to extract domain-invariant features and generalize them to the target domain. Although the feature transferability of ADA methods can be ensured through adversarial training to align the feature distributions of source and target domains, improving feature discriminability remains a crucial issue. In this paper, we propose a novel polarimetric scattering characteristics-guided adversarial network (PSCAN) for unsupervised PolSAR image classification. Compared with classical ADA methods, we designed an auxiliary task for PSCAN based on the polarimetric scattering characteristics-guided pseudo-label construction. This approach utilizes the rich information contained in the PolSAR data itself, without the need for expensive manual annotations or complex automatic labeling mechanisms. During the training of PSCAN, the auxiliary task receives category semantic information from pseudo-labels and helps promote the discriminability of the learned domain-invariant features, thereby enabling the model to have a better target prediction function. The effectiveness of the proposed method was demonstrated using data captured with different PolSAR systems in the San Francisco and Qingdao areas. Experimental results show that the proposed method can obtain satisfactory unsupervised classification results. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) was applied in sonar image target recognition tasks, but it is very difficult to obtain enough sonar images that contain a target; as a result, the direct use of a small amount of data to train a DNN will cause overfitting and other problems. Transfer learning is the most effective way to address such scenarios. However, there is a large domain gap between optical images and sonar images, and common transfer learning methods may not be able to effectively handle it. In this paper, we propose a transfer learning method for sonar image classification and object detection called the texture feature removal network. We regard the texture features of an image as domain-specific features, and we narrow the domain gap by discarding the domain-specific features, and hence, make it easier to complete knowledge transfer. Our method can be easily embedded into other transfer learning methods, which makes it easier to apply to different application scenarios. Experimental results show that our method is effective in side-scan sonar image classification tasks and forward-looking sonar image detection tasks. For side-scan sonar image classification tasks, the classification accuracy of our method is enhanced by 4.5% in a supervised learning experiment, and for forward-looking sonar detection tasks, the average precision (AP) is also significantly improved. [ABSTRACT FROM AUTHOR]

Clouds have an enormous influence on the hydrological cycle, Earth's radiation budget, and climate changes. Accurate automatic recognition of cloud shape based on ground-based cloud images is beneficial to analyze the atmospheric motion state and water vapor content, and then to predict weather trends and identify severe weather processes. Cloud type classification remains challenging due to the variable and diverse appearance of clouds. Deep learning-based methods have improved the feature extraction ability and the accuracy of cloud type classification, but face the problem of lack of labeled samples. In this paper, we proposed a novel classification approach of ground-based cloud images based on contrastive self-supervised learning (CSSL) to reduce the dependence on the number of labeled samples. First, data augmentation is applied to the input data to obtain augmented samples. Then contrastive self-supervised learning is used to pre-train the deep model with a contrastive loss and a momentum update-based optimization. After pre-training, a supervised fine-tuning procedure is adopted on labeled data to classify ground-based cloud images. Experimental results have confirmed the effectiveness of the proposed method. This study can provide inspiration and technical reference for the analysis and processing of other types of meteorological remote sensing data under the scenario of insufficient labeled samples. [ABSTRACT FROM AUTHOR]

Most of the existing Non-Cooperative Target Recognition (NCTR) systems follow the "closed world" assumption, i.e., they only work with what was previously observed. Nevertheless, the real world is relatively "open" in the sense that the knowledge of the environment is incomplete. Therefore, unknown targets can feed the recognition system at any time while it is operational. Addressing this issue, the Openmax classifier has been recently proposed in the optical domain to make convolutional neural networks (CNN) able to reject unknown targets. There are some fundamental limitations in the Openmax classifier that can end up with two potential errors: (1) rejecting a known target and (2) classifying an unknown target. In this paper, we propose a new classifier to increase the robustness and accuracy. The proposed classifier, which is inspired by the limitations of the Openmax classifier, is based on proportional similarity between the test image and different training classes. We evaluate our method by radar images of man-made targets from the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset. Moreover, a more in-depth discussion on the Openmax hyper-parameters and a detailed description of the Openmax functioning are given. [ABSTRACT FROM AUTHOR]

Deep Learning has been successfully applied to image recognition, speech recognition, and natural language processing in recent years. Therefore, there has been an incentive to apply it in other fields as well. The field of agriculture is one of the most important fields in which the application of deep learning still needs to be explored, as it has a direct impact on human well-being. In particular, there is a need to explore how deep learning models can be used as a tool for optimal planting, land use, yield improvement, production/disease/pest control, and other activities. The vast amount of data received from sensors in smart farms makes it possible to use deep learning as a model for decision-making in this field. In agriculture, no two environments are exactly alike, which makes testing, validating, and successfully implementing such technologies much more complex than in most other industries. This paper reviews some recent scientific developments in the field of deep learning that have been applied to agriculture, and highlights some challenges and potential solutions using deep learning algorithms in agriculture. The results in this paper indicate that by employing new methods from deep learning, higher performance in terms of accuracy and lower inference time can be achieved, and the models can be made useful in real-world applications. Finally, some opportunities for future research in this area are suggested. [ABSTRACT FROM AUTHOR]

With the development of hyperspectral sensors, the availability of hyperspectral images (HSIs) has increased significantly, prompting advancements in deep learning-based hyperspectral image classification (HSIC) methods. Recently, graph convolutional networks (GCNs) have been proposed to process graph-structured data in non-Euclidean domains, and have been used for HSIC. The superpixel segmentation should be implemented first in the GCN-based methods, however, it is difficult to manually select the optimal superpixel segmentation sizes to obtain the useful information for classification. To solve this problem, we constructed a HSIC model based on a multiscale feature search-based graph convolutional network (MFSGCN) in this study. Firstly, pixel-level features of HSIs are extracted sequentially using 3D asymmetric decomposition convolution and 2D convolution. Then, superpixel-level features at different scales are extracted using multilayer GCNs. Finally, the neural architecture search (NAS) method is used to automatically assign different weights to different scales of superpixel features. Thus, a more discriminative feature map is obtained for classification. Compared with other GCN-based networks, the MFSGCN network can automatically capture features and obtain higher classification accuracy. The proposed MFSGCN model was implemented on three commonly used HSI datasets and compared to some state-of-the-art methods. The results confirm that MFSGCN effectively improves accuracy. [ABSTRACT FROM AUTHOR]

The progress in hyperspectral image (HSI) classification owes much to the integration of various deep learning techniques. However, the inherent 3D cube structure of HSIs presents a unique challenge, necessitating an innovative approach for the efficient utilization of spectral data in classification tasks. This research focuses on HSI classification through the adoption of a recently validated deep-learning methodology. Challenges in HSI classification encompass issues related to dimensionality, data redundancy, and computational expenses, with CNN-based methods prevailing due to architectural limitations. In response to these challenges, we introduce a groundbreaking model known as "Crossover Dimensionality Reduction and Multi-branch Deep Learning" (CMD) for hyperspectral image classification. The CMD model employs a multi-branch deep learning architecture incorporating Factor Analysis and MNF for crossover feature extraction, with the selection of optimal features from each technique. Experimental findings underscore the CMD model's superiority over existing methods, emphasizing its potential to enhance HSI classification outcomes. Notably, the CMD model exhibits exceptional performance on benchmark datasets such as Salinas Scene (SC), Pavia University (PU), Kennedy Space Center (KSC), and Indian Pines (IP), achieving impressive overall accuracy rates of 99.35% and 99.18% using only 5% of the training data. [ABSTRACT FROM AUTHOR]

Although extensive research shows that CNNs achieve good classification results in HSI classification, they still struggle to effectively extract spectral sequence information from HSIs. Additionally, the high-dimensional features of HSIs, the limited number of labeled samples, and the common sample imbalance significantly restrict classification performance improvement. To address these issues, this article proposes a double-branch multi-scale dual-attention (DBMSDA) network that fully extracts spectral and spatial information from HSIs and fuses them for classification. The designed multi-scale spectral residual self-attention (MSeRA), as a fundamental component of dense connections, can fully extract high-dimensional and intricate spectral information from HSIs, even with limited labeled samples and imbalanced distributions. Additionally, this article adopts a dataset partitioning strategy to prevent information leakage. Finally, this article introduces a hyperspectral geological lithology dataset to evaluate the accuracy and applicability of deep learning methods in geology. Experimental results on the geological lithology hyperspectral dataset and three other public datasets demonstrate that the DBMSDA method exhibits superior classification performance and robust generalization ability compared to existing methods. [ABSTRACT FROM AUTHOR]

In hyperspectral image classification (HSIC), every pixel of the HSI is assigned to a land cover category. While convolutional neural network (CNN)-based methods for HSIC have significantly enhanced performance, they encounter challenges in learning the relevance of deep semantic features and grappling with escalating computational costs as network depth increases. In contrast, the transformer framework is adept at capturing the relevance of high-level semantic features, presenting an effective solution to address the limitations encountered by CNN-based approaches. This article introduces a novel adaptive learnable spectral–spatial fusion transformer (ALSST) to enhance HSI classification. The model incorporates a dual-branch adaptive spectral–spatial fusion gating mechanism (ASSF), which captures spectral–spatial fusion features effectively from images. The ASSF comprises two key components: the point depthwise attention module (PDWA) for spectral feature extraction and the asymmetric depthwise attention module (ADWA) for spatial feature extraction. The model efficiently obtains spectral–spatial fusion features by multiplying the outputs of these two branches. Furthermore, we integrate the layer scale and DropKey into the traditional transformer encoder and multi-head self-attention (MHSA) to form a new transformer with a layer scale and DropKey (LD-Former). This innovation enhances data dynamics and mitigates performance degradation in deeper encoder layers. The experiments detailed in this article are executed on four renowned datasets: Trento (TR), MUUFL (MU), Augsburg (AU), and the University of Pavia (UP). The findings demonstrate that the ALSST model secures optimal performance, surpassing some existing models. The overall accuracy (OA) is 99.70%, 89.72%, 97.84%, and 99.78% on four famous datasets: Trento (TR), MUUFL (MU), Augsburg (AU), and University of Pavia (UP), respectively. [ABSTRACT FROM AUTHOR]

Accurate roof information of buildings can be obtained from UAV high-resolution images. The large-scale accurate recognition of roof types (such as gabled, flat, hipped, complex and mono-pitched roofs) of rural buildings is crucial for rural planning and construction. At present, most UAV high-resolution optical images only have red, green and blue (RGB) band information, which aggravates the problems of inter-class similarity and intra-class variability of image features. Furthermore, the different roof types of rural buildings are complex, spatially scattered, and easily covered by vegetation, which in turn leads to the low accuracy of roof type identification by existing methods. In response to the above problems, this paper proposes a method for identifying roof types of complex rural buildings based on visible high-resolution remote sensing images from UAVs. First, the fusion of deep learning networks with different visual features is investigated to analyze the effect of the different feature combinations of the visible difference vegetation index (VDVI) and Sobel edge detection features and UAV visible images on model recognition of rural building roof types. Secondly, an improved Mask R-CNN model is proposed to learn more complex features of different types of images of building roofs by using the ResNet152 feature extraction network with migration learning. After we obtained roof type recognition results in two test areas, we evaluated the accuracy of the results using the confusion matrix and obtained the following conclusions: (1) the model with RGB images incorporating Sobel edge detection features has the highest accuracy and enables the model to recognize more and more accurately the roof types of different morphological rural buildings, and the model recognition accuracy (Kappa coefficient (KC)) compared to that of RGB images is on average improved by 0.115; (2) compared with the original Mask R-CNN, U-Net, DeeplabV3 and PSPNet deep learning models, the improved Mask R-CNN model has the highest accuracy in recognizing the roof types of rural buildings, with F1-score, KC and OA averaging 0.777, 0.821 and 0.905, respectively. The method can obtain clear and accurate profiles and types of rural building roofs, and can be extended for green roof suitability evaluation, rooftop solar potential assessment, and other building roof surveys, management and planning. [ABSTRACT FROM AUTHOR]