Search

In the era of the first Industrial Revolution, many buildings were built with red bricks, and the heritage buildings built at that time are more than 100 years old. In these old heritage buildings, damage is bound to occur due to chemical and physical effects. Technologies such as automatic damage detection can effectively manage damage, but they can be affected by other categories present in heritage buildings. Therefore, this paper proposes a CNN algorithm that can automatically detect cracks and damage that occur in heritage buildings, as well as multi-label classification, such as doors, windows, arches, artwork, brick walls, stonewalls, and vents. A total of 2400 thermal infrared images are collected for 8 categories and automatic classification was performed using the CNN algorithm. The average precision and average sensitivity for the eight categories of heritage buildings are 97.72% and 97.43%, respectively. This paper defines the causes of misclassification as the following two causes: misclassification by multiple objects and misclassification by the perception of the CNN algorithm. [ABSTRACT FROM AUTHOR]

Earth observation missions such as Sentinel and Landsat support the large-scale identification of agricultural crops by providing free radar and multispectral satellite images. The extraction of representative image information as well as the combination of different image sources for improved feature selection still represent a major challenge in the field of remote sensing. In this paper, we propose a novel three-dimensional (3D) deep learning U-Net model to fuse multi-level image features from multispectral and synthetic aperture radar (SAR) time series data for seasonal crop-type mapping at a regional scale. For this purpose, we used a dual-stream U-Net with a 3D squeeze-and-excitation fusion module applied at multiple stages in the network to progressively extract and combine multispectral and SAR image features. Additionally, we introduced a distinctive method for generating patch-based multitemporal multispectral composites by selective image sampling within a 14-day window, prioritizing those with minimal cloud cover. The classification results showed that the proposed network provided the best overall accuracy (94.5%) compared to conventional two-dimensional (2D) and three-dimensional U-Net models (2D: 92.6% and 3D: 94.2%). Our network successfully learned multi-modal dependencies between the multispectral and SAR satellite images, leading to improved field mapping of spectrally similar and heterogeneous classes while mitigating the limitations imposed by persistent cloud coverage. Additionally, the feature representations extracted by the proposed network demonstrated their transferability to a new cropping season, providing a reliable mapping of spatio-temporal crop type patterns. [ABSTRACT FROM AUTHOR]

In smart transportation, assisted driving relies on data integration from various sensors, notably LiDAR and cameras. However, their optical performance can degrade under adverse weather conditions, potentially compromising vehicle safety. Millimeter-wave radar, which can overcome these issues more economically, has been re-evaluated. Despite this, developing an accurate detection model is challenging due to significant noise interference and limited semantic information. To address these practical challenges, this paper presents the TC–Radar model, a novel approach that synergistically integrates the strengths of transformer and the convolutional neural network (CNN) to optimize the sensing potential of millimeter-wave radar in smart transportation systems. The rationale for this integration lies in the complementary nature of CNNs, which are adept at capturing local spatial features, and transformers, which excel at modeling long-range dependencies and global context within data. This hybrid approach allows for a more robust and accurate representation of radar signals, leading to enhanced detection performance. A key innovation of our approach is the introduction of the Cross-Attention (CA) module, which facilitates efficient and dynamic information exchange between the encoder and decoder stages of the network. This CA mechanism ensures that critical features are accurately captured and transferred, thereby significantly improving the overall network performance. In addition, the model contains the dense information fusion block (DIFB) to further enrich the feature representation by integrating different high-frequency local features. This integration process ensures thorough incorporation of key data points. Extensive tests conducted on the CRUW and CARRADA datasets validate the strengths of this method, with the model achieving an average precision (AP) of 83.99% and a mean intersection over union (mIoU) of 45.2%, demonstrating robust radar sensing capabilities. [ABSTRACT FROM AUTHOR]

Hazy weather reduces contrast, narrows the dynamic range, and blurs the details of the remote sensing image. Additionally, color fidelity deteriorates, causing color shifts and image distortion, thereby impairing the utility of remote sensing data. In this paper, we propose a lightweight remote sensing-image-dehazing network, named LRSDN. The network comprises two tailored, lightweight modules arranged in cascade. The first module, the axial depthwise convolution and residual learning block (ADRB), is for feature extraction, efficiently expanding the convolutional receptive field with little computational overhead. The second is a feature-calibration module based on the hybrid attention block (HAB), which integrates a simplified, yet effective channel attention module and a pixel attention module embedded with an observational prior. This joint attention mechanism effectively enhances the representation of haze features. Furthermore, we introduce a novel method for remote sensing hazy image synthesis using Perlin noise, facilitating the creation of a large-scale, fine-grained remote sensing haze image dataset (RSHD). Finally, we conduct both quantitative and qualitative comparison experiments on multiple publicly available datasets. The results demonstrate that the LRSDN algorithm achieves superior dehazing performance with fewer than 0.1M parameters. We also validate the positive effects of the LRSDN in road extraction and land cover classification applications. [ABSTRACT FROM AUTHOR]

Sea ice is a significant factor in influencing environmental change on Earth. Monitoring sea ice is of major importance, and one of the main objectives of this monitoring is sea ice classification. Currently, synthetic aperture radar (SAR) data are primarily used for sea ice classification, with a single polarization band or simple combinations of polarization bands being common choices. While much of the current research has focused on optimizing network structures to achieve high classification accuracy, which requires substantial training resources, we aim to extract more information from the SAR data for classification. Therefore we propose a multi-featured SAR sea ice classification method that combines polarization features calculated by polarization decomposition and spectrogram features calculated by joint time-frequency analysis (JTFA). We built a convolutional neural network (CNN) structure for learning the multi-features of sea ice, which combines spatial features and physical properties, including polarization and spectrogram features of sea ice. In this paper, we utilized ALOS PALSAR SLC data with HH, HV, VH, and VV, four types of polarization for the multi-featured sea ice classification method. We divided the sea ice into new ice (NI), first-year ice (FI), old ice (OI), deformed ice (DI), and open water (OW). Then, the accuracy calculation by confusion matrix and comparative analysis were carried out. Our experimental results demonstrate that the multi-feature method proposed in this paper can achieve high accuracy with a smaller data volume and computational effort. In the four scenes selected for validation, the overall accuracy could reach 95%, 91%, 96%, and 95%, respectively, which represents a significant improvement compared to the single-feature sea ice classification method. [ABSTRACT FROM AUTHOR]

This paper focuses on the problems of omission, misclassification, and inter-adhesion due to overly dense distribution, intraclass diversity, and interclass variability when extracting winter wheat (WW) from high-resolution images. This paper proposes a deep supervised network RAunet model with multi-scale features that incorporates a dual-attention mechanism with an improved U-Net backbone network. The model mainly consists of a pyramid input layer, a modified U-Net backbone network, and a side output layer. Firstly, the pyramid input layer is used to fuse the feature information of winter wheat at different scales by constructing multiple input paths. Secondly, the Atrous Spatial Pyramid Pooling (ASPP) residual module and the Convolutional Block Attention Module (CBAM) dual-attention mechanism are added to the U-Net model to form the backbone network of the model, which enhances the feature extraction ability of the model for winter wheat information. Finally, the side output layer consists of multiple classifiers to supervise the results of different scale outputs. Using the RAunet model to extract the spatial distribution information of WW from GF-2 imagery, the experimental results showed that the mIou of the recognition results reached 92.48%, an improvement of 2.66%, 4.15%, 1.42%, 2.35%, 3.76%, and 0.47% compared to FCN, U-Net, DeepLabv3, SegNet, ResUNet, and UNet++, respectively. The superiority of the RAunet model in high-resolution images for WW extraction was verified in effectively improving the accuracy of the spatial distribution information extraction of WW. [ABSTRACT FROM AUTHOR]

Accurate urban land cover information is crucial for effective urban planning and management. While convolutional neural networks (CNNs) demonstrate superior feature learning and prediction capabilities using image-level annotations, the inherent mixed-category nature of input image patches leads to classification errors along object boundaries. Fully convolutional neural networks (FCNs) excel at pixel-wise fine segmentation, making them less susceptible to heterogeneous content, but they require fully annotated dense image patches, which may not be readily available in real-world scenarios. This paper proposes an object-based semi-supervised spatial attention residual UNet (OS-ARU) model. First, multiscale segmentation is performed to obtain segments from a remote sensing image, and segments containing sample points are assigned the categories of the corresponding points, which are used to train the model. Then, the trained model predicts class probabilities for all segments. Each unlabeled segment's probability distribution is compared against those of labeled segments for similarity matching under a threshold constraint. Through label propagation, pseudo-labels are assigned to unlabeled segments exhibiting high similarity to labeled ones. Finally, the model is retrained using the augmented training set incorporating the pseudo-labeled segments. Comprehensive experiments on aerial image benchmarks for Vaihingen and Potsdam demonstrate that the proposed OS-ARU achieves higher classification accuracy than state-of-the-art models, including OCNN, 2OCNN, and standard OS-U, reaching an overall accuracy (OA) of 87.83% and 86.71%, respectively. The performance improvements over the baseline methods are statistically significant according to the Wilcoxon Signed-Rank Test. Despite using significantly fewer sparse annotations, this semi-supervised approach still achieves comparable accuracy to the same model under full supervision. The proposed method thus makes a step forward in substantially alleviating the heavy sampling burden of FCNs (densely sampled deep learning models) to effectively handle the complex issue of land cover information identification and classification. [ABSTRACT FROM AUTHOR]

Change detection is widely used in the field of building monitoring. In recent years, the progress of remote sensing image technology has provided high-resolution data. However, unlike other tasks, change detection focuses on the difference between dual-input images, so the interaction between bi-temporal features is crucial. However, the existing methods have not fully tapped the potential of multi-scale bi-temporal features to interact layer by layer. Therefore, this paper proposes a multi-scale feature interaction network (MFINet). The network realizes the information interaction of multi-temporal images by inserting a bi-temporal feature interaction layer (BFIL) between backbone networks at the same level, guides the attention to focus on the difference region, and suppresses the interference. At the same time, a double temporal feature fusion layer (BFFL) is used at the end of the coding layer to extract subtle difference features. By introducing the transformer decoding layer and improving the recovery effect of the feature size, the ability of the network to accurately capture the details and contour information of the building is further improved. The F1 of our model on the public dataset LEVIR-CD reaches 90.12%, which shows better accuracy and generalization performance than many state-of-the-art change detection models. [ABSTRACT FROM AUTHOR]

Drone-based photogrammetry typically requires the task of georeferencing aerial images by detecting the center of Ground Control Points (GCPs) placed in the field. Since this is a very labor-intensive task, it could benefit greatly from automation. In this study, we explore the extent to which traditional computer vision approaches can be generalized to deal with variability in real-world drone data sets and focus on training different residual neural networks (ResNet) to improve generalization. The models were trained to detect single keypoints of fixed-sized image tiles with a historic collection of drone-based Red–Green–Blue (RGB) images with black and white GCP markers in which the center was manually labeled by experienced photogrammetry operators. Different depths of ResNets and various hyperparameters (learning rate, batch size) were tested. The best results reached sub-pixel accuracy with a mean absolute error of 0.586. The paper demonstrates that this approach to drone-based mapping is a promising and effective way to reduce the human workload required for georeferencing aerial images. [ABSTRACT FROM AUTHOR]

Deep learning, especially convolutional neural network (CNN) techniques, has been shown to have superior performance in ship classification, as have small-target recognition studies in safety inspections of hydraulic structures such as ports and dams. High-resolution synthetic aperture radar (SAR)-based maritime ship classification plays an increasingly important role in marine surveillance, marine rescue, and maritime ship management. To improve ship classification accuracy and training efficiency, we proposed a CNN-based ship classification method. Firstly, the image characteristics of different ship structures and the materials of ship SAR images were analyzed. We then constructed a ship SAR image dataset and performed preprocessing operations such as averaging. Combined with a classic neural network structure, we created a new convolutional module, namely, the Inception-Residual Controller (IRC) module. A convolutional neural network was built based on the IRC module to extract image features and establish a ship classification model. Finally, we conducted simulation experiments for ship classification and analyzed the experimental results for comparison. The experimental results showed that the average accuracy of ship classification of the model in this paper reached 98.71%, which was approximately 3% more accurate than the traditional network model and approximately 1% more accurate compared with other recently improved models. The new module also performed well in evaluation metrics, such as the recall rate, with accurate classifications. The model could satisfactorily describe different ship types. Therefore, it could be applied to marine ship classification management with the possibility of being extended to hydraulic building target recognition tasks. [ABSTRACT FROM AUTHOR]

Machine learning, particularly deep learning (DL), has become a central and state-of-the-art method for several computer vision applications and remote sensing (RS) image processing. Researchers are continually trying to improve the performance of the DL methods by developing new architectural designs of the networks and/or developing new techniques, such as attention mechanisms. Since the attention mechanism has been proposed, regardless of its type, it has been increasingly used for diverse RS applications to improve the performances of the existing DL methods. However, these methods are scattered over different studies impeding the selection and application of the feasible approaches. This study provides an overview of the developed attention mechanisms and how to integrate them with different deep learning neural network architectures. In addition, it aims to investigate the effect of the attention mechanism on deep learning-based RS image processing. We identified and analyzed the advances in the corresponding attention mechanism-based deep learning (At-DL) methods. A systematic literature review was performed to identify the trends in publications, publishers, improved DL methods, data types used, attention types used, overall accuracies achieved using At-DL methods, and extracted the current research directions, weaknesses, and open problems to provide insights and recommendations for future studies. For this, five main research questions were formulated to extract the required data and information from the literature. Furthermore, we categorized the papers regarding the addressed RS image processing tasks (e.g., image classification, object detection, and change detection) and discussed the results within each group. In total, 270 papers were retrieved, of which 176 papers were selected according to the defined exclusion criteria for further analysis and detailed review. The results reveal that most of the papers reported an increase in overall accuracy when using the attention mechanism within the DL methods for image classification, image segmentation, change detection, and object detection using remote sensing images. [ABSTRACT FROM AUTHOR]

The development of convolution neural networks (CNNs) has become a significant means to solve the problem of remote sensing scene image classification. However, well-performing CNNs generally have high complexity and are prone to overfitting. To handle the above problem, we present a new classification approach using an mmsCNN–HMM combined model with stacking ensemble mechanism in this paper. First of all, a modified multi-scale convolution neural network (mmsCNN) is proposed to extract multi-scale structural features, which has a lightweight structure and can avoid high computational complexity. Then, we utilize a hidden Markov model (HMM) to mine the context information of the extracted features of the whole sample image. For different categories of scene images, the corresponding HMM is trained and all the trained HMMs form an HMM group. In addition, our approach is based on a stacking ensemble learning scheme, in which the preliminary predicted values generated by the HMM group are used in an extreme gradient boosting (XGBoost) model to generate the final prediction. This stacking ensemble learning mechanism integrates multiple models to make decisions together, which can effectively prevent overfitting while ensuring accuracy. Finally, the trained XGBoost model conducts the scene category prediction. In this paper, the six most widely used remote sensing scene datasets, UCM, RSSCN, SIRI-WHU, WHU-RS, AID, and NWPU, are selected to carry out all kinds of experiments. The numerical experiments verify that the proposed approach shows more important advantages than the advanced approaches. [ABSTRACT FROM AUTHOR]

Semantic segmentation of remote sensing images has been widely used in environmental protection, geological disaster discovery, and natural resource assessment. With the rapid development of deep learning, convolutional neural networks (CNNs) have dominated semantic segmentation, relying on their powerful local information extraction capabilities. Due to the locality of convolution operation, it can be challenging to obtain global context information directly. However, Transformer has excellent potential in global information modeling. This paper proposes a new hybrid convolutional and Transformer semantic segmentation model called CTFuse, which uses a multi-scale convolutional attention module in the convolutional part. CTFuse is a serial structure composed of a CNN and a Transformer. It first uses convolution to extract small-size target information and then uses Transformer to embed large-size ground target information. Subsequently, we propose a spatial and channel attention module in convolution to enhance the representation ability for global information and local features. In addition, we also propose a spatial and channel attention module in Transformer to improve the ability to capture detailed information. Finally, compared to other models used in the experiments, our CTFuse achieves state-of-the-art results on the International Society of Photogrammetry and Remote Sensing (ISPRS) Vaihingen and ISPRS Potsdam datasets. [ABSTRACT FROM AUTHOR]

Anomaly perception of infrared point targets has high application value in many fields, such as maritime surveillance, airspace surveillance, and early warning systems. This kind of abnormality includes the explosion of the target, the separation between stages, the disintegration caused by the abnormal strike, etc. By extracting the radiation characteristics of continuous frame targets, it is possible to analyze and warn the target state in time. Most anomaly detection methods adopt traditional outlier detection, which has the problems of poor accuracy and a high false alarm rate. Driven by data, this paper proposes a new network structure, called AC-LSTM, which combines Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM), and embeds the Periodic Time Series Data Attention module (PTSA). The network can better extract the spatial and temporal characteristics of one-dimensional time series data, and the PTSA module can consider the periodic characteristics of the target in the process of continuous movement, and focus on abnormal data. In addition, this paper also proposes a new time series data enhancement method, which slices and re-amplifies the long time series data. This method significantly improves the accuracy of anomaly detection. Through a large number of experiments, AC-LSTM has achieved higher scores on our collected datasets than other methods. [ABSTRACT FROM AUTHOR]

Floods occur throughout the world and are becoming increasingly frequent and dangerous. This is due to different factors, among which climate change and land use stand out. In Mexico, they occur every year in different areas. Tabasco is a periodically flooded region, causing losses and negative consequences for the rural, urban, livestock, agricultural, and service industries. Consequently, it is necessary to create strategies to intervene effectively in the affected areas. Different strategies and techniques have been developed to mitigate the damage caused by this phenomenon. Satellite programs provide a large amount of data on the Earth's surface and geospatial information processing tools useful for environmental and forest monitoring, climate change impacts, risk analysis, and natural disasters. This paper presents a strategy for the classification of flooded areas using satellite images obtained from synthetic aperture radar, as well as the U-Net neural network and ArcGIS platform. The study area is located in Los Rios, a region of Tabasco, Mexico. The results show that U-Net performs well despite the limited number of training samples. As the training data and epochs increase, its precision increases. [ABSTRACT FROM AUTHOR]

Understanding atmospheric motions and projecting climate changes depends significantly on cloud types, i.e., different cloud types correspond to different atmospheric conditions, and accurate cloud classification can help forecasts and meteorology-related studies to be more effectively directed. However, accurate classification of clouds is challenging and often requires certain manual involvement due to the complex cloud forms and dispersion. To address this challenge, this paper proposes an improved cloud classification method based on a densely connected hybrid convolutional network. A dense connection mechanism is applied to hybrid three-dimensional convolutional neural network (3D-CNN) and two-dimensional convolutional neural network (2D-CNN) architectures to use the feature information of the spatial and spectral channels of the FY-4A satellite fully. By using the proposed network, cloud categorization solutions with a high temporal resolution, extensive coverage, and high accuracy can be obtained without the need for any human intervention. The proposed network is verified using tests, and the results show that it can perform real-time classification tasks for seven different types of clouds and clear skies in the Chinese region. For the CloudSat 2B-CLDCLASS product as a test target, the proposed network can achieve an overall accuracy of 95.2% and a recall of more of than 82.9% for all types of samples, outperforming the other deep-learning-based techniques. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have been utilized extensively to improve the resolution of weather radar. Most existing CNN-based super-resolution algorithms using PPI (Plan position indicator, which provides a maplike presentation in polar coordinates of range and angle) images plotted by radar data lead to the loss of some valid information by using image processing methods for super-resolution reconstruction. To solve this problem, a weather radar that echoes the super-resolution reconstruction algorithm—based on residual attention back-projection network (RABPN)—is proposed to improve the the radar base data resolution. RABPN consists of multiple Residual Attention Groups (RAGs) connected with long skip connections to form a deep network; each RAG is composed of some residual attention blocks (RABs) connected with short skip connections. The residual attention block mined the mutual relationship between low-resolution radar echoes and high-resolution radar echoes by adding a channel attention mechanism to the deep back-projection network (DBPN). Experimental results demonstrate that RABPN outperforms the algorithms compared in this paper in visual evaluation aspects and quantitative analysis, allowing a more refined radar echo structure, especially in terms of echo details and edge structure features. [ABSTRACT FROM AUTHOR]

Food and economic security in West Africa rely heavily on rainfed agriculture and are threatened by climate change and demographic growth. Accurate rainfall information is therefore crucial to tackling these challenges. Particularly, information about the occurrence and length of droughts as well as the onset date of the rainy season is essential for agricultural planning. However, existing rainfall models fail to accurately represent the highly variable and sparsely monitored West African rainfall patterns. In this paper, we show the potential of deep learning (DL) to model rainfall in the region and propose a methodology to develop DL models in data-scarce areas. We built two DL models for satellite rainfall (rain/no-rain) detection over northern Ghana from Meteosat TIR data based on standard DL architectures: Convolutional neural networks (CNNs) and convolutional long short-term memory neural networks (ConvLSTM). The Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement (GPM) mission (IMERG) and Precipitation Estimation from Remotely Sensed Imagery Using an Artificial Neural Network Cloud Classification System (PERSIANN-CCS) products are used as benchmarks. We use rain gauge data from the Trans-African Hydro-Meteorological Observatory (TAHMO) for model development and performance evaluation. We show that our models compare well against existing products despite being considerably simpler, developed with a small training dataset—i.e., 8 stations covering 2.5 years with 20.4% of the data missing—and using TIR data alone. Concretely, our models consistently outperform PERSIANN-CCS for rain/no-rain detection at a sub-daily timescale. While IMERG is the overall best performer, the DL models perform better in the second half of the rainy season despite their simplicity (i.e., up to 120 k parameters). Our results suggest that DL-based regional models are a promising alternative to state-of-the-art global products for providing regional rainfall information, especially in meteorologically complex regions such as the (sub)tropics, which are poorly covered by ground-based rainfall observations. [ABSTRACT FROM AUTHOR]

With the development of remote sensing technology, classification has become a meaningful way to explore the rich information in hyperspectral images (HSIs). However, various environmental factors may cause noise and shadow areas in HSIs, resulting in weak signals and difficulties in fully utilizing information. In addition, classification methods based on deep learning have made considerable progress, but features extracted from most networks have much redundancy. Therefore, a method based on two-dimensional dynamic stochastic resonance (2D DSR) shadow enhancement and convolutional neural network (CNN) classification combined with an attention mechanism (AM) for HSIs is proposed in this paper. Firstly, to protect the spatial correlation of HSIs, an iterative equation of 2D DSR based on the pixel neighborhood relationship was derived, which made it possible to perform matrix SR in the spatial dimension of the image, instead of one-dimensional vector resonance. Secondly, by using the noise in the shadow area to generate resonance, 2D DSR can help increase the signals in the shadow regions by preserving the spatial characteristics, and enhanced HSIs can be obtained. Then, a 3DCNN embedded with two efficient channel attention (ECA) modules and one convolutional block attention module (CBAM) was designed to make the most of critical features that significantly affect the classification accuracy by giving different weights. Finally, the performance of the proposed method was evaluated on a real-world HSI, and comparative studies were carried out. The experimental results showed that the proposed approach has promising prospects in HSIs' shadow enhancement and information mining. [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) has unique advantages in building extraction due to its sensitivity to building structures and all-time/all-weather imaging capabilities. However, the structure of buildings is complex, and buildings are easily confused with other objects in polarimetric SAR images. The speckle noise of SAR images will affect the accuracy of building extraction. This paper proposes a novel building extraction approach from PolSAR images with statistical texture and polarization features by using a convolutional neural network and superpixel. A feature space that is sensitive to building, including G0 statistical texture and PualiRGB features, is constructed and used as CNN input. Considering that the building boundary of the CNN classification result is inaccurate due to speckle noise, the simple linear iterative cluster (SLIC) superpixel is utilized to constrain the building extraction result. Finally, the effectiveness of the proposed method has been verified by experimenting with PolSAR images from three different sensors, including ESAR, GF-3, and RADARSAT-2. Experiment results show that compared with the other five PolSAR building extraction methods including threshold, SVM, RVCNN, and PFDCNN, our method without superpixel constraint, the F1-score of this method is the highest, reaching 84.22%, 91.24%, and 87.49%, respectively. The false alarm rate of this method is at least 10% lower and the F1 index is at least 6% higher when the building extraction accuracy is comparable. Further, the discussion and method parameter analysis results show that increasing the use of G0 statistical texture parameters can improve building extraction accuracy and reduce false alarms, and the introduction of superpixel constraints can further reduce false alarms. [ABSTRACT FROM AUTHOR]

The full waveform inversion at this stage still has many problems in the recovery of deep background velocities. Velocity modeling based on end-to-end deep learning usually lacks a generalization capability. The proposed method is a multi-scale convolutional neural network velocity inversion (Ms-CNNVI) that incorporates a multi-scale strategy into the CNN-based velocity inversion algorithm for the first time. This approach improves the accuracy of the inversion by integrating a multi-scale strategy from low-frequency to high-frequency inversion and by incorporating a smoothing strategy in the multi-scale (MS) convolutional neural network (CNN) inversion process. Furthermore, using angle-domain reverse time migration (RTM) for dataset construction in Ms-CNNVI significantly improves the inversion efficiency. Numerical tests showcase the efficacy of the suggested approach. [ABSTRACT FROM AUTHOR]

Simple Summary: Excessive use of groundwater resources in the Central Valley in California has led to major land sinking over the years. In this study, we rely on satellite imagery to monitor and assess the extent of this sinking. Specifically, we use images from a satellite that emits microwaves which allows us to directly obtain the deformation at a specific time. Then, we apply a machine learning algorithm to the resulting data in an attempt to effectively predict short-term future deformation. We find that the algorithm applied has a low error when compared to the actual data. This shows that machine learning algorithms could be incorporated into models that assess potential hazards associated with land sinking. Land subsidence caused by excessive groundwater pumping in Central Valley, California, is a major issue that has several negative impacts such as reduced aquifer storage and damaged infrastructures which, in turn, produce an economic loss due to the high reliance on crop production. This is why it is of utmost importance to routinely monitor and assess the surface deformation occurring. Two main goals that this paper attempts to accomplish are deformation characterization and deformation prediction. The first goal is realized through the use of Principal Component Analysis (PCA) applied to a series of Interferomtric Synthetic Aperture Radar (InSAR) images that produces eigenimages displaying the key characteristics of the subsidence. Water storage changes are also directly analyzed by the use of data from the Gravity Recovery and Climate Experiment (GRACE) twin satellites and the Global Land Data Assimilation System (GLDAS). The second goal is accomplished by building a Long Short-Term Memory (LSTM) model to predict short-term deformation after developing an InSAR time series using LiCSBAS, an open-source InSAR time series package. The model is applied to the city of Madera and produces better results than a baseline averaging model and a one dimensional convolutional neural network (CNN) based on a mean squared error metric showing the effectiveness of machine learning in deformation prediction as well as the potential for incorporation in hazard mitigation models. The model results can directly aid policy makers in determining the appropriate rate of groundwater withdrawal while maintaining the safety and well-being of the population as well as the aquifers' integrity. [ABSTRACT FROM AUTHOR]

Remote image semantic segmentation technology is one of the core research elements in the field of computer vision and has a wide range of applications in production life. Most remote image semantic segmentation methods are based on CNN. Recently, Transformer provided a view of long-distance dependencies in images. In this paper, we propose RCCT-ASPPNet, which includes the dual-encoder structure of Residual Multiscale Channel Cross-Fusion with Transformer (RCCT) and Atrous Spatial Pyramid Pooling (ASPP). RCCT uses Transformer to cross fuse global multiscale semantic information; the residual structure is then used to connect the inputs and outputs. ASPP based on CNN extracts contextual information of high-level semantics from different perspectives and uses Convolutional Block Attention Module (CBAM) to extract spatial and channel information, which will further improve the model segmentation ability. The experimental results show that the mIoU of our method is 94.14% and 61.30% on the datasets Farmland and AeroScapes, respectively, and that the mPA is 97.12% and 84.36%, respectively, both outperforming DeepLabV3+ and UCTransNet. [ABSTRACT FROM AUTHOR]

In recent years, non-contact human–computer interactions have aroused much attention. In this paper, we mainly propose a dual view observation system based on the frontal and side millimeter-wave radars (MWR) to collect echo data of the Air writing digits "0~9", simultaneously. Additionally, we also propose a novel distance approximation method to make the trajectory reconstruction more efficient. To exploit these characteristics of spatial-temporal adjacency in handwriting digits, we propose a novel clustering algorithm, named the constrained density-based spatial clustering of application with noise (CDBSCAN), to remove background noise or clutter. Moreover, we also design a robust gesture segmentation method by using twice-difference and high–low thresholds. In our trials and comparisons, based on the trajectories formulated by echo data series of time–distance and time–velocity of dual views, we present a lightweight-based convolution neural network (CNN) to realize these digits recognition. Experiment results show that our system has a relatively high recognition accuracy, which would provide a feasible application for future human–computer interaction scenarios. [ABSTRACT FROM AUTHOR]

The mainstream algorithms used for ship classification and detection can be improved based on convolutional neural networks (CNNs). By analyzing the characteristics of ship images, we found that the difficulty in ship image classification lies in distinguishing ships with similar hull structures but different equipment and superstructures. To extract features such as ship superstructures, this paper introduces transformer architecture with self-attention into ship classification and detection, and a CNN and Swin transformer model (CNN-Swin model) is proposed for ship image classification and detection. The main contributions of this study are as follows: (1) The proposed approach pays attention to different scale features in ship image classification and detection, introduces a transformer architecture with self-attention into ship classification and detection for the first time, and uses a parallel network of a CNN and a transformer to extract features of images. (2) To exploit the CNN's performance and avoid overfitting as much as possible, a multi-branch CNN-Block is designed and used to construct a CNN backbone with simplicity and accessibility to extract features. (3) The performance of the CNN-Swin model is validated on the open FGSC-23 dataset and a dataset containing typical military ship categories based on open-source images. The results show that the model achieved accuracies of 90.9% and 91.9% for the FGSC-23 dataset and the military ship dataset, respectively, outperforming the existing nine state-of-the-art approaches. (4) The good extraction effect on the ship features of the CNN-Swin model is validated as the backbone of the three state-of-the-art detection methods on the open datasets HRSC2016 and FAIR1M. The results show the great potential of the CNN-Swin backbone with self-attention in ship detection. [ABSTRACT FROM AUTHOR]

Recently, unmanned aerial vehicles (UAVs) have been broadly applied to the remote sensing field. For a great number of UAV images, deep learning has been reinvigorated and performed many results in agricultural applications. The popular image datasets for deep learning model training are generated for general purpose use, in which the objects, views, and applications are for ordinary scenarios. However, UAV images possess different patterns of images mostly from a look-down perspective. This paper provides a verified annotated dataset of UAV images that are described in data acquisition, data preprocessing, and a showcase of a CNN classification. The dataset collection consists of one multi-rotor UAV platform by flying a planned scouting routine over rice paddies. This paper introduces a semi-auto annotation method with an ExGR index to generate the training data of rice seedlings. For demonstration, this study modified a classical CNN architecture, VGG-16, to run a patch-based rice seedling detection. The k-fold cross-validation was employed to obtain an 80/20 dividing ratio of training/test data. The accuracy of the network increases with the increase of epoch, and all the divisions of the cross-validation dataset achieve a 0.99 accuracy. The rice seedling dataset provides the training-validation dataset, patch-based detection samples, and the ortho-mosaic image of the field. [ABSTRACT FROM AUTHOR]

The segmentation algorithm for buildings and waters is extremely important for the efficient planning and utilization of land resources. The temporal and space range of remote sensing pictures is growing. Due to the generic convolutional neural network's (CNN) insensitivity to the spatial position information in remote sensing images, certain location and edge details can be lost, leading to a low level of segmentation accuracy. This research suggests a double-branch parallel interactive network to address these issues, fully using the interactivity of global information in a Swin Transformer network, and integrating CNN to capture deeper information. Then, by building a cross-scale multi-level fusion module, the model can combine features gathered using convolutional neural networks with features derived using Swin Transformer, successfully extracting the semantic information of spatial information and context. Then, an up-sampling module for multi-scale fusion is suggested. It employs the output high-level feature information to direct the low-level feature information and recover the high-resolution pixel-level features. According to experimental results, the proposed networks maximizes the benefits of the two models and increases the precision of semantic segmentation of buildings and waters. [ABSTRACT FROM AUTHOR]

An accurate and historical land cover monitoring dataset for Alaska could provide fundamental information for a range of studies, such as conservation habitats, biogeochemical cycles, and climate systems, in this distinctive region. This research addresses challenges associated with the extraction of training data for timely and accurate land cover classifications in Alaska over longer time periods (e.g., greater than 10 years). Specifically, we designed the "Region-Specific Model Adaptation (RSMA)" method for training data. The method integrates land cover information from the National Land Cover Database (NLCD), LANDFIRE's Existing Vegetation Type (EVT), and the National Wetlands Inventory (NWI) and machine learning techniques to generate robust training samples based on the Anderson Level II classification legend. The assumption of the method is that spectral signatures vary across regions because of diverse land surface compositions; however, despite these variations, there are consistent, collective land cover characteristics that span the entire region. Building upon this assumption, this research utilized the classification power of deep learning algorithms and the generalization ability of RSMA to construct a model for the RSMA method. Additionally, we interpreted existing vegetation plot information for land cover labels as validation data to reduce inconsistency in the human interpretation. Our validation results indicate that the RSMA method improved the quality of the training data derived solely from the NLCD by approximately 30% for the overall accuracy. The validation assessment also demonstrates that the RSMA method can generate reliable training data on large scales in regions that lack sufficient reliable data. [ABSTRACT FROM AUTHOR]

To address the high cost associated with acquiring hyperspectral data, spectral reconstruction (SR) has emerged as a prominent research area. However, contemporary SR techniques are more focused on image processing tasks in computer vision than on practical applications. Furthermore, the prevalent approach of employing single-dimensional features to guide reconstruction, aimed at reducing computational overhead, invariably compromises reconstruction accuracy, particularly in complex environments with intricate ground features and severe spectral mixing. Effectively utilizing both local and global information in spatial and spectral dimensions for spectral reconstruction remains a significant challenge. To tackle these challenges, this study proposes an integrated network of 3D CNN and U-shaped Transformer for heterogeneous spectral reconstruction, ICTH, which comprises a shallow feature extraction module (CSSM) and a deep feature extraction module (TDEM), implementing a coarse-to-fine spectral reconstruction scheme. To minimize information loss, we designed a novel spatial–spectral attention module (S2AM) as the foundation for constructing a U-transformer, enhancing the capture of long-range information across all dimensions. On three hyperspectral datasets, ICTH has exhibited remarkable strengths across quantitative, qualitative, and single-band detail assessments, while also revealing significant potential for subsequent applications, such as generalizability and vegetation index calculations) in two real-world datasets. [ABSTRACT FROM AUTHOR]

This paper focuses on the problems of omission, misclassification, and inter-adhesion due to overly dense distribution, intraclass diversity, and interclass variability when extracting winter wheat (WW) from high-resolution images. This paper proposes a deep supervised network RAunet model with multi-scale features that incorporates a dual-attention mechanism with an improved U-Net backbone network. The model mainly consists of a pyramid input layer, a modified U-Net backbone network, and a side output layer. Firstly, the pyramid input layer is used to fuse the feature information of winter wheat at different scales by constructing multiple input paths. Secondly, the Atrous Spatial Pyramid Pooling (ASPP) residual module and the Convolutional Block Attention Module (CBAM) dual-attention mechanism are added to the U-Net model to form the backbone network of the model, which enhances the feature extraction ability of the model for winter wheat information. Finally, the side output layer consists of multiple classifiers to supervise the results of different scale outputs. Using the RAunet model to extract the spatial distribution information of WW from GF-2 imagery, the experimental results showed that the mIou of the recognition results reached 92.48%, an improvement of 2.66%, 4.15%, 1.42%, 2.35%, 3.76%, and 0.47% compared to FCN, U-Net, DeepLabv3, SegNet, ResUNet, and UNet++, respectively. The superiority of the RAunet model in high-resolution images for WW extraction was verified in effectively improving the accuracy of the spatial distribution information extraction of WW.

In the era of the first Industrial Revolution, many buildings were built with red bricks, and the heritage buildings built at that time are more than 100 years old. In these old heritage buildings, damage is bound to occur due to chemical and physical effects. Technologies such as automatic damage detection can effectively manage damage, but they can be affected by other categories present in heritage buildings. Therefore, this paper proposes a CNN algorithm that can automatically detect cracks and damage that occur in heritage buildings, as well as multi-label classification, such as doors, windows, arches, artwork, brick walls, stonewalls, and vents. A total of 2400 thermal infrared images are collected for 8 categories and automatic classification was performed using the CNN algorithm. The average precision and average sensitivity for the eight categories of heritage buildings are 97.72% and 97.43%, respectively. This paper defines the causes of misclassification as the following two causes: misclassification by multiple objects and misclassification by the perception of the CNN algorithm.

The accurate recognition of maize growth stages is crucial for effective farmland management strategies. In order to overcome the difficulty of quickly obtaining precise information about maize growth stage in complex farmland scenarios, this study proposes a Maize Hybrid Vision Transformer (MaizeHT) that combines a convolutional algorithmic structure with self-attention for maize growth stage recognition. The MaizeHT model utilizes a ResNet34 convolutional neural network to extract image features to self-attention, which are then transformed into sequence vectors (tokens) using Patch Embedding. It simultaneously inserts category information and location information as a token. A Transformer architecture with multi-head self-attention is employed to extract token features and predict maize growth stage categories using a linear layer. In addition, the MaizeHT model is standardized and encapsulated, and a prototype platform for intelligent maize growth stage recognition is developed for deployment on a website. Finally, the performance validation test of MaizeHT was carried out. To be specific, MaizeHT has an accuracy of 97.71% when the input image resolution is 224 × 224 and 98.71% when the input image resolution is 512 × 512 on the self-built dataset, the number of parameters is 15.446 M, and the floating-point operations are 4.148 G. The proposed maize growth stage recognition method could provide computational support for maize farm intelligence. [ABSTRACT FROM AUTHOR]

Extracting water bodies from synthetic aperture radar (SAR) images plays a crucial role in the management of water resources, flood monitoring, and other applications. Recently, transformer-based models have been extensively utilized in the remote sensing domain. However, due to regular patch-partition and weak inductive bias, transformer-based models face challenges such as edge serration and high data dependency when used for water body extraction from SAR images. To address these challenges, we introduce a new model, the Superpixel-based Transformer (SPT), based on the adaptive characteristic of superpixels and knowledge constraints of the adjacency matrix. (1) To mitigate edge serration, the SPT replaces regular patch partition with superpixel segmentation to fully utilize the internal homogeneity of superpixels. (2) To reduce data dependency, the SPT incorporates a normalized adjacency matrix between superpixels into the Multi-Layer Perceptron (MLP) to impose knowledge constraints. (3) Additionally, to integrate superpixel-level learning from the SPT with pixel-level learning from the CNN, we combine these two deep networks to form SPT-UNet for water body extraction. The results show that our SPT-UNet is competitive compared with other state-of-the-art extraction models, both in terms of quantitative metrics and visual effects. [ABSTRACT FROM AUTHOR]

In classical machine learning algorithms, used in many analysis tasks, the data are centralized for training. That is, both the model and the data are housed within one device. Federated learning (FL), on the other hand, is a machine learning technique that breaks away from this traditional paradigm by allowing multiple devices to collaboratively train a model without each sharing their own data. In a typical FL setting, each device has a local dataset and trains a local model on that dataset. The local models are next aggregated at a central server to produce a global model. The global model is then distributed back to the devices, which update their local models accordingly. This process is repeated until the global model converges. In this article, a FL approach is applied for remote sensing scene classification for the first time. The adopted approach uses three different RS datasets while employing two types of CNN models and two types of Vision Transformer models, namely: EfficientNet-B1, EfficientNet-B3, ViT-Tiny, and ViT-Base. We compare the performance of FL in each model in terms of overall accuracy and undertake additional experiments to assess their robustness when faced with scenarios of dropped clients. Our classification results on test data show that the two considered Transformer models outperform the two models from the CNN family. Furthermore, employing FL with ViT-Base yields the highest accuracy levels even when the number of dropped clients is significant, indicating its high robustness. These promising results point to the notion that FL can be successfully used with ViT models in the classification of RS scenes, whereas CNN models may suffer from overfitting problems. [ABSTRACT FROM AUTHOR]

Antimony (Sb) has gained significance as a critical raw material (CRM) within the European Union (EU) due to its strategic importance in various industrial sectors, particularly in the textile industry for flame retardants and as a component of Sb-based semiconductor materials. Moreover, Sb is emerging as a potential alternative for anodes used in lithium-ion batteries, a key element in the energy transition. This study explored the feasibility of identifying and quantifying Sb mineralisations through the spectral signature of soils using laboratory reflectance spectroscopy, a non-invasive remote sensing technique, and by employing convolutional neural networks (CNNs). Standard signal pre-processing techniques were applied to the spectral data, and the soils were analysed by inductively coupled plasma mass spectrometry (ICP-MS). Despite achieving high R-squared (0.7) values and an RMSE of 173 ppm for Sb, the study faces a significant challenge of generalisation of the model to new data. Despite the limitations, this study provides valuable insights into potential strategies for future research in this field. [ABSTRACT FROM AUTHOR]

Forest fires pose a significant threat worldwide, with Algeria being no exception. In 2020 alone, Algeria witnessed devastating forest fires, affecting over 16,000 hectares of land, a phenomenon largely attributed to the impacts of climate change. Understanding the severity of these fires is crucial for effective management and mitigation efforts. This study focuses on the Akfadou forest and its surrounding areas in Algeria, aiming to develop a robust method for mapping fire severity. We employed a comprehensive approach that integrates satellite imagery analysis, machine learning techniques, and geographic information systems (GIS) to assess fire severity. By evaluating various remote sensing attributes from the Sentinel-2 and Planetscope satellites, we compared different methodologies for fire severity classification. Specifically, we examined the effectiveness of reflectance indices-based metrics such as Relative Burn Ratio (RBR) and Difference Burned Area Index for Sentinel-2 (dBIAS2), alongside machine learning algorithms including Support Vector Machines (SVM) and Convolutional Neural Networks (CNN), implemented in ArcGIS Pro 3.1.0. Our analysis revealed promising results, particularly in identifying high-severity fire areas. By comparing the output of our methods with ground truth data, we demonstrated the robust performance of our approach, with both SVM and CNN achieving accuracy scores exceeding 0.84. An innovative aspect of our study involved semi-automating the process of training sample labeling using spectral indices rasters and masks. This approach optimizes raster selection for distinct fire severity classes, ensuring accuracy and efficiency in classification. This research contributes to the broader understanding of forest fire dynamics and provides valuable insights for fire management and environmental monitoring efforts in Algeria and similar regions. By accurately mapping fire severity, we can better assess the impacts of climate change and land use changes, facilitating proactive measures to mitigate future fire incidents. [ABSTRACT FROM AUTHOR]

Synthetic aperture radar (SAR) images are often disturbed by speckle noise, making SAR image interpretation tasks more difficult. Therefore, speckle suppression becomes a pre-processing step. In recent years, approaches based on convolutional neural network (CNN) achieved good results in synthetic aperture radar (SAR) images despeckling. However, these CNN-based SAR images despeckling approaches usually require large computational resources, especially in the case of huge training data. In this paper, we proposed a SAR image despeckling method using a CNN platform with a new learnable spatial activation function, which required significantly fewer network parameters without incurring any degradation in performance over the state-of-the-art despeckling methods. Specifically, we redefined the rectified linear units (ReLU) function by adding a convolutional kernel to obtain the weight map of each pixel, making the activation function learnable. Meanwhile, we designed several experiments to demonstrate the advantages of our method. In total, 400 images from Google Earth comprising various scenes were selected as a training set in addition to 10 Google Earth images including athletic field, buildings, beach, and bridges as a test set, which achieved good despeckling effects in both visual and index results (peak signal to noise ratio (PSNR): 26.37 ± 2.68 and structural similarity index (SSIM): 0.83 ± 0.07 for different speckle noise levels). Extensive experiments were performed on synthetic and real SAR images to demonstrate the effectiveness of the proposed method, which proved to have a superior despeckling effect and higher ENL magnitudes than the existing methods. Our method was applied to coniferous forest, broad-leaved forest, and conifer broad-leaved mixed forest and proved to have a good despeckling effect (PSNR: 23.84 ± 1.09 and SSIM: 0.79 ± 0.02). Our method presents a robust framework inspired by the deep learning technology that realizes the speckle noise suppression for various remote sensing images. [ABSTRACT FROM AUTHOR]

In this paper a non-parametric model based on Wasserstein CNN is proposed for color correction. It is suitable for large-scale remote sensing image preprocessing from multiple sources under various viewing conditions, including illumination variances, atmosphere disturbances, and sensor and aspect angles. Color correction aims to alter the color palette of an input image to a standard reference which does not suffer from the mentioned disturbances. Most of current methods highly depend on the similarity between the inputs and the references, with respect to both the contents and the conditions, such as illumination and atmosphere condition. Segmentation is usually necessary to alleviate the color leakage effect on the edges. Different from the previous studies, the proposed method matches the color distribution of the input dataset with the references in a probabilistic optimal transportation framework. Multi-scale features are extracted from the intermediate layers of the lightweight CNN model and are utilized to infer the undisturbed distribution. The Wasserstein distance is utilized to calculate the cost function to measure the discrepancy between two color distributions. The advantage of the method is that no registration or segmentation processes are needed, benefiting from the local texture processing potential of the CNN models. Experimental results demonstrate that the proposed method is effective when the input and reference images are of different sources, resolutions, and under different illumination and atmosphere conditions. [ABSTRACT FROM AUTHOR]

The development of convolution neural networks (CNNs) has become a significant means to solve the problem of remote sensing scene image classification. However, well-performing CNNs generally have high complexity and are prone to overfitting. To handle the above problem, we present a new classification approach using an mmsCNN–HMM combined model with stacking ensemble mechanism in this paper. First of all, a modified multi-scale convolution neural network (mmsCNN) is proposed to extract multi-scale structural features, which has a lightweight structure and can avoid high computational complexity. Then, we utilize a hidden Markov model (HMM) to mine the context information of the extracted features of the whole sample image. For different categories of scene images, the corresponding HMM is trained and all the trained HMMs form an HMM group. In addition, our approach is based on a stacking ensemble learning scheme, in which the preliminary predicted values generated by the HMM group are used in an extreme gradient boosting (XGBoost) model to generate the final prediction. This stacking ensemble learning mechanism integrates multiple models to make decisions together, which can effectively prevent overfitting while ensuring accuracy. Finally, the trained XGBoost model conducts the scene category prediction. In this paper, the six most widely used remote sensing scene datasets, UCM, RSSCN, SIRI-WHU, WHU-RS, AID, and NWPU, are selected to carry out all kinds of experiments. The numerical experiments verify that the proposed approach shows more important advantages than the advanced approaches.

With the rise of artificial intelligence, many advanced Synthetic Aperture Radar (SAR) ship classifiers based on convolutional neural networks (CNNs) have achieved better accuracies than traditional hand-crafted feature ones. However, most existing CNN-based models uncritically abandon traditional hand-crafted features, and rely excessively on abstract ones of deep networks. This may be controversial, potentially creating challenges to improve classification performance further. Therefore, in view of this situation, this paper explores preliminarily the possibility of injection of traditional hand-crafted features into modern CNN-based models to further improve SAR ship classification accuracy. Specifically, we will—(1) illustrate what this injection technique is, (2) explain why it is needed, (3) discuss where it should be applied, and (4) describe how it is implemented. Experimental results on the two open three-category OpenSARShip-1.0 and seven-category FUSAR-Ship datasets indicate that it is effective to perform injection of traditional hand-crafted features into CNN-based models to improve classification accuracy. Notably, the maximum accuracy improvement reaches 6.75%. Hence, we hold the view that it is not advisable to abandon uncritically traditional hand-crafted features, because they can also play an important role in CNN-based models. [ABSTRACT FROM AUTHOR]

For the actual collected point cloud data, there are widespread challenges such as semantic inconsistency, density variations, and sparse spatial distribution. A network called PointMM is developed in this study to enhance the accuracy of point cloud semantic segmentation in complex scenes. The main contribution of PointMM involves two aspects: (1) Multi-spatial feature encoding. We leverage a novel feature encoding module to learn multi-spatial features from the neighborhood point set obtained by k-nearest neighbors (KNN) in the feature space. This enhances the network's ability to learn the spatial structures of various samples more finely and completely. (2) Multi-head attention pooling. We leverage a multi-head attention pooling module to address the limitations of symmetric function-based pooling, such as maximum and average pooling, in terms of losing detailed feature information. This is achieved by aggregating multi-spatial and attribute features of point clouds, thereby enhancing the network's ability to transmit information more comprehensively and accurately. Experiments on publicly available point cloud datasets S3DIS and ISPRS 3D Vaihingen demonstrate that PointMM effectively learns features at different levels, while improving the semantic segmentation accuracy of various objects. Compared to 12 state-of-the-art methods reported in the literature, PointMM outperforms the runner-up by 2.3% in OA on the ISPRS 3D Vaihingen dataset, and achieves the third best performance in both OA and MioU on the S3DIS dataset. Both achieve a satisfactory balance between OA, F1, and MioU. [ABSTRACT FROM AUTHOR]

The timely, accurate acquisition of geographic spatial information such as the location, scope, and distribution of built-up areas is of great importance for urban planning, management, and decision-making. Due to the diversity of target features and the complexity of spatial layouts, the large-scale mapping of urban built-up areas using high-resolution (HR) satellite imagery still faces considerable challenges. To address this issue, this study adopted a block-based processing strategy and constructed a lightweight multilevel feature-fusion (FF) convolutional neural network for the feature representation and discrimination of built-up areas in HR images. The proposed network consists of three feature extraction modules composed of lightweight convolutions to extract features at different levels, which are further fused sequentially through two attention-based FF modules. Furthermore, to improve the problem of incorrect discrimination and severe jagged boundaries caused by block-based processing, a majority voting method based on a grid offset is adopted to achieve a refined extraction of built-up areas. The effectiveness of this method is evaluated using Gaofen-2 satellite image data covering Shenzhen, China. Compared with several state-of-the-art algorithms for detecting built-up areas, the proposed method achieves a higher detection accuracy and preserves better shape integrity and boundary smoothness in the extracted results. [ABSTRACT FROM AUTHOR]

Since the early 1950s, the development of human settlements and over-exploitation of agriculture in the China side of the Amur River Basin (CARB) have had a major impact on the water environment of the surrounding lakes, resulting in a decrease of aquatic vegetation. According to the United Nations Sustainable Development Goals, a comprehensive understanding of the extent and variability of aquatic vegetation is crucial for preserving the structure and functionality of stable aquatic ecosystems. Currently, there is a deficiency in the CARB long-sequence dataset of aquatic vegetation distribution in China. This shortage hampers effective support for actual management. Therefore, the development of a fast, robust, and automatic method for accurate extraction of aquatic vegetation becomes crucial for large-scale applications. Our objective is to gather information on the spatial and temporal distribution as well as changes in aquatic vegetation within the CARB. Utilizing a hybrid approach that combines the maximum spectral index composite and Otsu algorithm, along with the integration of convolutional neural networks (CNN) and random forest, we applied this methodology to obtain an annual dataset of aquatic vegetation spanning from 1985 to 2020 using Landsat series imagery. The accuracy of this method was validated through both field investigations and Google Images. Upon assessing the confusion matrix spanning from 1985 to 2020, the producer accuracy for aquatic vegetation classification consistently exceeded 87%. Further quantitative analysis unveiled a discernible decreasing trend in both the water and vegetation areas of lakes larger than 20 km2 within the CARB over the past 36 years. Specifically, the total water area decreased from 3575 km2 to 3412 km2, while the vegetation area decreased from 745 km2 to 687 km2. These changes may be attributed to a combination of climate change and human activities. These quantitative data hold significant practical implications for establishing a scientific restoration path for lake aquatic vegetation. They are particularly valuable for constructing the historical background and reference indices of aquatic vegetation. [ABSTRACT FROM AUTHOR]

The recently developed Ice, Cloud, and Land Elevation Satellite 2 (ICESat-2), furnished with the Advanced Terrain Laser Altimeter System (ATLAS), delivers considerable benefits in providing accurate bathymetric data across extensive geographical regions. By integrating active lidar-derived reference seawater depth data with passive optical remote sensing imagery, efficient bathymetry mapping is facilitated. In recent times, machine learning models are frequently used to define the nonlinear connection between remote sensing spectral data and water depths, which consequently results in the creation of bathymetric maps. A salient model among these is the convolutional neural network (CNN), which effectively integrates contextual information concerning bathymetric points. However, current CNN models and other machine learning approaches mainly concentrate on recognizing mathematical relationships within the data to determine a water depth function and remote sensing spectral data, while oftentimes disregarding the physical light propagation process in seawater before reaching the seafloor. This study presents a physics-informed CNN (PI-CNN) model which incorporates radiative transfer-based data into the CNN structure. By including the shallow water double-band radiative transfer physical term (swdrtt), this model enhances seawater spectral features and also considers the context surroundings of bathymetric pixels. The effectiveness and reliability of our proposed PI-CNN model are verified using in situ data from St. Croix and St. Thomas, validating its correctness in generating bathymetric maps with a broad experimental R2 accuracy exceeding 95% and remaining errors below 1.6 m. Preliminary results suggest that our PI-CNN model surpasses conventional methodologies. [ABSTRACT FROM AUTHOR]

Imaging spectroscopy (hyperspectral sensing) is a proven tool for mapping and monitoring the spatial distribution of vegetation species composition. However, there exists a gap when it comes to the availability of high-resolution spatial and spectral imagery for accurate tree species mapping, particularly in complex forest environments, despite the continuous advancements in operational remote sensing and field sensor technologies. Here, we aim to bridge this gap by enhancing our fundamental understanding of imaging spectrometers via complex simulated environments. We used DIRSIG, a physics-based, first-principles simulation approach to model canopy-level reflectance for 3D plant models and species-level leaf reflectance in a synthetic forest scene. We simulated a realistic scene, based on the same species composition, found at Harvard Forest, MA (USA). Our simulation approach allowed us to better understand the interplay between instrument parameters and landscape characteristics, and facilitated comprehensive traceability of error budgets. To enhance our understanding of the impact of sensor design on classification performance, we simulated image samples at different spatial, spectral, and scale resolutions (by modifying the pixel pitch and the total number of pixels in the sensor array, i.e., the focal plane dimension) of the imaging sensor and assessed the performance of a deep learning-based convolutional neural network (CNN) and a traditional machine learning classifier, support vector machines (SVMs), to classify vegetation species. Overall, across all resolutions and species mixtures, the highest classification accuracy varied widely from 50 to 84%, and the number of genus-level species classes identified ranged from 2 to 17, among 24 classes. Harnessing this simulation approach has provided us valuable insights into sensor configurations and the optimization of data collection methodologies to improve the interpretation of spectral signatures for accurate tree species mapping in forest scenes. Note that we used species classification as a proxy for a host of imaging spectroscopy applications. However, this approach can be extended to other ecological scenarios, such as in evaluating the changing ecosystem composition, detecting invasive species, or observing the effects of climate change on ecosystem diversity. [ABSTRACT FROM AUTHOR]

Anomaly perception of infrared point targets has high application value in many fields, such as maritime surveillance, airspace surveillance, and early warning systems. This kind of abnormality includes the explosion of the target, the separation between stages, the disintegration caused by the abnormal strike, etc. By extracting the radiation characteristics of continuous frame targets, it is possible to analyze and warn the target state in time. Most anomaly detection methods adopt traditional outlier detection, which has the problems of poor accuracy and a high false alarm rate. Driven by data, this paper proposes a new network structure, called AC-LSTM, which combines Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM), and embeds the Periodic Time Series Data Attention module (PTSA). The network can better extract the spatial and temporal characteristics of one-dimensional time series data, and the PTSA module can consider the periodic characteristics of the target in the process of continuous movement, and focus on abnormal data. In addition, this paper also proposes a new time series data enhancement method, which slices and re-amplifies the long time series data. This method significantly improves the accuracy of anomaly detection. Through a large number of experiments, AC-LSTM has achieved higher scores on our collected datasets than other methods.

Learning efficient image representations is at the core of the scene classification task of remote sensing imagery. The existing methods for solving the scene classification task, based on either feature coding approaches with low-level hand-engineered features or unsupervised feature learning, can only generate mid-level image features with limited representative ability, which essentially prevents them from achieving better performance. Recently, the deep convolutional neural networks (CNNs), which are hierarchical architectures trained on large-scale datasets, have shown astounding performance in object recognition and detection. However, it is still not clear how to use these deep convolutional neural networks for high-resolution remote sensing (HRRS) scene classification. In this paper, we investigate how to transfer features from these successfully pre-trained CNNs for HRRS scene classification. We propose two scenarios for generating image features via extracting CNN features from different layers. In the first scenario, the activation vectors extracted from fully-connected layers are regarded as the final image features; in the second scenario, we extract dense features from the last convolutional layer at multiple scales and then encode the dense features into global image features through commonly used feature coding approaches. Extensive experiments on two public scene classification datasets demonstrate that the image features obtained by the two proposed scenarios, even with a simple linear classifier, can result in remarkable performance and improve the state-of-the-art by a significant margin. The results reveal that the features from pre-trained CNNs generalize well to HRRS datasets and are more expressive than the low- and mid-level features. Moreover, we tentatively combine features extracted from different CNN models for better performance. [ABSTRACT FROM AUTHOR]

The random cropping data augmentation method is widely used to train convolutional neural network (CNN)-based target detectors to detect targets in optical images (e.g., COCO datasets). It can expand the scale of the dataset dozens of times while consuming only a small amount of calculations when training the neural network detector. In addition, random cropping can also greatly enhance the spatial robustness of the model, because it can make the same target appear in different positions of the sample image. Nowadays, random cropping and random flipping have become the standard configuration for those tasks with limited training data, which makes it natural to introduce them into the training of CNN-based synthetic aperture radar (SAR) image ship detectors. However, in this paper, we show that the introduction of traditional random cropping methods directly in the training of the CNN-based SAR image ship detector may generate a lot of noise in the gradient during back propagation, which hurts the detection performance. In order to eliminate the noise in the training gradient, a simple and effective training method based on feature map mask is proposed. Experiments prove that the proposed method can effectively eliminate the gradient noise introduced by random cropping and significantly improve the detection performance under a variety of evaluation indicators without increasing inference cost. [ABSTRACT FROM AUTHOR]

Earth observation missions such as Sentinel and Landsat support the large-scale identification of agricultural crops by providing free radar and multispectral satellite images. The extraction of representative image information as well as the combination of different image sources for improved feature selection still represent a major challenge in the field of remote sensing. In this paper, we propose a novel three-dimensional (3D) deep learning U-Net model to fuse multi-level image features from multispectral and synthetic aperture radar (SAR) time series data for seasonal crop-type mapping at a regional scale. For this purpose, we used a dual-stream U-Net with a 3D squeeze-and-excitation fusion module applied at multiple stages in the network to progressively extract and combine multispectral and SAR image features. Additionally, we introduced a distinctive method for generating patch-based multitemporal multispectral composites by selective image sampling within a 14-day window, prioritizing those with minimal cloud cover. The classification results showed that the proposed network provided the best overall accuracy (94.5%) compared to conventional two-dimensional (2D) and three-dimensional U-Net models (2D: 92.6% and 3D: 94.2%). Our network successfully learned multi-modal dependencies between the multispectral and SAR satellite images, leading to improved field mapping of spectrally similar and heterogeneous classes while mitigating the limitations imposed by persistent cloud coverage. Additionally, the feature representations extracted by the proposed network demonstrated their transferability to a new cropping season, providing a reliable mapping of spatio-temporal crop type patterns.

In smart transportation, assisted driving relies on data integration from various sensors, notably LiDAR and cameras. However, their optical performance can degrade under adverse weather conditions, potentially compromising vehicle safety. Millimeter-wave radar, which can overcome these issues more economically, has been re-evaluated. Despite this, developing an accurate detection model is challenging due to significant noise interference and limited semantic information. To address these practical challenges, this paper presents the TC–Radar model, a novel approach that synergistically integrates the strengths of transformer and the convolutional neural network (CNN) to optimize the sensing potential of millimeter-wave radar in smart transportation systems. The rationale for this integration lies in the complementary nature of CNNs, which are adept at capturing local spatial features, and transformers, which excel at modeling long-range dependencies and global context within data. This hybrid approach allows for a more robust and accurate representation of radar signals, leading to enhanced detection performance. A key innovation of our approach is the introduction of the Cross-Attention (CA) module, which facilitates efficient and dynamic information exchange between the encoder and decoder stages of the network. This CA mechanism ensures that critical features are accurately captured and transferred, thereby significantly improving the overall network performance. In addition, the model contains the dense information fusion block (DIFB) to further enrich the feature representation by integrating different high-frequency local features. This integration process ensures thorough incorporation of key data points. Extensive tests conducted on the CRUW and CARRADA datasets validate the strengths of this method, with the model achieving an average precision (AP) of 83.99% and a mean intersection over union (mIoU) of 45.2%, demonstrating robust radar sensing capabilities.