Search

Effective features play an important role in synthetic aperture radar (SAR) image interpretation. However, since SAR images contain a variety of terrain types, it is not easy to extract effective features of different terrains from SAR images. Deep learning methods require a large amount of labeled data, but the difficulty of SAR image annotation limits the performance of deep learning models. SAR images have inevitable geometric distortion and coherence speckle noise, which makes it difficult to extract effective features from SAR images. If effective semantic context features cannot be learned for SAR images, the extracted features struggle to distinguish different terrain categories. Some existing terrain classification methods are very limited and can only be applied to some specified SAR images. To solve these problems, a jigsaw puzzle self-supervised learning (JPSSL) framework is proposed. The framework comprises a jigsaw puzzle pretext task and a terrain classification downstream task. In the pretext task, the information in the SAR image is learned by completing the SAR image jigsaw puzzle to extract effective features. The terrain classification downstream task is trained using only a small number of labeled data. Finally, fully connected conditional random field processing is performed to eliminate noise points and obtain a high-quality terrain classification result. Experimental results on three large-scene high-resolution SAR images confirm the effectiveness and generalization of our method. Compared with the supervised methods, the features learned in JPSSL are highly discriminative, and the JPSSL achieves good classification accuracy when using only a small amount of labeled data. [ABSTRACT FROM AUTHOR]

Panchromatic (PAN) and multispectral (MS) images have coordinated and paired spatial spectral information, which can complement each other and make up for their shortcomings for image interpretation. In this article, a novel classification method called the deep group spatial–spectral attention fusion network is proposed for PAN and MS images. First, the MS image is processed by unpooling to obtain the same resolution as that of the PAN image. Second, the group spatial attention and group spectral attention modules are proposed to extract image features. The PAN and the processed MS images are regarded as the input of the two modules, respectively. Third, the features from the previous step are fused by the attention fusion module, which aims to fully fuse multilevel features, take into account both the low-level features and the high-level features, and maintain the global abstract and local detailed information of the pixels. Finally, the fusion feature is fed into the classifier and the resulting map is obtained by pixel level. Extensive experiments and analysis on four datasets show that the proposed method achieves comparable results. [ABSTRACT FROM AUTHOR]

Deep descriptors have advantages over handcrafted descriptors on local image patch matching. However, due to the complex imaging mechanism of remote sensing images and the significant differences in appearance between multi-modal images, existing deep learning descriptors are unsuitable for multi-modal remote sensing image registration directly. To solve this problem, this article proposes a deep feature correlation learning network (Cnet) for multi-modal remote sensing image registration. First, Cnet builds a feature learning network based on the deep convolutional network with the attention learning module, to enhance feature representation by focusing on meaningful features. Second, this article designs a novel feature correlation loss function for Cnet optimization. It focuses on the relative feature correlation between matching and nonmatching samples, which can improve the stability of network training and decrease the risk of overfitting. In addition, the proposed feature correlation loss with a scale factor can further enhance network training and accelerate network convergence. Extensive experimental results on image patch matching (Brown, HPatches), cross-spectral image registration (VIS-NIR), multi-modal remote sensing image registration, and single-modal remote sensing image registration have demonstrated the effectiveness and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Remote sensing object tracking (RSOT) is a novel and challenging problem due to the negative effects of weak features and background noise. In this article, from the perspective of attention-focus deep learning, we propose a joint Siamese attention-aware network (JSANet) for efficient remote sensing tracking which contains both the self-attention and cross-attention modules. First, the self-attention modules we propose emphasize the interdependent channel-wise coefficient via channel attention and conduct corresponding space transformation of spatial domain information with spatial attention. Second, the cross-attention is designed to aggregate rich contextual interdependencies between the Siamese branches via channel attention and excavate association produces reliable correspondence with spatial attention. In addition, a composite feature combine strategy is designed to fuse multiple attention features. Experimental results on the Jilin-1 satellite video datasets demonstrate that the proposed JSANet achieves state-of-the-art performance in terms of precision and success rate, demonstrating the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

With the development of the imaging technology of various sensors, multisource image classification has become a key challenge in the field of image interpretation. In this article, a novel classification method, called the deep multiview union learning network (DMULN), is proposed to classify multisensor data. First, an associated feature extractor is designed to process the multisource data by canonical correlation analysis (CCA) in the head of the network. Second, an improved deep learning architecture with two branches is presented to extract high-level view features from the associated features. Third, a novel pooling, called view union pooling, is proposed to fuse the multiview feature from the deep model. Finally, the fused feature is fed into the classifier. The proposed framework is easy to optimize since it is an end-to-end network. Extensive experiments and analysis on the datasets $IEEE\_{}grss\_{}dfc\_{}2017$ and $IEEE\_{}grss\_{}dfc\_{}2018$ show that the proposed method achieves comparable results. Our results demonstrate that abundant multisource information can improve the classification performance. [ABSTRACT FROM AUTHOR]

Recently, multicategory object detection in high-resolution remote sensing images is still a challenge. First, objects with significant scale differences exist in one scene simultaneously, so it is generally difficult for the detectors to balance the detection performance of large and small objects. Second, because of the complex background and the objects’ densely distributed characteristics in the remote sensing images, the extracted features usually have noise and blurred boundaries, which interfere with the detection performance of the object detectors. With this observation, we propose an end-to-end scale-aware network called feature split–merge–enhancement network (SME-Net) for remote sensing object detection, composed of the feature split-and-merge (FSM) module, the offset-error rectification (OER) module, and the object saliency enhancement (OSE) strategy. FSM eliminates salient information of large objects to highlight the features of small objects in the shallow feature maps. It also transmits the effective detailed features of large objects to the deep feature maps, alleviating feature confusion between multiscale objects. OER corrects the inconsistency of the features spatial layout among the multilayer feature maps by the proposed offset loss, so as to achieve supervised elimination and transmission in FSM. OSE enhances the features of interests and suppresses the background information by the proposed membership function, thus preventing false detection and missed detection caused by noise and blurred boundaries. The effectiveness of the proposed algorithm has been verified on multiple datasets. Our code is available at: https://github.com/Momuli/SMENet.git [ABSTRACT FROM AUTHOR]

Recently, deep learning methods have attracted much attention in the field of polarimetric synthetic aperture radar (PolSAR) data interpretation and understanding. However, for supervised methods, it requires large-scale labeled data to achieve better performance, and getting enough labeled data is a time-consuming and laborious task. Aiming to obtain a good classification result with limited labeled data, we focus on learning discriminative high-level features between multiple representations, which we call mutual information. As PolSAR data have multi-modal representations, there should have strong similarity between multi-modal features of the same pixel. In addition, each pixel has its own unique geocoding and scattering information. Hence, every pixel has great difference from other pixels in a specific representation space. Based on the above observations, this article proposes a mutual information-based self-supervised learning (MI-SSL) model to learn an implicit representation from unlabeled data. In this article, the self-supervised learning idea is first applied to PolSAR data processing. Furthermore, a reasonable pretext task, which is suitable for PolSAR data, is designed to extract mutual information for classification tasks. Compared with the state-of-the-art classification methods, experimental results on four PolSAR data sets demonstrate that our MI-SSL model produces impressive overall accuracy with fewer labeled data. [ABSTRACT FROM AUTHOR]

Synthetic aperture radar (SAR) image classification is a fundamental research direction in image interpretation. With the development of various intelligent technologies, deep learning techniques are gradually being applied to SAR image classification. In this study, a new SAR classification algorithm known as the multiscale convolutional neural network with an autoencoder regularization joint contextual attention network (MCAR-CAN) is proposed. The MCAR-CAN has two branches: the autoencoder regularization branch and the context attention branch. First, autoencoder regularization is used for the reconstruction of the input to regularize the classification in the autoencoder regularization branch. Multiscale input and an asymmetric structure of the autoencoder branch cause the network more to be focused on classification than on reconstruction. Second, the attention mechanism is used to produce an attention map in which each attention weight corresponds to a context correlation in attention branch. The robust features are obtained by the attention mechanism. Finally, the features obtained by the two branches are spliced for classification. In addition, a new training strategy and a postprocessing method are designed to further improve the classification accuracy. Experiments performed on the data from three SAR images demonstrated the effectiveness and robustness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In order to better interpret polarimetric synthetic aperture radar (PolSAR) images, many scholars tend to do target decomposition for PolSAR images and utilize the obtained features to perform subsequent classification. These target decomposition features play an important role in terrain classification but completely utilizing them produces a high computational complexity. Furthermore, some features have a negative impact on the classification task. Therefore, selecting the appropriate amount of high-quality features is of great significance to the classification task. In this paper, we propose a convolutional neural network (CNN)-based feature selection algorithm for PolSAR image classification. First, we design a 1-D CNN for feature selection, then train the designed network with all the decomposition features to obtain a trained model. Second, the Kullback–Leibler distance (KLD) between different features is utilized as a standard to select feature subsets. Third, feature subsets with excellent performance form the final results. Due to the special structure of the 1-D CNN, repetitively training model is avoided when the input changes. Different from traditional feature selection methods, our method considers the performance of features combination rather than single feature contribution. To this end, the feature subsets selected by the proposed method are more useful to the classification task. Innovatively introducing KLD in the selection stage avoids random selection and improves the selection efficiency. Finally, we validate the performance of selected feature subsets in traditional and deep learning classification frameworks. Experiments demonstrate that features selected by the proposed method have a good performance comparing with others on three real PolSAR data sets. [ABSTRACT FROM AUTHOR]

Supervised deep neural networks (DNNs) have been extensively used in diverse tasks. Generally, training such DNNs with superior performance requires a large amount of labeled data. However, it is time-consuming and expensive to manually label the data, especially for tasks in remote sensing, e.g., change detection. The situation motivates us to resort to the existing related images with labels, from which the concept of change can be adapted to new images. However, the distributions of the related labeled images (source domain) and unlabeled new images (target domain) are similar but not identical. It impedes a change detection model learned from source domains being well applied to the target domain. In this paper, we propose a transferred deep learning-based change detection framework to solve this problem. It consists of pretraining and fine-tuning stages. In the pretraining process, we propose two tasks to be learned simultaneously, namely, change detection for the source domain with labels and reconstruction of the unlabeled target data. The auxiliary task aims to reconstruct the difference image (DI) for the target domain. DI is an effective feature, such that the auxiliary task is of much relevance to change detection. The lower layers are shared between these two tasks in the training process. It mitigates the distribution discrepancy between the source and target domains and makes the concept of change from the source domain adapt to the target domain. In addition, we evaluate three modes of the U-net architecture to merge the information for a pair of patches. To fine-tune the change detection network (CDN) for the target domain, two strategies are exploited to select the pixels that have a high possibility of being correctly classified by an unsupervised approach. The proposed method demonstrates an excellent capacity for adapting the concept of change from the source domain to the target domain. It outperforms the state-of-the-art change detection methods via experimental results on real remote sensing data sets. [ABSTRACT FROM AUTHOR]

Deep learning has obtained state-of-the-art results in a variety of computer vision tasks and has also been used to solve SAR image classification problems. Deep learning algorithms typically require a large amount of training data to achieve high accuracy. In contrast, the size of SAR image datasets is often comparatively limited. Therefore, this paper proposes a novel method, deep memory convolution neural networks (M-Net), to alleviate the problem of overfitting caused by insufficient SAR image samples. Based on the convolutional neural networks (CNN), M-Net adds an information recorder to remember and store samples’ spatial features, and then it uses spatial similarity information of the recorded features to predict unknown sample labels. M-Net's use of this information recorder may cause difficulties for convergence if conventional CNN training methods were directly used to train M-Net. To overcome this problem, we propose a transfer parameter technique to train M-Net in two steps. The first step is to train a CNN, which has the same structure as the part of CNN incorporated in M-Net, to obtain initial training parameters. The second step applies the initialized parameters to M-Net and then trains the entire M-Net. This two-step training approach helps us to overcome the nonconvergence issue, and also reduces training time. We evaluate M-Net using the public benchmark MSTAR dataset, and achieve higher accuracy than several other well-known SAR image classification algorithms. [ABSTRACT FROM AUTHOR]

Ship detection is a challenging task for synthetic aperture radar (SAR) images. Ships have arbitrary directionality and multiple scales in SAR images. Furthermore, there is a lot of clutter near the ships. Traditional detection algorithms are not robust to these situations and easily cause redundancy in the detection area. With the continuous improvement in resolution, the traditional algorithms cannot achieve high-precision ship detection in SAR images. An increasing number of deep learning algorithms have been applied to SAR ship detection. In this study, a new ship detection network, known as the instance segmentation assisted ship detection network (ISASDNet), is presented. ISASDNet is a two-stage detection network with two branches. A branch is called an object branch and can extract object-level information to obtain positioning bounding boxes and classification results. Another branch called the pixel branch can be utilized for instance segmentation. In the pixel branch, the designed global relational inference layer maps the features to interaction space to learn the relationship between ship and background. The global reasoning module (GRM) based on global relational inference layers can better extract the instance segmentation results of ships. A mask assisted ship detection module (MASDM) is behind the two branches. The MASDM can improve detection results by interacting with the outputs of the two branches. In addition, a strategy is designed to extract the mask of SAR ships, which enables ISASDNet to perform object detection training and instance segmentation training at the same time. Experiments carried out two different datasets demonstrated the superiority of ISASDNet over other networks. [ABSTRACT FROM AUTHOR]

Recently, with the popularity of space-borne earth satellites, the resolution of high-resolution panchromatic (PAN) and multispectral (MS) remote sensing images is also increasing year by year, multiresolution remote sensing classification has become a research hotspot. In this paper, from the perspective of deep learning, we design a dual-branch interactive spatial-channel collaborative attention enhancement network (SCCA-net) for multiresolution classification. It aims to combine sample enhancement and feature enhancement to improve classification accuracy. In the part of sample enhancement, we propose an adaptive neighbourhood transfer sampling strategy (ANTSS). Different from the traditional pixel-centric sampling strategy with orthogonal sampling angle, our algorithm allows each patch to adaptively transfer the neighbourhood range by finding the homogeneous region of the pixel to be classified. And it also adaptively adjust the sampling angle according to the texture distribution of the homogeneous region to capture neighbourhood information that is more conducive for classification. Moreover, in the part of feature enhancement part, we design a local spatial attention module (LSA-module) for PAN data to highlight the spatial resolution advantages and a global channel attention module (GCA-module) for MS data to improve the multi-channel representation. It not only highlights the spatial resolution advantage of PAN data and the multi-channel advantage of MS data, but also improves the difference between features through the interaction between the two modules. Quantitative and qualitative experimental results verify the robustness and effectiveness of the method. [ABSTRACT FROM AUTHOR]

Recently, classification methods based on deep learning have attained sound results for the classification of Polarimetric synthetic aperture radar (PolSAR) data. However, they generally require a great deal of labeled data to train their models, which limits their potential real-world applications. This paper proposes a novel semi-supervised deep metric learning network (SSDMLN) for feature learning and classification of PolSAR data. Inspired by distance metric learning, we construct a network, which transforms the linear mapping of metric learning into the non-linear projection in the layer-by-layer learning. With the prior knowledge of the sample categories, the network also learns a distance metric under which all pairs of similarly labeled samples are closer and dissimilar samples have larger relative distances. Moreover, we introduce a new manifold regularization to reduce the distance between neighboring samples since they are more likely to be homogeneous. The categorizing is achieved by using a simple classifier. Several experiments on both synthetic and real-world PolSAR data from different sensors are conducted and they demonstrate the effectiveness of SSDMLN with limited labeled samples, and SSDMLN is superior to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• A dual branch attention fusion deep network for classification is proposed. • An adaptive center offset sampling strategy for each patch is proposed. • Two attention based modules to highlight the advantages of two different data. • The feature level fusion is achieved by the proposed network. In recent years, with the diversification of acquisition methods of very high resolution panchromatic (PAN) and multispectral (MS) remote sensing images, multiresolution remote sensing classification has become a research hotspot. In this paper, from the perspective of data–driven deep learning, we design a dual–branch attention fusion deep network (DBAF–Net) for the multiresolution classification. It aims to integrate the feature–level fusion and classification into an end–to–end network model. In the process of establishing a training sample library, unlike the traditional pixel–centric sampling strategy with fixed patch size, we propose an adaptive center–offset sampling strategy (ACO–SS), which allows each patch to adaptively determine the neighborhood range by finding the texture structure of the pixel to be classified. And the neighborhood range is not symmetrical with this pixel, we expect to capture the neighborhood information that is more conducive to its classification. In network structure, based on the captured patches by ACO–SS, we design a spatial attention module (SA–module) for PAN data and a channel attention module (CA–module) for MS data, thus highlighting the spatial resolution advantages of PAN data and the multi–channel advantages of MS data, respectively. Then these two features are interfused to improve and strengthen the fusion features in both spatial and channel. The quantitative and qualitative experimental results verify the robustness and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract In this paper, a fast unsupervised deep fusion framework for change detection of multitemporal synthetic aperture radar (SAR) images is presented. It mainly aim at generating a difference image (DI) in the feature learning procedure by stacked auto-encoders (SAEs). Stacked auto-encoders, as one kind of deep neural network, can learn feature maps that retain the structural information but suppress the noise in the SAR images, which will be beneficial for DI generation. Compared with shallow network, the proposed framework can extract more available features, and be favorable for getting better change results. Different with other common deep neural networks, our proposed method does not need labeled data to train the network. In addition, we find a subset of the entire samples that appropriately represent the whole dataset to speed up the training of the deep neural network without under-fitting. Moreover, we design a fusion network structure that can combine ratio operator based method to ensure that the representations of higher layers are better than that of the lower ones. To summarize, the main contribution of our work lies in using of deep fusion network for generation of DI in a fast and unsupervised way. Experiments on four real SAR images confirm that our network performs better than traditional ratio methods and convolutional neural network. [ABSTRACT FROM AUTHOR]