Search

Synthetic aperture radar (SAR) automatic target recognition (ATR) has been widely applied in multiple fields. However, the special imaging mechanism of SAR results in different visual features of the same target at different azimuth angles, so single-aspect SAR target recognition has the limitation of observing the target from a single perspective. Multi-aspect SAR target recognition technology can overcome this limitation by utilizing information from different azimuths and effectively improve target recognition performance. Considering the order dependency and data limitation of existing methods, this paper proposes a multi-aspect SAR recognition method based on Non-Local, which applies a self-attention calculation to feature maps to learn the correlation between multi-aspect SAR images. Meanwhile, in order to improve the generalization ability of the proposed method under limited data, a network based on contrastive learning was designed to pre-train the feature extraction part of the whole network. The experimental results using the MSTAR dataset show that the proposed method has excellent recognition accuracy and good robustness.

Radio frequency electromagnetic interference is a relatively common phenomenon, especially for synthetic aperture radar (SAR) systems working in P- or L-band. Compared with the suppression of narrowband interference, that of wideband interference, particularly of those whose signal parameters have frequently changing property, is still a sophisticated problem. In this letter, a suppression algorithm for interference with wideband and complicated parameters is proposed, based on low-rank and sparse matrix decomposition (LRSMD) in time-frequency domain (TFD) of the signal. The proposed algorithm begins with transforming the SAR signal into TFD. After that, LRSMD based on bilateral random projection (BRP) is applied to decompose the time-frequency spectrum matrix into three parts, a low-rank matrix standing for interference, a sparse matrix standing for SAR signal, and a noise matrix. Finally, inversely transform the sparse matrix into the time domain to obtain SAR signal without interference. The proposed algorithm is applied to a single look complex (SLC) SAR data of Sentinel-1 to validate its effect and efficiency.

Understanding the physical properties and scattering mechanisms contributes to synthetic aperture radar (SAR) image interpretation. For single-polarized SAR data, however, it is difficult to extract the physical scattering mechanisms due to lack of polarimetric information. Time–frequency analysis (TFA) on complex-valued SAR image provides extra information in frequency perspective beyond the “image” domain. Based on TFA theory, we propose to generate the subband scattering pattern for every object in complex-valued SAR image as the physical property representation, which reveals backscattering variations along slant-range and azimuth directions. In order to discover the inherent patterns and generate a scattering classification map from single-polarized SAR image, an unsupervised hierarchical deep embedding clustering (HDEC) algorithm based on TFA (HDEC-TFA) is proposed to learn the embedded features and cluster centers simultaneously and hierarchically. The polarimetric analysis result for quad-pol SAR images is applied as reference data of physical scattering mechanisms. In order to compare the scattering classification map obtained from single-polarized SAR data with the physical scattering mechanism result from full-polarized SAR, and to explore the relationship and similarity between them in a quantitative way, an information theory based evaluation method is proposed. We take Gaofen-3 quad-polarized SAR data for experiments, and the results and discussions demonstrate that the proposed method is able to learn valuable scattering properties from single-polarization complex-valued SAR data, and to extract some specific targets as well as polarimetric analysis. At last, we give a promising prospect to future applications.

Deep convolutional neural networks (DCNNs) have attracted extensive attention in synthetic aperture radar (SAR) image target detection. However, there is usually a lack of labeled data in SAR images which results in difficulties to improve the detection performance with DCNN based methods. Transfer learning borrows generic knowledge from other tasks for the target task and can be an effective solution to the small sample problem, but the features’ generality and specificity across various models are not discussed in target detection. In addition, conventional DCNN based models have too many parameters related to the prior information of the targets which increase the difficulty of transfer learning between different tasks. In this paper, we mainly conduct research in three folds to address these mentioned issues. First, the generality and specificity of features in different network models are analyzed in the field of SAR target detection. Second, a transitive transfer framework is developed for detection under small sample condition. Third, an anchor free rotatable detector with flexible structure is designed for SAR images detection to suit the transfer detection framework proposed in this paper and a reasonable explanation is given for its upper hand with small samples. Experimental results demonstrate that: 1) transitive transfer framework is valid in SAR target detection; 2) the anchor free detector in this framework is superior to the anchor based one; 3) the proposed detector achieves better performance under small sample condition by comparison with state-of-the-art methods.

This paper introduces a novel Synthetic Aperture Radar (SAR) specific deep learning framework for complex-valued SAR images. The conventional deep convolutional neural networks based methods usually take the amplitude information of single-polarization SAR images as the input to learn hierarchical spatial features automatically, which may have difficulties in discriminating objects with similar texture but discriminative scattering patterns. Our novel deep learning framework, Deep SAR-Net, takes complex-valued SAR images into consideration to learn both spatial texture information and backscattering patterns of objects on the ground. On the one hand, we transfer the detected SAR images pre-trained layers to extract spatial features from intensity images. On the other hand, we dig into the Fourier domain to learn physical properties of the objects by joint time-frequency analysis on complex-valued SAR images. We evaluate the effectiveness of Deep SAR-Net on three complex-valued SAR datasets from Sentinel-1 and TerraSAR-X satellite and demonstrate how it works better than conventional deep CNNs, especially on man-made objects classes. The proposed datasets and the trained Deep SAR-Net model with all codes are provided.

Deep learning has been successfully utilized in synthetic aperture radar (SAR) automatic target recognition tasks and obtained state-of-the-art results. However, current deep learning algorithms do not perform well when SAR images are occluded, noisy, or with a great depression angle variance. This article proposes a novel method, SAR capsule network, to achieve the accurate and robust classification of SAR images without significantly increasing network complexity. Specifically, we develop a convolutional neural network extension based on Hinton's capsule network to capture spatial relationships specialized in classification between different entities in a SAR image. The SAR capsules are learned by a vector-based full connection operation instead of the traditional routing process, which not only alleviates the computational burden but also improves recognition accuracy. For occlusion, additive noise, and multiplicative noise tests, SAR capsule network shows superior robustness compared with typical convolution neural networks. When missing training data in a certain aspect angle range or existing a large depression angle variance between training data and test data, the proposed network achieves better performance than the existing works and reveals some competitive advantages in several test scenarios.

Recently, deep learning-based methods have made great improvements in object detection in remote sensing images (RSIs). However, detecting tiny objects in low-resolution images is still challenging. The features of these objects are not distinguishable enough due to their tiny size and confusing backgrounds and can be easily lost as the network deepens or downsamples. To address these issues, we propose an effective Tiny Ship Detector for Low-Resolution RSIs, abbreviated as LR-TSDet, consisting of three key components: a filtered feature aggregation (FFA) module, a hierarchical-atrous spatial pyramid (HASP) module, and an IoU-Joint loss. The FFA module captures long-range dependencies by calculating the similarity matrix so as to strengthen the responses of instances. The HASP module obtains deep semantic information while maintaining the resolution of feature maps by aggregating four parallel hierarchical-atrous convolution blocks of different dilation rates. The IoU-Joint loss is proposed to alleviate the inconsistency between classification and regression tasks, and guides the network to focus on samples that have both high localization accuracy and high confidence. Furthermore, we introduce a new dataset called GF1-LRSD collected from the Gaofen–1 satellite for tiny ship detection in low-resolution RSIs. The resolution of images is 16m and the mean size of objects is about 10.9 pixels, which are much smaller than public RSI datasets. Extensive experiments on GF1-LRSD and DOTA-Ship show that our method outperforms several competitors, proving its effectiveness and generality.

Radar detection is an effective earth observation means, and target recognition in radar images is its important research direction. Deep learning has been successfully applied to many fields but training deep neural networks requires a mass of data. The lack of samples has become the major factor that impedes the application of deep learning approaches to target recognition in radar images. This paper reviews the research progress of deep learning based target recognition in radar images, with representative methods being combed and summarized. First, data augmentation and neural network models designed for the task of radar image target recognition are introduced. The paper then presents in detail the target recognition methods based on transfer learning, metric learning, and semi-supervised learning in radar images with few samples, which are proposed by our research group. Finally, existing problems and future development trends are discussed.

At present, synthetic aperture radar (SAR) automatic target recognition (ATR) has been deeply researched and widely used in military and civilian fields. SAR images are very sensitive to the azimuth aspect of the imaging geomety, the same target at different aspects differs greatly. Thus, the multi-aspect SAR image sequence contains more information for classification and recognition, which requires the reliable and robust multi-aspect target recognition method. Nowadays, SAR target recognition methods are mostly based on deep learning. However, the SAR dataset is usually expensive to obtain, especially for a certain target. It is difficult to obtain enough samples for deep learning model training. This paper proposes a multi-aspect SAR target recognition method based on a prototypical network. Furthermore, methods such as multi-task learning and multi-level feature fusion are also introduced to enhance the recognition accuracy under the case of a small number of training samples. The experiments by using the MSTAR dataset have proven that the recognition accuracy of our method can be close to the accruacy level by all samples and our method can be applied to other feather extraction models to deal with small sample learning problems.

The classification of large-scale high-resolution synthetic aperture radar (SAR) land cover images acquired by satellites is a challenging task, facing several difficulties such as semantic annotation with expertise, changing data characteristics due to varying imaging parameters or regional target area differences, and complex scattering mechanisms being different from optical imaging. Given a large-scale SAR land cover data set collected from TerraSAR-X images with a hierarchical three-level annotation of 150 categories and comprising more than 100 000 patches, three main challenges in automatically interpreting SAR images of highly imbalanced classes, geographic diversity, and label noise are addressed. In this letter, a deep transfer learning method is proposed based on a similarly annotated optical land cover data set (NWPU-RESISC45). Besides, a top-2 smooth loss function with cost-sensitive parameters was introduced to tackle the label noise and imbalanced classes’ problems. The proposed method shows high efficiency in transferring information from a similarly annotated remote sensing data set, a robust performance on highly imbalanced classes, and is alleviating the overfitting problem caused by label noise. What is more, the learned deep model has a good generalization for other SAR-specific tasks, such as MSTAR target recognition with a state-of-the-art classification accuracy of 99.46%.

Ship target recognition has always been a hot issue in the field of ocean surveillance. Due to the serious shortage of samples in ship target recognition for synthetic aperture radar (SAR) images, the template-based method is still one of the most effective ways to solve the problem. In this letter, we put forward a novel ship recognition method based on the projection shape template (PST), aiming at increasing both the accuracy and the robustness of the recognition. The PST of each category is calculated by projecting the 3-D model obtained from the two-view images of the target to the 2-D slant-plane image according to the SAR imaging model. Then, we propose a contour extraction method to detect the profile of ships, which served as the feature. Finally, the identity of the query ship is obtained through contour matching. Experimental results indicate that the proposed method is effective even when the number of samples is extremely small, consequently providing a promising way for the automatic interpretation of ship targets in the SAR images.

Single image super-resolution (SISR) has been widely studied in recent years as a crucial technique for remote sensing applications. In this paper, a dense residual generative adversarial network (DRGAN)-based SISR method is proposed to promote the resolution of remote sensing images. Different from previous super-resolution (SR) approaches based on generative adversarial networks (GANs), the novelty of our method mainly lies in the following factors. First, we made a breakthrough in terms of network architecture to improve performance. We designed a dense residual network as the generative network in GAN, which can make full use of the hierarchical features from low-resolution (LR) images. We also introduced a contiguous memory mechanism into the network to take advantage of the dense residual block. Second, we modified the loss function and altered the model of the discriminative network according to the Wasserstein GAN with a gradient penalty (WGAN-GP) for stable training. Extensive experiments were performed using the NWPU-RESISC45 dataset, and the results demonstrated that the proposed method outperforms state-of-the-art methods in terms of both objective evaluation and subjective perspective.

Objects in remote sensing images are difficult to detect due to arbitrarily rotated angles and the wide variance of scales. As bounding box plays an important role in object detection, we proposes a new bounding box type named rotated bounding box (rBox). With the application of rBox, we have proposed a series of detection techniques (DrBox, DrBoxLight, DrBoxSemi, DrBoxPro) to effectively handle the situation where the orientation angles of the objects are arbitrary. This article is a brief overview of these techniques. The original DrBox detector applies VGG-net as its main network framework, with image pyramid input to address multi-scale problem. DrBoxLight is a mini version of DrBox, which applies MobileNet and knowledge distillation to be deployed on embedded devices. DrBoxSemi is a semi-supervised version of DrBox, so annotation of all training samples is no longer necessary. DrBoxPro is the most important update for DrBox with professional designing of abundant prior-rBoxes on feature pyramid networks. In our experiments, we demonstrated how rBox helps to improve the performance of object detection compared with traditional bounding boxes. Besides, we evaluated the performance of our DrBox family on a series of object detection tasks.

A Siamese network based metric learning method is proposed for SAR target classification with few training samples. The network consists of two identical CNNs sharing the weights. Different from classification networks that predict the category of one sample, the Siamese network implements a metric learning to measure the similarity between two samples. Since the input is the sample pair, the amount of training data dramatically increases which contributes to training a better network. When generating the pairs, a hard negative mining scheme is proposed for improving the performance. To avoid computing the similarity between the test sample and each training sample at the test stage, which is time consuming, a two stages scheme is employed with an additional classification network taking the output of the single branch of Siamese network as the input and predicting the category. Experiments on the MSTAR dataset validate the effectiveness of the proposed method.

Deep learning algorithms are widely used in remote sensing image scene understanding. Generally, a large-scale annotated dataset is essential to train a deep neural network for classification. In practical terms, however, a large amount of unknown remote sensing images obtained from different sensors need to be understood which may vary from resolution, geolocation and imaging conditions compared with annotated datasets. In this paper, an unsupervised domain adaptation framework based on ResNet-18 is presented to transfer the knowledge of an existing annotated land cover dataset to other remote sensing data, decreasing the discrepancy among images across sensors. The results show a significant improvement in scene understanding of new remote sensing images.

Deep convolutional neural networks (DCNNs) have attracted much attention in remote sensing recently. Compared with the large-scale annotated dataset in natural images, the lack of labeled data in remote sensing becomes an obstacle to train a deep network very well, especially in SAR image interpretation. Transfer learning provides an effective way to solve this problem by borrowing the knowledge from the source task to the target task. In optical remote sensing application, a prevalent mechanism is to fine-tune on an existing model pre-trained with a large-scale natural image dataset, such as ImageNet. However, this scheme does not achieve satisfactory performance for SAR application because of the prominent discrepancy between SAR and optical images. In this paper, we attempt to discuss three issues that are seldom studied before in detail: (1) what network and source tasks are better to transfer to SAR targets, (2) in which layer are transferred features more generic to SAR targets and (3) how to transfer effectively to SAR targets recognition. Based on the analysis, a transitive transfer method via multi-source data with domain adaptation is proposed in this paper to decrease the discrepancy between the source data and SAR targets. Several experiments are conducted on OpenSARShip. The results indicate that the universal conclusions about transfer learning in natural images cannot be completely applied to SAR targets, and the analysis of what and where to transfer in SAR target recognition is helpful to decide how to transfer more effectively.

Clouds are one of the most serious disturbances when using satellite imagery for ground observations. The semi-translucent nature of thin clouds provides the possibility of 2D ground scene reconstruction based on a single satellite image. In this paper, we propose an effective framework for thin cloud removal involving two aspects: a network architecture and a training strategy. For the network architecture, a Wasserstein generative adversarial network (WGAN) in YUV color space called YUV-GAN is proposed. Unlike most existing approaches in RGB color space, our method performs end-to-end thin cloud removal by learning luminance and chroma components independently, which is efficient at reducing the number of unrecoverable bright and dark pixels. To preserve more detailed features, the generator adopts a residual encoding–decoding network without down-sampling and up-sampling layers, which effectively competes with a residual discriminator, encouraging the accuracy of scene identification. For the training strategy, a transfer-learning-based method was applied. Instead of using either simulated or scarce real data to train the deep network, adequate simulated pairs were used to train the YUV-GAN at first. Then, pre-trained convolutional layers were optimized by real pairs to encourage the applicability of the model to real cloudy images. Qualitative and quantitative results on RICE1 and Sentinel-2A datasets confirmed that our YUV-GAN achieved state-of-the-art performance compared with other approaches. Additionally, our method combining the YUV-GAN with a transfer-learning-based training strategy led to better performance in the case of scarce training data.

An identity regularized sparse representation (IRSR) based SAR target recognition method is proposed in this paper. The method aims to find a transformation that can map the data to a transformed space, in which targets from the same class are close with each other, no matter the distance of them in the original space. This identity constraint can be formulated as a l 1 -norm minimization problem. By decoupling the problem into the sparse coding problem and the dictionary learning problem, the solution can be obtained iteratively. The solution is simply the weighted average of the sparse coding of all training data. Experimental results demonstrate that the proposed method is superior to several related methods.

Target classification is an important part in automatic target recognition (ATR) systems. Deep learning methods get state of the art performance in SAR target classification. Simulation is a useful data augmentation method when the numbers of real samples for training is not sufficient. This article discusses how to release the full potential of simulated samples which is used to improve performance of SAR target classifier. The proposed method is based on cycle adversarial network (CycleGAN), which can transfer simulated samples to be more similar with real samples in image domain. Experiments show that adding simulated samples straightforward into training dataset is not helpful to improve the performance. However, adding the transferred simulated samples for training results in about 10% increase in accuracy in the designed SAR airplane classification experiment, compared with training without data augmentation.

Inshore ship detection in SAR image faces difficulties on correctly identifying near-shore ships and onshore objects. This article proposes a multi-scale full convolutional network (MS-FCN) based sea-land segmentation method and applies a rotatable bounding box based object detection method (DR-Box) to solve the inshore ship detection problem. The sea region and land region are separated by MS-FCN then DR-Box is applied on sea region. The proposed method combines global information and local information of SAR image to achieve high accuracy. The networks are trained with Chinese Gaofen-3 satellite images. Experiments on the testing image show most inshore ships are successfully located by the proposed method.

Single image super-resolution (SR) has been widely studied in recent years as a crucial technique for remote sensing applications. This paper proposes a SR method for remote sensing images based on a transferred generative adversarial network (TGAN). Different from the previous GAN-based SR approaches, the novelty of our method mainly reflects from two aspects. First, the batch normalization layers are removed to reduce the memory consumption and the computational burden, as well as raising the accuracy. Second, our model is trained in a transfer-learning fashion to cope with the insufficiency of training data, which is the crux of applying deep learning methods to remote sensing applications. The model is firstly trained on an external dataset DIV2K and further fine-tuned with the remote sensing dataset. Our experimental results demonstrate that the proposed method is superior to SRCNN and SRGAN in terms of both the objective evaluation and the subjective perspective.