Search

With the widespread application of deep learning methods in Synthetic Aperture Radar (SAR) image interpretation, the explainability of SAR target recognition deep networks has gradually attracted the attention of scholars. Class Activation Mapping (CAM), a commonly used explainability algorithm, can visually display the salient regions influencing the recognition task through heatmaps. However, as a post hoc explanation method, CAM can only statically display the salient regions during the current recognition process and cannot dynamically show the variation patterns of the salient regions upon changing the input. This study introduces the concept of perturbation into CAM, proposing an algorithm called SAR Clutter Characteristics CAM (SCC-CAM). By introducing globally distributed perturbations to the input image, interference is gradually applied to deep SAR recognition networks, causing decision flips. The degree of change in the activation values of network neurons is also calculated. This method addresses the issue of perturbation propagation and allows for dynamic observation and measurement of variation patterns of salient regions during the recognition process. Thus, SCC-CAM enhances the explainability of deep networks. Experiments on the MSTAR and OpenSARShip-1.0 datasets demonstrate that the proposed algorithm can more accurately locate salient regions. Compared with traditional methods, the algorithm in this study shows stronger explainability in terms of average confidence degradation rates, confidence ascent ratios, information content, and other evaluation metrics. This algorithm can serve as a universal method for enhancing the explainability of networks.

Abstract Radar‐based hand gesture recognition (HGR) has attracted growing interest in human–computer interaction. A rich diversity in how people perform gestures causes a large intra‐class variance, and the sample quality varies from person to person. It makes HGR more challenging to identify dynamic, complicated, and deforming hand gestures. It is urgent for the real world to explore a robust method that better identifies the gestures from non‐specified users. To address the above issues, an adaptive framework is proposed for gesture recognition, and it has two main contributions. First of all, a trajectory range Doppler map (t‐RDM) is obtained by non‐coherent accumulating for inter‐frame dependencies, and then t‐RDM is enhanced to highlight the trajectory information. Taking into account different movement patterns of the gestures, a two‐pathway convolutional neural network targeted for raw and enhanced t‐RDMs is proposed, which independently mines discriminative information from the two t‐RDMs with different salient features. Second, an adaptive individual cost (AIC) loss is proposed, aiming to establish a powerful feature representation by adaptively extracting the commonalities in variant gestures according to the sample quality. Based on a public dataset using soli radar, the proposed method is evaluated on two tasks: cross‐person recognition and cross‐scenario recognition. These two recognition modes require that the training set and the test set are mutually exclusive not only at the sample level but also at the source level. Extensive experiments demonstrate that the proposed method is superior to the existing approaches for alleviating the low recognition performance caused by gesture diversity.

Multiscale object detection in Synthetic Aperture Radar (SAR) images can locate and recognize key objects in large-scene SAR images, and it is one of the key technologies in SAR image interpretation. However, for the simultaneous detection of SAR objects with large size differences, that is, cross-scale object detection, existing object detection methods are difficult to extract the features of cross-scale objects, and also difficult to realize cross-scale object simultaneous detection. In this study, we propose a multiscale object detection method based on the Feature-Transferable Pyramid Network (FTPN) for SAR images. In the feature extraction stage, the feature migration method is used to obtain an effective mosaic of the feature images of each layer and extract feature images with different scales. Simultaneously, the void convolution method is used to increase the receptive field of feature extraction and aid the network in extracting large object features. These steps can effectively preserve the features of objects of different sizes, to realize the simultaneous detection of cross-scale objects in SAR images. The experiments based on the GaoFen-3 SAR dataset, SAR Ship Detection Dataset (SSDD), and high-resolution SSDD-2.0 show that the proposed method can detect cross-scale objects, such as airports and ships in SAR images, and the mean Average Precision (mAP) can reach 96.5% on the existing dataset, which is 8.1% higher than that of the characteristic pyramid network algorithm. Moreover, the overall performance of the proposed method is better than that of the latest YOLOv4 and other object detection algorithms.

In recent years, deep convolutional neural networks (DCNNs) have been widely used for object detection tasks in remote sensing images. However, the over-parametrization problem of DCNNs hinders their application in resource-constrained remote sensing devices. In order to solve this problem, we propose a network pruning method (named absorption pruning) to compress the remote sensing object detection network. Unlike the classical iterative three-stage pruning pipeline used in existing methods, absorption pruning is designed as a four-stage pruning pipeline that only needs to be executed once, which differentiates it from existing methods. Furthermore, the absorption pruning no longer identifies unimportant filters, as in existing pruning methods, but instead selects filters that are easy to learn. In addition, we design a method for pruning ratio adjustment based on the object characteristics in remote sensing images, which can help absorption pruning to better compress deep neural networks for remote sensing image processing. The experimental results on two typical remote sensing data sets—SSDD and RSOD—demonstrate that the absorption pruning method not only can remove 60% of the filter parameters from CenterNet101 harmlessly but also eliminate the over-fitting problem of the pre-trained network.

Synthetic aperture radar (SAR) ship recognition can obtain location and class information from SAR scene images, which is important in military and civilian fields, and has turned into a very important research focus recently. Limited by data conditions, the current research mainly includes two aspects: ship detection in SAR scene images and ship classification in SAR slice images. These two parts are not yet integrated, but it is necessary to integrate detection and classification in practical applications, although it will cause an imbalance of training samples for different classes. To solve these problems, this paper proposes a ship recognition method on the basis of a deep network to detect and classify ship targets in SAR scene images under imbalance data. First, RetinaNet is used as the backbone network of the method in this paper for the integration of ship detection and classification in SAR scene images. Then, taking into account the issue that there are high similarities among various SAR ship classes, the squeeze-and-excitation (SE) module is introduced for amplifying the difference features as well as reducing the similarity features. Finally, considering the problem of class imbalance in ship target recognition in SAR scene images, a loss function, the central focal loss (CEFL), based on depth feature aggregation is constructed to reduce the differences within classes. Based on the dataset from OpenSARShip and Sentinel-1, the results of the experiment suggest that the the proposed method is feasible and the accuracy of the proposed method is improved by 3.9 percentage points compared with the traditional RetinaNet.

Multi-scale target detection in synthetic aperture radar (SAR) images is one of the key techniques of SAR image interpretation, which is widely used in national defense and security. However, multi-scale targets include several types. For example, targets with similar-scale, large-scale, and ultra-large-scale differences coexist in SAR images. In particular, it is difficult for existing target detection methods to detect both ultra-large-scale targets and ultra-small-scale targets in SAR images, resulting in poor detection results for these two types of targets. To solve these problems, this paper proposes an ultra-high precision deep learning network (UltraHi-PrNet) to detect dense multi-scale targets. Firstly, a novel scale transfer layer is constructed to transfer the features of targets of different scales from bottom networks to top networks, ensuring that the features of ultra-small-scale, small-scale, and medium-scale targets in SAR images can be extracted more easily. Then, a novel scale expansion layer is constructed to increase the range of the receptive field of feature extraction without increasing the feature resolution, ensuring that the features of large-scale and ultra-large-scale targets in SAR images can be extracted more easily. Next, the scale expansion layers with different expansion rates are densely connected to different stages of the backbone network, and the features of the target with ultra-large-scale differences are extracted. Finally, the classification and regression of targets were achieved based on Faster R-CNN. Based on the SAR ship detection dataset (SSDD), AIR-SARShip-1.0, high-resolution SAR ship detection dataset-2.0 (high-resolution SSDD-2.0), the SAR-ship-dataset, and the Gaofen-3 airport dataset, the experimental results showed that this method can detect similar-scale, large-scale, and ultra-large-scale targets more easily. At the same time, compared with other advanced SAR target detection methods, the proposed method can achieve higher accuracy.

As a mission-critical sensor, SAR has been applied in environmental monitoring and battlefield surveillance; moreover, SAR target recognition is one of the most important applications of SAR technology. However, in practical applications, the number of samples available for training is relatively small, so the SAR target recognition can be regarded as a small sample recognition problem. One of the main directions to solve the small sample recognition problem is to realize the data augmentation. Therefore, a SAR image data augmentation method via Generative Adversarial Nets (GAN) is proposed in this paper. The method uses Wasserstein GAN with a gradient penalty (WGAN-GP) to generate new samples based on existing SAR data, which can augment the sample number in training dataset. Meanwhile, the sample selection filters are designed to extract the generated samples with high quality and specific azimuth, which can avoid the randomness of the data augmentation, and improve the quality of the newly generated training samples. The experiments based on MSTAR data show that, for three-class recognition problem, when the training sample is only 108, the proposed method can improve the recognition rate from 79% to 91.6%; and for ten-class recognition problem, when the training sample is only 360, the proposed method can improve the recognition rate from 57.48% to 79.59%. Compared with the traditional data linear generation method, the proposed method shows significant improvement on the quantity and quality of the training samples, and can effectively solve the problem of the small sample recognition.

Convolutional neural networks (CNNs) have been widely used in synthetic aperture radar (SAR) target recognition. Traditional CNNs suffer from expensive computation and high memory consumption, impeding their deployment in real-time recognition systems of SAR sensors, as these systems have low memory resources and low speed of calculation. In this paper, a micro CNN (MCNN) for real-time SAR recognition system is proposed. The proposed MCNN has only two layers, and it is compressed from a deep convolutional neural network (DCNN) with 18 layers by a novel knowledge distillation algorithm called gradual distillation. MCNN is a ternary network, and all its weights are either -1 or 1 or 0. Following a student-teacher paradigm, the DCNN is the teacher network and MCNN is its student network. The gradual distillation makes MCNN a better learning route than traditional knowledge distillation. The experiments on the MSTAR dataset show that the proposed MCNN can obtain a high recognition rate which is almost the same as the DCNN. However, compared with the DCNN, the memory footprint of the proposed MCNN is compressed 177 times, and the calculated amount is 12.8 times less, which means that the proposed MCNN can obtain better performance with the smaller network.

Face recognition (FR) with a single sample per person (SSPP) is one of the most challenging problems in computer vision. In this scenario, it is difficult to predict facial variation such as pose, illumination, and disguise due to the lack of enough training samples. Therefore, the development of the FR system with only a small number of training samples is hindered. To address this problem, this paper proposes a scheme combined transfer learning and sample expansion in feature space. First, it uses transfer learning by training a deep convolutional neural network on a common multi-sample face dataset and then applies the well-trained model to a target data set. Second, it proposes a sample expansion method in feature space called k class feature transfer (KCFT) to enrich intra-class variation information for a single-sample face feature. Compared with other expanding sample methods in the image domain, this method of expanding the samples in the feature domain is novel and easy to implement. Third, it trains a softmax classifier with expanded face features. The experimental results on ORL, FERET, and LFW face databases demonstrate the effectiveness and robustness of the proposed method for various facial variations.

The constant false alarm rate with convolution and pooling (CP-CFAR) method, which can improve the detection efficiency via GPU parallel acceleration in the airborne synthetic aperture radar (SAR) images, is proposed in this paper. The constant false alarm rate (CFAR) method is one of the most widely used methods for target detection in airborne SAR images. However, since the CFAR is performed on a pixel-by-pixel basis, the time consumption will increase rapidly with the expansion of image scene. Even if the GPU is used for acceleration, the efficiency improvement is still limited, which cannot meet the real-time processing requirements. Therefore, the CP-CFAR method for the target detection of SAR images is proposed in this paper. The convolution layer uses the horizontal and vertical Sobel operators to improve the contrast between targets and background, and the pooling layer can reduce the processing dimension of the images. The convolution and pooling layers are added before the two-parameter CFAR, which can reduce the computational elements but without losing the main feature of the original image. More importantly, compared to the traditional CFAR, the proposed CP-CFAR is more suitable for GPU acceleration, which can improve the detection efficiency significantly. Experiments on the moving and stationary target acquisition and recognition SAR images with a size of 1478 × 1784 show that, compared with the traditional cell-averaging CFAR, two-parameter CFAR and their CPU, multithread CPU, and GPU acceleration modes, the proposed CP-CFAR with GPU acceleration can obtain the best detection performance with the highest acceleration ratio, and the operation time is less than 192 ms.

In dealing with the problem of target detection in high-resolution Synthetic Aperture Radar (SAR) images, segmenting before detecting is the most commonly used approach. After the image is segmented by the superpixel method, the segmented area is usually a mixture of target and background, but the existing regional feature model does not take this into account, and cannot accurately reflect the features of the SAR image. Therefore, we propose a target detection method based on iterative outliers and recursive saliency depth. At first, we use the conditional entropy to model the features of the superpixel region, which is more in line with the actual SAR image features. Then, through iterative anomaly detection, we achieve effective background selection and detection threshold design. After that, recursing saliency depth is used to enhance the effective outliers and suppress the background false alarm to realize the correction of superpixel saliency value. Finally, the local graph model is used to optimize the detection results. Compared with Constant False Alarm Rate (CFAR) and Weighted Information Entropy (WIE) methods, the results show that our method has better performance and is more in line with the actual situation.

Target detection is an important part of synthetic aperture radar (SAR) image interpretation, and its performance directly affects the accuracy of localisation and target recognition results in the region of interest. However, the SAR system resolution largely determines the SAR image index and imaging quality, and the target detection performance is closely related to the SAR system resolution in addition to the detection method. To improve the detection performance, the correlation between system resolution and target detection is investigated in this study. Multiple SAR images are used to test a SAR system with reduced resolution obtained by multi-look processing. At the theoretical level, the effect of different resolutions on target detection performance is analysed. MSTAR images, containing image chips with centred targets of varying orientation, are used to analyse the accuracy of the results. By comparing the measured experimental data with the simulation results, it can be obtained that the detection performance is the best when the resolution is roughly equal to the target scale.

Although automatic target recognition (ATR) models based on data-driven algorithms have achieved excellent performance in recent years, the synthetic aperture radar (SAR) ATR model often suffered from performance degradation when it encountered a small sample set. In this paper, an integrated counterfactual sample generation and filtering approach is proposed to alleviate the negative influence of a small sample set. The proposed method consists of a generation component and a filtering component. First, the proposed generation component utilizes the overfitting characteristics of generative adversarial networks (GANs), which ensures the generation of counterfactual target samples. Second, the proposed filtering component is built by learning different recognition functions. In the proposed filtering component, multiple SVMs trained by different SAR target sample sets provide pseudo-labels to the other SVMs to improve the recognition rate. Then, the proposed approach improves the performance of the recognition model dynamically while it continuously generates counterfactual target samples. At the same time, counterfactual target samples that are beneficial to the ATR model are also filtered. Moreover, ablation experiments demonstrate the effectiveness of the various components of the proposed method. Experimental results based on the Moving and Stationary Target Acquisition and Recognition (MSTAR) and OpenSARship dataset also show the advantages of the proposed approach. Even though the size of the constructed training set was 14.5% of the original training set, the recognition performance of the ATR model reached 91.27% with the proposed approach.

Automatic target recognition (ATR) can obtain important information for target surveillance from Synthetic Aperture Radar (SAR) images. Thus, a direct automatic target recognition (D-ATR) method, based on a deep neural network (DNN), is proposed in this paper. To recognize targets in large-scene SAR images, the traditional methods of SAR ATR are comprised of four major steps: detection, discrimination, feature extraction, and classification. However, the recognition performance is sensitive to each step, as the processing result from each step will affect the following step. Meanwhile, these processes are independent, which means that there is still room for processing speed improvement. The proposed D-ATR method can integrate these steps as a whole system and directly recognize targets in large-scene SAR images, by encapsulating all of the computation in a single deep convolutional neural network (DCNN). Before the DCNN, a fast sliding method is proposed to partition the large image into sub-images, to avoid information loss when resizing the input images, and to avoid the target being divided into several parts. After the DCNN, non-maximum suppression between sub-images (NMSS) is performed on the results of the sub-images, to obtain an accurate result of the large-scene SAR image. Experiments on the MSTAR dataset and large-scene SAR images (with resolution 1478 × 1784) show that the proposed method can obtain a high accuracy and fast processing speed, and out-performs other methods, such as CFAR+SVM, Region-based CNN, and YOLOv2.

The classic ship detection methods in synthetic aperture radar (SAR) images suffer from an extreme variance of ship scale. Generating a set of ship proposals before detection operation can effectively alleviate the multi-scale problem. In order to construct a scale-independent proposal generator for SAR images, we suggest four characteristics of ships in SAR images and the corresponding four procedures in this paper. Based on these characteristics and procedures, we put forward a framework to explore multi-scale ship proposals. The designed framework mainly contains two stages: hierarchical grouping and proposal scoring. Firstly, we extract edges, superpixels and strong scattering components from SAR images. The ship proposals are obtained at hierarchical grouping stage by combining the strong scattering components with superpixel grouping. Considering the difference of edge density and the completeness and tightness of contour, we obtain the scores to measure the confidence that a proposal contains a ship. Finally, the ranking proposals are obtained. Extensive experiments demonstrate the effectiveness of the four procedures. Our method achieves 0.70 the average best overlap (ABO) score, 0.59 the area under the curve (AUC) score and 0.85 best recall on a challenging dataset. In addition, the recall of our method on three scale subsets are all above 0.80. Experimental results demonstrate that our algorithm outperforms the approaches previously used for SAR images.

In this paper, a new Region-based Convolutional Neural Networks (RCNN) method is proposed for target recognition in large scene synthetic aperture radar (SAR) images. To locate and recognize the targets in SAR images, there are three steps in the traditional procedure: detection, discrimination, classification and recognition. Each step is supposed to provide optimal processing results for the next step, but this is difficult to implement in real-life applications because of speckle noise and inefficient connection among these procedures. To solve this problem, the RCNN is applied to large scene SAR target recognition, which can detect the objects while recognizing their classes based on its regression method and the sharing network structure. However, size of the input images to RCNN is limited so that the classification could be accomplished, which leads to a problem that RCNN is not able to handle the large scene SAR images directly. Thus, before the RCNN, a fast sliding method is proposed to segment the scene image into sub-images with suitable size and avoid dividing targets into different sub-images. After the RCNN, candidate regions on different slices are predicted. To locate targets on large scene SAR images from these candidate regions on small slices, the Non-maximum Suppression between Regions (NMSR) is proposed, which could find the most proper candidate region among all the overlapped regions. Experiments on 1476 × 1784 simulated MSTAR images of simple scenes and complex scenes show that the proposed method can recognize all targets with the best accuracy and fastest speed, and outperform the other methods, such as constant false alarm rate (CFAR) detector + support vector machine (SVM), Visual Attention+SVM, and Sliding-RCNN.

In synthetic aperture radar (SAR) target recognition, the amount of target data increases continuously, and thus SAR automatic target recognition (ATR) systems are required to provide updated feature models in real time. Most recent SAR feature extraction methods have to use both existing and new samples to retrain a new model every time new data is acquired. However, this repeated calculation of existing samples leads to an increased computing cost. In this paper, a dynamic feature learning method called incremental nonnegative matrix factorization with L p sparse constraints (L p -INMF) is proposed as a solution to that problem. In contrast to conventional nonnegative matrix factorization (NMF) whereby existing and new samples are computed to retrain a new model, incremental NMF (INMF) computes only the new samples to update the trained model incrementally, which can improve the computing efficiency. Considering the sparse characteristics of scattering centers in SAR images, we set the updating process under a generic sparse constraint (L p ) for matrix decomposition of INMF. Thus, L p -INMF can extract sparse characteristics in SAR images. Experimental results using Moving and Stationary Target Acquisition and Recognition (MSTAR) benchmark data illustrate that the proposed L p -INMF method can not only update models with new samples more efficiently than conventional NMF, but also has a higher recognition rate than NMF and INMF.