Search

Real-world data are long-tailed, the lack of tail samples leads to a significant limitation in the generalization ability of the model. Although numerous approaches of class re-balancing perform well for moderate class imbalance problems, additional knowledge needs to be introduced to help the tail class recover the underlying true distribution when the observed distribution from a few tail samples does not represent its true distribution properly, thus allowing the model to learn valuable information outside the observed domain. In this work, we propose to leverage the geometric information of the feature distribution of the well-represented head class to guide the model to learn the underlying distribution of the tail class. Specifically, we first systematically define the geometry of the feature distribution and the similarity measures between the geometries, and discover four phenomena regarding the relationship between the geometries of different feature distributions. Then, based on four phenomena, feature uncertainty representation is proposed to perturb the tail features by utilizing the geometry of the head class feature distribution. It aims to make the perturbed features cover the underlying distribution of the tail class as much as possible, thus improving the model's generalization performance in the test domain. Finally, we design a three-stage training scheme enabling feature uncertainty modeling to be successfully applied. Experiments on CIFAR-10/100-LT, ImageNet-LT, and iNaturalist2018 show that our proposed approach outperforms other similar methods on most metrics. In addition, the experimental phenomena we discovered are able to provide new perspectives and theoretical foundations for subsequent studies. The code will be available at https://github.com/mayanbiao1234/Geometric-Prior [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) image classification is a pixel-wise issue, which has become increasingly prevalent in recent years. As a variant of the Convolutional Neural Network (CNN), the Fully Convolutional Network (FCN), which is designed for pixel-to-pixel tasks, has obtained enormous success in semantic segmentation. Therefore, effectively using the FCN model combined with polarimetric characteristics for PolSAR image classification is quite promising. This paper proposes a novel FCN model by adopting complex-valued domain stacked-dilated convolution (CV-SDFCN). Firstly, a stacked-dilated convolution layer with different dilation rates is constructed to capture multi-scale features of PolSAR image; meanwhile, the sharing weight is employed to reduce the calculation burden. Unfortunately, the labeled training samples of PolSAR image are usually limited. Then, the encoder–decoder structure of the original FCN is reconstructed with a U-net model. Finally, in view of the significance of the phase information for PolSAR images, the proposed model is trained in the complex-valued domain rather than the real-valued domain. The experiment results show that the classification performance of the proposed method is better than several state-of-the-art PolSAR image classification methods. [ABSTRACT FROM AUTHOR]

This work delves into unveiling the singularity issue latent in global attention-based Transformers. Empirical and theoretical analyses elucidate that interrelationships among token channels lead to singularities, impeding the training of attention weights. Concretely, the similar neighbor pixels within image patches can form intercorrelated channels after being flattened. Images that one color dominates can possess correlated channels. Furthermore, the fixed global connection architecture retains correlation relationships, contributing to the persistence of singularities. High singularity risks reducing Transformers' performance and robustness. Based on the singularity analysis, we propose the Token Singularity Removal (TSR) strategy. It incorporates the Dual-Tree Complex Wavelet Transform (DTCWT) stem and Feature Decorrelation (FD) loss, aiming to encourage Transformers to learn tokens with unrelated channels and eliminate singularities. Experimental validation across various image classification datasets and corruption image data sets demonstrate improved accuracy and robustness of Transformers utilizing the TSR strategy. Our code is publicly available at https://github.com/wdanc/TSR. [ABSTRACT FROM AUTHOR]

Recently, many state-of-the-art results for remote sensing image scene classification have been achieved by convolutional neural networks (CNNs) due to their large learning capability. However, in the forward process of CNNs, the high-frequency/texture features are gradually blurred with hierarchical downsampling and convolution operations. High-frequency features are important to capture the diversity within a class and the similarity between classes. For example, the line features are crucial to distinguish a tennis court from a basketball court. For tennis court in different scenes, the highlight of line features can effectively avoid the influence of diverse backgrounds. As a consequence, we propose a Laplacian high-frequency convolutional block (LHCB) based on CNN to extract useful high-frequency features by trainable Laplacian operator. To propagate high-frequency features, we embed LHCB into the existing CNN structures and obtain LHNet. In LHNet, there are two pathways. The original CNN architecture can be taken as the low-frequency pathway, and we propose a high-frequency pathway (HFP) based on LHCB that propagates the residual high-frequency features blurred in each low-frequency layer. Considering that the high-frequency features usually show large variance between images of the same class, we propose a new objective for HFP to enhance the intraclass similarity of high-frequency features. The final objective function is obtained by combining the new objective and the baseline classification objective. Numerous experiments on three public available remote sensing image scene classification datasets, the Northwestern Polytechnical University-Remote Sensing Image Scene Classification (NWPU-RESISC45), Aerial Image Dataset (AID), and the University of California at Merced (UC Merced), demonstrate the superior performance of the proposed method. [ABSTRACT FROM AUTHOR]

Remote sensing object tracking (RSOT) is a novel and challenging problem due to the negative effects of weak features and background noise. In this article, from the perspective of attention-focus deep learning, we propose a joint Siamese attention-aware network (JSANet) for efficient remote sensing tracking which contains both the self-attention and cross-attention modules. First, the self-attention modules we propose emphasize the interdependent channel-wise coefficient via channel attention and conduct corresponding space transformation of spatial domain information with spatial attention. Second, the cross-attention is designed to aggregate rich contextual interdependencies between the Siamese branches via channel attention and excavate association produces reliable correspondence with spatial attention. In addition, a composite feature combine strategy is designed to fuse multiple attention features. Experimental results on the Jilin-1 satellite video datasets demonstrate that the proposed JSANet achieves state-of-the-art performance in terms of precision and success rate, demonstrating the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Contrastive self-supervised learning (CSSL) has achieved promising results in extracting visual features from unlabeled data. Most of the current CSSL methods are used to learn global image features with low-resolution that are not suitable or efficient for pixel-level tasks. In this paper, we propose a coarse-to-fine CSSL framework based on a novel contrasting strategy to address this problem. It consists of two stages, one for encoder pre-training to learn global features and the other for decoder pre-training to derive local features. Firstly, the novel contrasting strategy takes advantage of the spatial structure and semantic meaning of different regions and provides more cues to learn than that relying only on data augmentation. Specifically, a positive pair is built from two nearby patches sampled along the direction of the texture if they fall into the same cluster. A negative pair is generated from different clusters. When the novel contrasting strategy is applied to the coarse-to-fine CSSL framework, global and local features are learned successively by forcing the positive pair close to each other and the negative pair apart in an embedding space. Secondly, a discriminant constraint is incorporated into the per-pixel classification model to maximize the inter-class distance. It makes the classification model more competent at distinguishing between different categories that have similar appearance. Finally, the proposed method is validated on four SAR images for land-cover classification with limited labeled data and substantially improves the experimental results. The effectiveness of the proposed method is demonstrated in pixel-level tasks after comparison with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, a novel unsupervised change detection method called adaptive Contourlet fusion clustering based on adaptive Contourlet fusion and fast non-local clustering is proposed for multi-temporal synthetic aperture radar (SAR) images. A binary image indicating changed regions is generated by a novel fuzzy clustering algorithm from a Contourlet fused difference image. Contourlet fusion uses complementary information from different types of difference images. For unchanged regions, the details should be restrained while highlighted for changed regions. Different fusion rules are designed for low frequency band and high frequency directional bands of Contourlet coefficients. Then a fast non-local clustering algorithm (FNLC) is proposed to classify the fused image to generate changed and unchanged regions. In order to reduce the impact of noise while preserve details of changed regions, not only local but also non-local information are incorporated into the FNLC in a fuzzy way. Experiments on both small and large scale datasets demonstrate the state-of-the-art performance of the proposed method in real applications. [ABSTRACT FROM AUTHOR]

This article proposes a practical end-to-end neural network framework to detect tiny moving vehicles in satellite videos with low imaging quality. Some instability factors, such as illumination changes, motion blurs, and low contrast to the cluttered background, make it difficult to distinguish true objects from noise and other point-shaped distractors. Moving vehicle detection in satellite videos can be carried out based on background subtraction or frame differencing. However, these methods are prone to produce lots of false alarms and miss many positive targets. Appearance-based detection can be an alternative but is not well-suited since classifier models are of weak discriminative power for the vehicles in top view at such low resolution. This article addresses these issues by integrating motion information from adjacent frames to facilitate the extraction of semantic features and incorporating the transformer to refine the features for key points estimation and scale prediction. Our proposed model can well identify the actual moving targets and suppress interference from stationary targets or background. The experiments and evaluations using satellite videos show that the proposed approach can accurately locate the targets under weak feature attributes and improve the detection performance in complex scenarios. [ABSTRACT FROM AUTHOR]

Semantic segmentation is an important yet unsolved problem in aerial scenes understanding. One of the major challenges is the intense variations of scenes and object scales. In this article, we propose a novel multi-scale aware-relation network (MANet) to tackle this problem in remote sensing. Inspired by the process of human perception of multi-scale (MS) information, we explore discriminative and diverse MS representations. For discriminative MS representations, we propose an inter-class and intra-class region refinement (IIRR) method to reduce feature redundancy caused by fusion. IIRR utilizes the refinement maps with intra-class and inter-class scale variation to guide MS fine-grained features. Then, we propose multi-scale collaborative learning (MCL) to enhance the diversity of MS feature representations. The MCL constrains the diversity of MS feature network parameters to obtain diverse information. Also, the segmentation results are rectified according to the dispersion of the multilevel network predictions. In this way, MANet can learn MS features by collaboratively exploiting the correlation among different scales. Extensive experiments on image and video datasets, which have large-scale variations, have demonstrated the effectiveness of our proposed MANet. [ABSTRACT FROM AUTHOR]

The excellent performance of graph convolutional networks (GCNs) on non-Euclidean data has drawn widespread attention from the hyperspectral image classification (HSIC) community, where the predefined graph (including node modeling and adjacency matrix calculation) plays a key role. However, existing GCN-based methods rely on manual efforts in constructing and updating graphs, and the superpixel-based node features lack high-level semantics. In this article, we propose an automatic graph learning convolutional network (Auto-GCN), which unifies the graph learning and HSIC in a “network-in-network” manner. Specifically, the graph is employed to model the interaction of the high-order tensors. Considering the powerful learning and representation capabilities of convolutional neural networks (CNNs), the semisupervised Siamese network (SiamNet) is embedded into GCNs and HSIC networks to accomplish the automatic learning and dynamic updating of the graph. GCNs further encode and infer the dynamic graph, and then, the learnable graph reprojection matrix is designed to assign graph representations to pixels. The dynamic graph serves the HSIC task during forward propagation, while the HSIC task continuously corrects the graph during backward propagation. Therefore, the “automatic” of the proposed Auto-GCN is not only reflected in the fact that the graph representation is designed and updated by an end-to-end network but is also HSIC task-oriented. The experimental results show that the proposed Auto-GCN outperforms other state-of-the-art methods on four publicly available hyperspectral datasets. [ABSTRACT FROM AUTHOR]

Many objective tracking methods are based on the framework of correlation filtering (CF) due to its high efficiency. In this paper, we propose a $l_{2}$ -norm based sparse response regularization term to restrain unexpected crests in response for CF framework. CF trackers learn online to regress the region of interest into a Gaussian response. However, due to the uncertain transformations of tracked object, there are many unexpected crests in the response map. When the response of tracked object is corrupted by other crests, the tracker will lost the object. Therefore, the sparse response is used to increase the robustness to transformations of tracked object. Since the novel term is directly incorporated into the objective function of the CF framework, it can be used to improve the performance of many methods which are based on this framework. Moreover, from the solutions we derive, the new method will not increase the computational complexity. Through the experiments on benchmarks of OTB-100, TempleColor, VOT2016 and VOT2017, the proposed regularization term can improve the tracking performance of various CF trackers, including those based on standard discriminative CF framework and those based on context-aware CF framework. We also embed the sparse response regularization term in the state-of-the-art integrated tracker MCCT to test its generalization performance. Although MCCT is an expert integrated tracker and owns an exquisite algorithm for selecting experts, the experimental results show that our method can still improve its long-term tracking performance without increasing computational complexity. [ABSTRACT FROM AUTHOR]

Supervised change detection methods always face a big challenge that the current scene (target domain) is fully unlabeled. In remote sensing, it is common that we have sufficient labels in another scene (source domain) with a different but related data distribution. In this article, we try to detect changes in the target domain with the help of the prior knowledge learned from multiple source domains. To achieve this goal, we propose a change detection framework based on selective adversarial adaptation. The adaptation between multisource and target domains is fulfilled by two domain discriminators. First, the first domain discriminator regards each scene as an individual domain and is designed for identifying the domain to which each input sample belongs. According to the output of the first domain discriminator, a subset of important samples is selected from multisource domains to train a deep neural network (DNN)-based change detection model. As a result, not only the positive transfer is enhanced but also the negative transfer is alleviated. Second, as for the second domain discriminator, all the selected samples are thought from one domain. Adversarial learning is introduced to align the distributions of the selected source samples and the target ones. Consequently, it further adapts the knowledge of change from the source domain to the target one. At the fine-tuning stage, target samples with reliable labels and the selected source ones are used to jointly fine-tune the change detection model. As the target domain is fully unlabeled, homogeneity- and boundary-based strategies are exploited to make the pseudolabels from a preclassification map reliable. The proposed method is evaluated on three SAR and two optical data sets, and the experimental results have demonstrated its effectiveness and superiority. [ABSTRACT FROM AUTHOR]

In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral–spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral–spatial feature learning. Third, a sequential spectral–spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC). [ABSTRACT FROM AUTHOR]

• A supervised classification based MRF model is proposed for depth map up-sampling. • Low-resolution depth segmentation is used to supervise color image classification. • Classification result is further introduced in the design of MRF energy function. • The designed MRF energy function is optimized with a gradient descent algorithm. • Comparisons with the state-of-the-art show the superiority of the proposed SC-MRF model. In this paper, a Supervised Classification assisted Markov Random Field (SC-MRF) model is proposed for generating high-quality up-sampled depth maps. The proposed model aims to reduce depth bleeding and depth confusion artifacts that can be produced at boundary regions of the up-sampled depth maps. In the proposed model, segmentation of low-resolution (LR) depth map is first used to supervise the classification of corresponding high-resolution (HR) color image. With this supervised classification, not only can the depth edges be retained, but redundant textures in the HR color image can be omitted. The classification result is then introduced into the design of a MRF energy function, and the final up-sampled depth map is obtained by optimizing this energy function with the gradient descent algorithm. For simplicity, classical K -means clustering is adopted to segment the LR depth map into several classes, and a feature-based K -nearest neighbour (K -NN) method is utilized for the supervised classification. With the proposed SC-MRF model, interaction between depths of different classes will be strongly suppressed, meaning depth edges are well preserved. Comparisons with the state-of-the-art demonstrate the strong performance of the proposed method both visually and by quantitative evaluation. [ABSTRACT FROM AUTHOR]

One way to understand the network function and analyze the network structure is to find the communities of the network accurately. Now, there are many works about designing algorithms for community detection. Most community detection algorithms are based on modularity optimization. However, these methods not only have disadvantages in computational complexity, but also have the problem of resolution restriction. Designing a community detection algorithm that is fast and effective remains a challenge in the field. We attempt to solve the community detection problem in a new perspective in this paper, believing that the assumption used to solve the link prediction problem is useful for the problem of community detection. By using the similarity between modules of the network, we propose a new method to extract the community structure in this paper. Our algorithm consists of three steps. First, we initialize a community partition based on the distribution of the node degree; second, we calculate the similarity between different communities, where the similarity is the index to describe the closeness of the different communities. We assume that the much closer the two different communities are, the greater the likelihood of being divided together; finally, merge the pairs of communities which has the highest similarity value as possible as we can and stop when the condition is not satisfied. Because the convergence of our algorithm is very fast in the process of merging, we find that our method has advantages both in the computational complexity and in the accuracy when compared with other six classical algorithms. Moreover, we design a new measure to describe how difficulty the network division is. [ABSTRACT FROM AUTHOR]

Sparse representation (SR) method has the advantages of good category distinguishing performance, noise robustness, and data adaptiveness. In this article, a multi-feature weighted sparse graph (MWSG) is presented for synthetic aperture radar (SAR) image analysis. First, multiple types of features are extracted to fully describe the characteristics of SAR image. Then, multiple SRs of samples in multiple feature spaces are obtained by solving a weighted joint SR model, in which the weight is the Gaussian kernel distance among samples. Moreover, a new fusion mechanism is given to integrate multiple weighted SRs, which aims to eliminate the negative influence of the singular data, so the MWSG is obtained. Afterward, the brief steps of the SAR image segmentation and semisupervised classification based on MWSG are stated. A series of experiments on the simulated and real SAR images shows that the MWSG has better performance than other existing relevant methods. [ABSTRACT FROM AUTHOR]

Supervised deep neural networks (DNNs) have been extensively used in diverse tasks. Generally, training such DNNs with superior performance requires a large amount of labeled data. However, it is time-consuming and expensive to manually label the data, especially for tasks in remote sensing, e.g., change detection. The situation motivates us to resort to the existing related images with labels, from which the concept of change can be adapted to new images. However, the distributions of the related labeled images (source domain) and unlabeled new images (target domain) are similar but not identical. It impedes a change detection model learned from source domains being well applied to the target domain. In this paper, we propose a transferred deep learning-based change detection framework to solve this problem. It consists of pretraining and fine-tuning stages. In the pretraining process, we propose two tasks to be learned simultaneously, namely, change detection for the source domain with labels and reconstruction of the unlabeled target data. The auxiliary task aims to reconstruct the difference image (DI) for the target domain. DI is an effective feature, such that the auxiliary task is of much relevance to change detection. The lower layers are shared between these two tasks in the training process. It mitigates the distribution discrepancy between the source and target domains and makes the concept of change from the source domain adapt to the target domain. In addition, we evaluate three modes of the U-net architecture to merge the information for a pair of patches. To fine-tune the change detection network (CDN) for the target domain, two strategies are exploited to select the pixels that have a high possibility of being correctly classified by an unsupervised approach. The proposed method demonstrates an excellent capacity for adapting the concept of change from the source domain to the target domain. It outperforms the state-of-the-art change detection methods via experimental results on real remote sensing data sets. [ABSTRACT FROM AUTHOR]

Recently, many deep learning methods are applied with the spatial information to learn features for polarimetric synthetic aperture radar (PolSAR) image classification. However, without considering the multi-scale information, the classification performance of these methods are limited. Hence, this paper proposes a multi-scale feature extraction method based on stacked sparse autoencoder (SSAE), named the multi-scale SSAE (MS-SSAE), to improve the classification performance. This method extracts the deep multi-scale features by a two-stage framework. In the first stage, the SSAE uses training data at different scales to extract the multi-scale features. Then, a 1-D average pooling strategy is proposed to reduce the feature dimensionality at the second stage. Therefore, the MS-SSAE can capture discriminative multi-scale features. The experimental results certify that the MS-SSAE can not only improve the classification accuracy, but also remain the details in the image. [ABSTRACT FROM AUTHOR]

The approaches for analyzing the polarimetric scattering matrix of polarimetric synthetic aperture radar (PolSAR) data have always been the focus of PolSAR image classification. Generally, the polarization coherent matrix and the covariance matrix obtained by the polarimetric scattering matrix are used as the main research object to extract features. In this paper, we focus on the original polarimetric scattering matrix and propose a polarimetric scattering coding way to deal with polarimetric scattering matrix and obtain a close complete feature. This encoding mode can also maintain polarimetric information of scattering matrix completely. At the same time, in view of this encoding way, we design a corresponding classification algorithm based on the convolution network to combine this feature. Based on the polarimetric scattering coding and convolution neural network, the polarimetric convolutional network is proposed to classify PolSAR images by making full use of polarimetric information. We perform the experiments on the PolSAR images acquired by AIRSAR and RADARSAT-2 to verify the proposed method. The experimental results demonstrate that the proposed method get better results and has huge potential for PolSAR data classification. Source code for polarimetric scattering coding is available at https://github.com/liuxuvip/Polarimetric-Scattering-Coding. [ABSTRACT FROM AUTHOR]

Polarimetric synthetic aperture radar (PolSAR) image classification plays an important role in remote sensing image processing. In recent years, stacked auto-encoder (SAE) has obtained a series of excellent results in PolSAR image classification. The recently proposed projective dictionary pair learning (DPL) model takes both accuracy and time consumption into consideration, and another recently proposed semicoupled dictionary learning (SCDL) model gives a new way to fit different features. Based on the SAE, DPL, and SCDL models, we propose a novel semicoupled projective DPL method with SAE (SAE-SDPL) for PolSAR image classification. Our method can get the classification result efficiently and correctly and meanwhile giving a new method to fit different features. In this paper, three PolSAR images are used to test the performance of SAE-SDPL. Compared with some state-of-the-art methods, our method obtains excellent results in PolSAR image classification. [ABSTRACT FROM AUTHOR]

Abstract In this paper, a fast unsupervised deep fusion framework for change detection of multitemporal synthetic aperture radar (SAR) images is presented. It mainly aim at generating a difference image (DI) in the feature learning procedure by stacked auto-encoders (SAEs). Stacked auto-encoders, as one kind of deep neural network, can learn feature maps that retain the structural information but suppress the noise in the SAR images, which will be beneficial for DI generation. Compared with shallow network, the proposed framework can extract more available features, and be favorable for getting better change results. Different with other common deep neural networks, our proposed method does not need labeled data to train the network. In addition, we find a subset of the entire samples that appropriately represent the whole dataset to speed up the training of the deep neural network without under-fitting. Moreover, we design a fusion network structure that can combine ratio operator based method to ensure that the representations of higher layers are better than that of the lower ones. To summarize, the main contribution of our work lies in using of deep fusion network for generation of DI in a fast and unsupervised way. Experiments on four real SAR images confirm that our network performs better than traditional ratio methods and convolutional neural network. [ABSTRACT FROM AUTHOR]

A novel deep learning method for face sketch synthesis is proposed in this work. It builds a lightweight neural network which contains two convolutional layers, a pooling layer and a multilayer perceptron convolutional layer to learn a mapping from face photos to sketches. Unlike conventional example-based methods which need to solve complex optimization problems, the proposed method only computes convolution and pooling operations, hence significantly improves the synthesis efficiency. Besides, due to the global feature extraction of the convolutional layer, it achieves more continuous and faithful facial contours. Experiments on three benchmark datasets demonstrate that compared with several state-of-the-arts, the proposed method achieves highly competitive numerical results and is more robust to illumination and expression variations. [ABSTRACT FROM AUTHOR]

In this paper, a new random subspace based ensemble sparse representation (RS_ESR) algorithm is proposed, where the random subspace is introduced into sparse representation model. For high-dimensional data, the random subspace method can not only reduce dimension of data but also make full use of effective information of data. It is not like traditional dimensionality reduction methods that may lose some information of original data. Additionally, a joint sparse representation model is emloyed to obtain the sparse representation of a sample set in the low dimensional random subspace. Then the sparse representations in multiple random subspaces are integrated as an ensemble sparse representation. Moreover, the obtained RS_ESR is applied in classical clustering and semi-supervised classification. The experimental results on different real-world data sets show the superiority of RS_ESR over traditional methods. [ABSTRACT FROM AUTHOR]

Image reranking is a popular postprocessing method for remote sensing image retrieval (RSIR), which aims at enhancing the initial retrieval performance. In general, it takes either users’ opinions or the relationships between images into consideration to find an optimal reranked list based on the initial retrieved results. In this paper, we present a reranking method for improving RSIR, which is named two-stage reranking (TSR). Suppose the k -nearest neighbors of a query RS image have been obtained by the initial retrieval. The first step of our TSR is to edit these neighbors using the editing scheme. A handful of informative and representative RS images are selected by the active learning algorithm, and their binary labels are provided by the users relative to the query image. Then, a binary classifier is trained using the selected RS images and their labels to classify the rest of the neighbors. Finally, both classification results and rank information in the initial retrieval results are considered to decide which neighbor should be excluded. In the next step, the remaining RS images are reranked by the proposed reranking scheme, i.e., multisimilarity fusion reranking. Both the user’s experience and image relationships are taken into account in TSR to ensure the performance of the reranking. The efficiency and the robustness of our method are validated by experiments conducted on two different types of RS images. Compared with the existing visual reranking approaches, our method achieves improved performance. [ABSTRACT FROM PUBLISHER]

Most of the existing spatial-spectral-based hyperspectral image classification (HSIC) methods mainly extract the spatial-spectral information by combining the pixels in a small neighborhood or aggregating the statistical and morphological characteristics. However, those strategies can only generate shallow appearance features with limited representative ability for classes with high interclass similarity and spatial diversity and therefore reduce the classification accuracy. To this end, we present a novel HSIC framework, named deep multiscale spatial-spectral feature extraction algorithm, which focuses on learning effective discriminant features for HSIC. First, the well pretrained deep fully convolutional network based on VGG-verydeep-16 is introduced to excavate the potential deep multiscale spatial structural information in the proposed hyperspectral imaging framework. Then, the spectral feature and the deep multiscale spatial feature are fused by adopting the weighted fusion method. Finally, the fusion feature is put into a generic classifier to obtain the pixelwise classification. Compared with the existing spectral-spatial-based classification techniques, the proposed method provides the state-of-the-art performance and is much more effective, especially for images with high nonlinear distribution and spatial diversity. [ABSTRACT FROM PUBLISHER]

An approach of weighted Wishart distance learning, shorted for W2-based distance learning, is proposed for polarimetric synthetic aperture radar (PolSAR) image classification. It aims to adjust the Wishart distance by enhancing discrimination as well as exploiting spatial information. The proposed distance learning keeps samples within the same category close and separates samples from the different classes far apart. It is effectively implemented by solving a linear programming. Input of W2-based distance learning is called weighted Wishart feature, which is designed specifically for PolSAR data to describe the Wishart distribution, achieve regional consistency, and reduce speckle noise. Weight is calculated according to an adaptive window, where homogeneous samples are derived based on a connected region and extracted edge information. With this feature, W2-based distance learning is a whole scheme to adjust the Wishart distance. Furthermore, our experiments with benchmark data sets suggest that the proposed scheme provides both improved performance in terms of visual effect and classification accuracy. The achieved overall accuracy is better by more than 7% compared to other state-of-art methods. [ABSTRACT FROM PUBLISHER]

The multispectral (MS) and the panchromatic (PAN) images represent complementary and synergistic spatial spectral information, how to make optimal use of the advantages of them has become a hot research topic. This paper proposes a selectable Transformer and Gist CNN network (STGC-Net). It designs a subspace similar recombination module (SSR-Module) based on non-negative matrix factorization (NMF) and the self-attention mechanism for feature decomposition. This can alleviate the redundant information of multi-modal data and extract their own singular and common features. Considering that the MS and the PAN images exhibit different advantageous properties, a selectable self-attention spectral feature extraction module (S 3 FE-Module) and a multi-stream Gist spatial feature extraction module (MGSFE-Module) are proposed for the different singular features. The former can refine the Transformer's input and simultaneously characterize the sequence information between channels for the MS image. The latter introduces the positional relationship between local features while extracting spatial features for the PAN image, thereby improving the accuracy of scene classification. Experimental results indicate that the proposed method performs better than the other methods. The relevant code of this paper is provided at: https://github.com/ru-willow/ST-GC-Net. • A subspace similar recombination module is proposed. • The selectable self-attention spectral feature extraction module is designed. • The multi-stream Gist spatial feature extraction module is introduced. [ABSTRACT FROM AUTHOR]

Recently, very high resolution (VHR) panchromatic and multispectral (MS) remote-sensing images can be acquired easily. However, it is still a challenging task to fuse and classify these VHR images. Generally, there are two ways for the fusion and classification of panchromatic and MS images. One way is to use a panchromatic image to sharpen an MS image, and then classify a pan-sharpened MS image. Another way is to extract features from panchromatic and MS images, respectively, and then combine these features for classification. In this paper, we propose a superpixel-based multiple local convolution neural network (SML-CNN) model for panchromatic and MS images classification. In order to reduce the amount of input data for the CNN, we extend simple linear iterative clustering algorithm for segmenting MS images and generating superpixels. Superpixels are taken as the basic analysis unit instead of pixels. To make full advantage of the spatial-spectral and environment information of superpixels, a superpixel-based multiple local regions joint representation method is proposed. Then, an SML-CNN model is established to extract an efficient joint feature representation. A softmax layer is used to classify these features learned by multiple local CNN into different categories. Finally, in order to eliminate the adverse effects on the classification results within and between superpixels, we propose a multi-information modification strategy that combines the detailed information and semantic information to improve the classification performance. Experiments on the classification of Vancouver and Xi’an panchromatic and MS image data sets have demonstrated the effectiveness of the proposed approach. [ABSTRACT FROM PUBLISHER]

Link prediction is a ubiquitous application in many fields which uses partially observed information to predict absence or presence of links between node pairs. The group evolving study provides reasonable explanations on the behaviors of nodes, relations between nodes and community formation in a network. Possible events in group evolution include continuing, growing, splitting, forming and so on. The changes discovered in networks are to some extent the result of these events. In this work, we present a group evolving-based characterization of node’s behavioral patterns, and via which we can estimate the probability they tend to interact. In general, the primary aim of this paper is to offer a minimal toy model to detect missing links based on evolution of groups and give a simpler explanation on the rationality of the model. We first introduce perturbations into networks to obtain stable cluster structures, and the stable clusters determine the stability of each node. Then fluctuations, another node behavior, are assumed by the participation of each node to its own belonging group. Finally, we demonstrate that such characteristics allow us to predict link existence and propose a model for link prediction which outperforms many classical methods with a decreasing computational time in large scales. Encouraging experimental results obtained on real networks show that our approach can effectively predict missing links in network, and even when nearly 40% of the edges are missing, it also retains stationary performance. [ABSTRACT FROM AUTHOR]

A novel visual object tracking method based on consistency-constrained nonnegative coding (CNC) is proposed in this paper. For the purpose of computational efficiency, superpixels are first extracted from each observed video frame. And then CNC is performed based on those obtained superpixels, where the locality on manifold is preserved by enforcing the temporal and spatial smoothness. The coding result is achieved via an iterative update scheme, which is proved to converge. The proposed method enhances the coding stability and makes the tracker more robust for object tracking. The tracking performance has been evaluated based on ten challenging benchmark sequences involving drastic motion, partial or severe occlusions, large variation in pose, and illumination variation. The experimental results demonstrate the superior performance of our method in comparison with ten state-of-art trackers. [ABSTRACT FROM PUBLISHER]

Complex network research has attracted lots of attention in both academic community and various application fields. Complex network clustering, as one of the key issues in complex network, explores the internal organization of the nodes in a complex network. The discrete particle swarm optimization strategy has been successfully proposed for network clustering, while the existing method works with weak robust. In this paper, we model the task of complex network clustering as a multi-objective optimization problem and solve the problem with the quantum mechanism based particle swarm optimization algorithm, which is a parallel algorithm. To our knowledge, this is the first attempt to apply the quantum mechanism based discrete particle swarm optimization algorithm into network clustering. In addition, the non-dominant sorting selection operation is employed for individual replacement. Consequently, a quantum-behaved discrete multi-objective particle swarm optimization algorithm is proposed for complex network clustering. The experimental results demonstrate that the proposed algorithm performs effectively and achieves competitive performance with the state-of-the-art approaches on the extension of Girvan and Newman benchmarks and real-world networks, especially on large-scale networks. [ABSTRACT FROM AUTHOR]

Recently sparse representation (SR) based clustering has attracted a growing interests in the field of image processing and pattern recognition. Since the SR technology has favorable category distinguishing ability, we introduce it into the fuzzy clustering in this paper, and propose a new clustering algorithm, called sparse learning based fuzzy c-means (SL_FCM). Firstly, to reduce the computation complexity of the SR based FCM method, most energy of discriminant feature obtained by solving a SR model is reserved and the remainder is discarded. By this way, some redundant information (i.e. the correlation among samples of different classes) in the discriminant feature can be also removed, which can improve the clustering quality. Furthermore, to further enhance the clustering performance, the position information of valid values in discriminant feature is also used to re-define the distance between sample and clustering center in SL_FCM. The weighted distance in SL_FCM can enhance the similarity of the samples from the same class and the difference of the samples of different classes, thus to improve the clustering result. In addition, as the dimension of stored discriminant feature of each sample is different, we use set operations to formulate the distance and cluster center in SL_FCM. The comparisons on several datasets and images demonstrate that SL_FCM performs better than other state-of-art methods with higher accuracy, while keeps low spatial and computational complexity, especially for the large scale dataset and image. [ABSTRACT FROM AUTHOR]

Discriminant analysis (DA) is a well-known dimensionality reduction tool in pattern classification. With enough efficient labeled samples, the optimal projections could be found by maximizing the between-class scatter variance meanwhile minimizing the within-class scatter variance. However, the acquisition of label information is difficult in practice. So, semi-supervised discriminant analysis has attracted much attention in recent years, where both few labeled samples and many unlabeled samples are utilized during learning process. Sparse graph learned by sparse representation contains local structure information about data and is widely employed in dimensionality reduction. In this paper, semi-supervised double sparse graphs (sDSG) based dimensionality reduction is proposed, which considers both the positive and negative structure relationship of data points by using double sparse graphs. Aiming to explore the discriminant information among unlabeled samples, joint k nearest neighbor selection strategy is proposed to select pseudo-labeled samples which contain some precise discriminant information. In the following procedures, the data subset consisting of labeled samples and pseudo-labeled samples are used instead of the original data. Based on two different criterions, two sDSG based discriminant analysis methods are designed and denoted by sDSG-dDA (distance-based DA) and sDSG-rDA (reconstruction-based DA), which also use different strategies to reduce the effect of pseudo-labels’ inaccuracy. Finally, the experimental results both on UCI datasets and hyperspectral images validate the effectiveness and advantage of the proposed methods compared with some classical dimensionality reduction methods. [ABSTRACT FROM AUTHOR]

Classification plays an important role in many fields of synthetic aperture radar (SAR) image understanding and interpretation. Many scholars have devoted to design features to characterize the content of SAR images. However, it is still a challenge to design discriminative and robust features for SAR image classification. Recently, the deep learning has attracted much attention and has been successfully applied in many fields of computer vision. In this paper, a novel feature learning approach that is called discriminant deep belief network (DisDBN) is proposed to learning high-level features for SAR image classification, in which the discriminant features are learned by combining ensemble learning with a deep belief network in an unsupervised manner. Firstly, some subsets of SAR image patches are selected and marked with pseudo-labels to train weak classifiers. Secondly, the specific SAR image patch is characterized by a set of projection vectors that are obtained by projecting the SAR image patch onto each weak decision space spanned by each weak classifier. Finally, the discriminant features are generated by feeding the projection vectors to a DBN for SAR image classification. Experimental results demonstrate that better classification performance can be achieved by the proposed approach than the other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

In this paper, a novel unsupervised saliency-guided synthetic aperture radar (SAR) image change detection method is proposed. Salient areas of an image always are discriminative and different from other areas, which make them easily noticed. The strong visual contrast of local areas makes saliency suitable to guide the change detection of SAR images, where exists a difference between the two images. By applying the saliency extraction on an initial difference map obtained via the log ratio operator, a saliency map can be obtained in which most of the changed areas are included and the false changed pixels raised by speckle noises are well neglected, simultaneously. Then, by thresholding the saliency map, most of the interest regions can be preserved and further used to extract regions from the initial SAR images to generate difference image. The principal component analysis (PCA) method is used to extract features from local patches to incorporate the spatial information and reduce the influence of isolated pixels. Finally, k -means clustering is employed to obtain the change map on the extracted features, which are clustered into two classes: changed areas and unchanged areas. Experimental results on five real and two simulated SAR image data sets have demonstrated the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Spectral–spatial-based classification methods demonstrate satisfying performance for hyperspectral imagery (HSI) classification. In this letter, in order to make full use of spectral and contexture information with simultaneously considering within-class information, we propose a new algorithm for HSI classification based on within-class collaborative representation and column generation (CG) strategy. The proposed accelerated homogeneous patch mean kernel (HPMK) can automatically assign a homogeneous patch for the target sample and represent the similarities between training set and assigned homogeneous patch in kernel feature space based on CG strategy. Further, for including intra-class information and improve classification efficiency, within-class collaborative representation classification (WCRC) is incorporated into new feature space to enhance the classification performance. Experiments on two real HSI data sets demonstrate that the proposed algorithm presents satisfying results in terms of classification accuracy and efficiency. [ABSTRACT FROM PUBLISHER]

Inspired by the compressive sensing (CS) theory, a new classifier based on compressed dictionary and Least Squares Support Vector Machine (LS-SVM) is proposed to deal with large scale problems. The coefficients of support vectors can be recovered from a few measurements if LS-SVM is approximated to sparse structure. Using the known Cholesky decomposition, we approximate the given kernel matrix to represent the coefficients of support vectors sparsely by a low-rank matrix that we have used as a dictionary. The proposed measurement matrix being coupled with the dictionary forms a compressed dictionary that proves to satisfy the restricted isometry property (RIP). Our classifier has the quality of low storage and computational complexity, high degree of sparsity and information preservation. Experiments on benchmark data sets show that our classifier has positive performance. [ABSTRACT FROM AUTHOR]

Clustering learning and classification learning are two major tasks in pattern recognition. The traditional hybrid clustering and classification algorithms handle them in a sequential way rather than a simultaneous way. Fortunately, multiobjective optimization provides a way to solve this problem. In this paper, an algorithm that learns simultaneous clustering and classification adaptively via multiobjective evolutionary algorithm is proposed. The main idea of this paper is to optimize two objective functions which represent fuzzy cluster connectedness and classification error rate to achieve the goal of simultaneous learning. Firstly, we adopt a graph based representation scheme to encode so that it can generate a set of solutions with different number of clusters in a single run. Then the relationship between clustering and classification is built via the Bayesian theory during the optimization process. The quality of clustering and classification is measured by the objective functions and the feedback drawn from both aspects is used to guide the mutation. At last, a set of nondominated solutions are generated, from which the final Pareto optimal solution is selected by using Adjusted Rand Index. The results on synthetic datasets and real-life datasets demonstrate the rationality and effectiveness of the proposed algorithm. Furthermore, we apply the proposed algorithm to image segmentation including texture images and synthetic aperture radar images, the experimental results show the superiority of the proposed algorithm compared with other five algorithms. [ABSTRACT FROM AUTHOR]

In hyperspectral images, band selection plays a crucial role for land-cover classification. Multiple kernel learning (MKL) is a popular feature selection method by selecting the relevant features and classifying the images simultaneously. Unfortunately, a large number of spectral bands in hyperspectral images result in excessive kernels, which limit the application of MKL. To address this problem, a novel MKL method based on discriminative kernel clustering (DKC) is proposed. In the proposed method, a discriminative kernel alignment (KA) (DKA) is defined. Traditional KA measures kernel similarity independently of the current classification task. Compared with KA, DKA measures the similarity of discriminative information by introducing the comparison of intraclass and interclass similarities. It can evaluate both kernel redundancy and kernel synergy for classification. Then, DKA-based affinity-propagation clustering is devised to reduce the kernel scale and retain the kernels having high discrimination and low redundancy for classification. Additionally, an analysis of necessity for DKC in hyperspectral band selection is provided by empirical Rademacher complexity. Experimental results on several hyperspectral images demonstrate the effectiveness of the proposed band selection method in terms of classification performance and computation efficiency. [ABSTRACT FROM PUBLISHER]

It remains one of the most challenging tasks to distinguish different terrain materials from a single SAR image. With the increase of ground resolution, it allows us to model the SAR image directly by exploiting spatial structures and texture information that are extracted by several machine learning approaches. In this paper, a novel feature learning approach is proposed to capture discriminant features of high-resolution SAR images. In the first stage, a weighted discriminant filter bank is learned from some labeled SAR image patches to generate low-level features. Then, the locality constraint is introduced to produce the high-level features in both the encoding and the spatial pooling procedure. In this work, the superpixels are employed as the basic operational units instead of the pixels for terrain classification. With some learned domain patterns which are learned from all of the high-level features of each pixel, the superpixel is characterized by a hyper-feature. In the last stage, a linear-kernel support vector machine is utilized to classify all of these hyper-features which are generated for each superpixel. The experimental results show a better classification performance of the proposed approach than several available state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Deformable ConvNets achieve excellent performance due to learnable spatial sampling locations. However, the spatial support may deviate from the region of interest, according to observations of its behavior. More Deformable ConvNets reduce abnormal behavior mainly by relying on a modulation mechanism, but they do not identify the reason for the behavior. In this work, we find that the unusual behavior is caused by the interaction between offsets, and we present a novel method named Entire Deformable ConvNets for reducing its influence. This updated version helps the spatial support better focus on pertinent image content. In the new method, deformable convolution is replaced with a new module, namely, entire deformable convolution. The new module avoids mutual interference by changing the manner of offset allocation. Using this simple pattern, entire deformable convolution inherits all the advantages of deformable convolution and achieves better performance. Furthermore, this strategy is also applicable to More Deformable ConvNets. Similarly, Entire More Deformable ConvNets contain a new module called entire modulated deformable convolution. Extensive experiments on PASCAL VOC and MS COCO benchmarks show that our methods realize excellent improvement over the original versions. [ABSTRACT FROM AUTHOR]

Many works have been recently presented to extract efficient features for automatic target recognition of synthetic aperture radar (SAR) images. However, they are limited in the discriminative ability of similar targets and robustness to the remarkable speckle noises and background clutters existed in images. In this paper, we propose a Complementary Spatial Pyramid Coding (CSPC) approach in the framework of Spatial Pyramid Matching (SPM). Both the coding coefficients and coding residuals are explored to develop more discriminative and robust features for representing SAR images. Multiple codebooks are first built from some training example images, where each codebook is formulated by local features of a certain class of samples. Then multiple sparse coding models are developed to derive features of a target under these codebooks. Additionally, these coding residuals are further sparsely encoded in the same way to that of local features. Finally, the encoded local features and the residual features are pooled according to spatial pyramid respectively, then concatenated to form the complementary features for the subsequent classification. The experiments on Moving and Stationary Target Acquisition and Recognition (MSTAR) public database verify the superior performance of the proposed method to some related approaches. [ABSTRACT FROM AUTHOR]