Search

Content-based remote sensing (RS) image retrieval (CBRSIR) is a critical way to organize high-resolution RS (HRRS) images in the current big data era. The increasing volume of HRRS images from different satellites and sensors leads to more attention to the cross-source CSRSIR (CS-CBRSIR) problem. Due to the data drift, one crucial problem in CS-CBRSIR is the modality discrepancy. Most existing methods focus on finding a common feature space for various HRRS images to address this issue. In this space, their similarity relations can be measured directly to obtain the cross-source retrieval results straight. This way is feasible and reasonable, however, the specific information corresponding to HRRS images from different sources is always ignored, limiting retrieval performance. To overcome this limitation, we develop a new model for CS-CBRSIR in this paper named dual modality collaborative learning (DMCL). To fully explore the specific information from diverse HRRS images, DMCL first introduces ResNet50 as the feature extractor. Then, a common space mutual learning module is developed to map the specific features into a common space. Here, the modality discrepancy is reduced from the aspects of features and their distributions. Finally, to supplement the specific knowledge to the common features, we develop modality transformation and the dual-modality feature learning modules. Their function is to transmit the specific knowledge from different sources mutually and fuse the specific and common features adaptively. The comprehensive experiments are conducted on a public dataset. Compared with many existing methods, the behavior of our DMCL is stronger. These encouraging results for a public dataset indicate that the proposed DMCL is useful in CS-CBRSIR tasks. [ABSTRACT FROM AUTHOR]

With the explosive growth of the volume and resolution of high-resolution remote-sensing (HRRS) images, the management of them becomes a challenging task. The traditional content-based remote-sensing image retrieval (CBRSIR) technologies cannot meet what we expect due to the large volume of image archives and complex contents within HRRS images. As a successful approximate nearest neighborhood (ANN) search technique, Hash learning has received wide attention, especially when deep convolutional neural networks (DCNNs) appear. Due to DCNNs’ strong capacity of feature learning, many DCNN-based hashing methods have been proposed and achieved good performance for large-scale CBRSIR tasks. Nevertheless, their limitation is that a large of labeled training samples should be collected for training the deep models. To overcome this limitation, this article, therefore, develops a new supervised hash learning method for the large-scale HRRS CBRSIR task based on meta-learning, which could achieve well-retrieval performance with a few labeled training samples. First, taking the characteristics of HRRS into account, we develop a self-adaptive convolution (SAP-Conv) block and design a hashing net based on the block. SAP-Conv can learn robust features from HRRS images by exploring their multiscale information. Second, to enhance the generalization of the hashing net under a few labeled training samples, the hash learning is formulated in a meta-way, and we name it meta-hashing. Meta-hashing can effectively preserve the similarities between support and query set, and the similarities between samples within support set by the developed loss function. To further improve the performance of meta-hashing, we expand it to a dynamic version named dynamic-meta-hashing, in which the numbers of support and query are changeable in the training phase. Experimental results counted on the three widely used HRRS datasets demonstrate our dynamic-meta-hashing and meta-hashing can achieve promising performance in large-scale HRRS CBRSIR tasks based on a few training samples. Our source codes are available at https://github.com/TangXu-Group/Meta-hashing. [ABSTRACT FROM AUTHOR]

The content-based remote sensing image retrieval (CBRSIR) has attracted increasing attention with the number of remote sensing (RS) images growing explosively. Benefiting from the strong capacity of the deep convolutional neural network (DCNN), the performance of CBRSIR has been improved in recent years. Although great successes have been obtained, learning the RS images’ representative features and enhancing the retrieval efficiency for the large-scale CBRSIR tasks are still two challenging problems. In this article, we propose a new CBRSIR method named feature and hash (FAH) learning, which consists of a deep feature learning model (DFLM) and an adversarial hash learning model (AHLM). The DFLM aims at learning the RS images’ dense features to guarantee the retrieval precision. In the DFLM, the DCNN and the proposed feature aggregation are integrated to capture the multiscale features. Then, the discrimination of the obtained features can be highlighted by the attention map in the developed attention branch. The AHLM maps the dense features onto the compact hash codes so that the retrieval efficiency can be improved. The AHLM contains a hash learning submodel and an adversarial regularization submodel. In particular, the hash learning submodel learns the real-valued hash codes that are similarity preserved by semantic supervisions. The adversarial regularization submodel regularizes the real-valued hash codes to learn the discrete uniform distribution with possible values 0 and 1. In this way, the hash codes are coding-balanced and the quantization errors are reduced. Encouraging experimental results counted on three public benchmark data sets demonstrate that our FAH can achieve competitive performance in the CBRSIR task compared with many existing hash learning methods. [ABSTRACT FROM AUTHOR]

Image reranking is a popular postprocessing method for remote sensing image retrieval (RSIR), which aims at enhancing the initial retrieval performance. In general, it takes either users’ opinions or the relationships between images into consideration to find an optimal reranked list based on the initial retrieved results. In this paper, we present a reranking method for improving RSIR, which is named two-stage reranking (TSR). Suppose the k -nearest neighbors of a query RS image have been obtained by the initial retrieval. The first step of our TSR is to edit these neighbors using the editing scheme. A handful of informative and representative RS images are selected by the active learning algorithm, and their binary labels are provided by the users relative to the query image. Then, a binary classifier is trained using the selected RS images and their labels to classify the rest of the neighbors. Finally, both classification results and rank information in the initial retrieval results are considered to decide which neighbor should be excluded. In the next step, the remaining RS images are reranked by the proposed reranking scheme, i.e., multisimilarity fusion reranking. Both the user’s experience and image relationships are taken into account in TSR to ensure the performance of the reranking. The efficiency and the robustness of our method are validated by experiments conducted on two different types of RS images. Compared with the existing visual reranking approaches, our method achieves improved performance. [ABSTRACT FROM PUBLISHER]

This paper presents a new content-based synthetic aperture radar (SAR) image retrieval method to search out SAR image patches, which consists of two essential parts: an initial retrieval and later refined results. To obtain the proper initial retrievals, we develop a similarity measure named region-based fuzzy matching (RFM) to evaluate the similarities between SAR image patches. First, to reduce the negative influence of speckle noise, we segment the SAR image patches into brightness-texture regions at the superpixel level rather than the pixel level. Second, a multiscale edge detector is utilized to resolve the multiscale property of the SAR image patches, and then the edge regions of the SAR image patches are defined by those edge features. Third, to overcome the segmented uncertainty and the blurry boundaries, the obtained regions are described by fuzzy features. Finally, the RFM similarity between two SAR image patches is converted into the linear summation of the resemblance between different fuzzy feature sets. After we obtain the initial retrieval results, the multiple relevance feedback (MRF) scheme is proposed to refine the original ranked list. In this scheme, different relevance feedback approaches are carried out respectively, and then their results are fused to improve the initial retrieval. In addition, a new kernel function based on the RFM measure is developed for MRF. The encouraging experimental results counted on a manually constructed ground truth SAR image patch dataset demonstrate that our retrieval method is effective for SAR images compared with some existing approaches proposed in the remote sensing community [ABSTRACT FROM PUBLISHER]

A new reranking method, fusion similarity-based reranking, is proposed in this letter to improve the performance of synthetic aperture radar (SAR) image retrieval. First, the top ranked SAR images within the initial retrieval results are picked for reranking. Considering the negative influence of the speckle noise, three SAR-oriented visual features are selected to represent them. In addition, the different relevance scores corresponding to an SAR image are estimated in various modalities (i.e., different feature spaces). Second, a fusion similarity is defined under the relevance score space to measure the resemblance between two SAR images. This fusion similarity is calculated using the modal-image matrix, which is construed by the estimated scores to integrate the contributions of all modalities. Finally, an existing reranking function is adopted to rerank the SAR images with the help of the estimated scores and calculated fusion similarities. The positive experimental results demonstrate that our reranking method is effective and efficient. [ABSTRACT FROM PUBLISHER]

Based on the semantic categorization and region-based similarity measure, a novel synthetic aperture radar (SAR) image retrieval method is proposed in this paper, which is inspired by the existing content-based image retrieval (CBIR) techniques and is oriented toward the Earth observation (EO). First, due to the large sizes of SAR images, new method semantically classifies the land covers in the patch level rather than the pixel level by the classic semisupervised learning (SSL), which could reduce the workload of selecting the representative image patch and decrease the searching space in the similarity calculation component. Furthermore, to overcome the inevitable classification error, our method provides an error recovery scheme, preventing the errors produced in categorization to contaminate the retrieval results. Third, the similarity between two patches is calculated by the improved integrated region matching (IIRM) measure based on the region-based similarity measure, which fails to meet the expectation in SAR images. The proposed method can be embedded into any EO mining systems to help them complete the EO missions. After comparing the method presented in this paper to others, it is evident that our method performs more effectively than others from the CBIR aspect. [ABSTRACT FROM PUBLISHER]

Remote sensing image retrieval (RSIR), a superior content organization technique, plays an important role in the remote sensing (RS) community. With the number of RS images increases explosively, not only the retrieval precision but also the retrieval efficiency is emphasized in the large-scale RSIR scenario. Therefore, the approximate nearest neighborhood (ANN) search attracts the researchers' attention increasingly. In this paper, we propose a new hash learning method, named semi-supervised deep adversarial hashing (SDAH), to accomplish the ANN for the large-scale RSIR task. The assumption of our model is that the RS images have been represented by the proper visual features. First, a residual auto-encoder (RAE) is developed to generate the class variable and hash code. Second, two multi-layer networks are constructed to regularize the obtained latent vectors using the prior distribution. These two modules mentioned are integrated under the generator adversarial framework. Through the minimax learning, the class variable would be a one-hot-like vector while the hash code would be the binary-like vector. Finally, a specific hashing function is formulated to enhance the quality of the generated hash code. The effectiveness of the hash codes learned by our SDAH model was proved by the positive experimental results counted on three public RS image archives. Compared with the existing hash learning methods, the proposed method reaches improved performance. [ABSTRACT FROM AUTHOR]

Due to the specific characteristics and complicated contents of remote sensing (RS) images, remote sensing image retrieval (RSIR) is always an open and tough research topic in the RS community. There are two basic blocks in RSIR, including feature learning and similarity matching. In this paper, we focus on developing an effective feature learning method for RSIR. With the help of the deep learning technique, the proposed feature learning method is designed under the bag-of-words (BOW) paradigm. Thus, we name the obtained feature deep BOW (DBOW). The learning process consists of two parts, including image descriptor learning and feature construction. First, to explore the complex contents within the RS image, we extract the image descriptor in the image patch level rather than the whole image. In addition, instead of using the handcrafted feature to describe the patches, we propose the deep convolutional auto-encoder (DCAE) model to deeply learn the discriminative descriptor for the RS image. Second, the k-means algorithm is selected to generate the codebook using the obtained deep descriptors. Then, the final histogrammic DBOW features are acquired by counting the frequency of the single code word. When we get the DBOW features from the RS images, the similarities between RS images are measured using L1-norm distance. Then, the retrieval results can be acquired according to the similarity order. The encouraging experimental results counted on four public RS image archives demonstrate that our DBOW feature is effective for the RSIR task. Compared with the existing RS image features, our DBOW can achieve improved behavior on RSIR. [ABSTRACT FROM AUTHOR]