Search

How to recover geometric transformations is one of the most challenging issues in image registration. To alleviate the effect of large geometric distortion in multimodal remote sensing image registration, a scale and rotate transform prediction net is proposed in this paper. First, to reduce the scale between the reference and sensed images, the image scale regression module is constructed via CNN feature extraction and FFT correlation, and the scale of sensed image can be recovered roughly. Second, the rotation estimate module is developed for predicting the rotation angles between the reference and the scale-recovered images. Finally, to obtain the accurate registration results, LoFTR is employed to match the geometric-recovered images. The proposed registration network was evaluated on GoogleEarth, HRMS, VIS-NIR and UAV datasets with contrast differences and geometric distortions. The experimental results show that the number of correct matches of our model reached 74.6%, and the RMSE of the registration results achieved 1.236, which is superior to the related methods. [ABSTRACT FROM AUTHOR]

The accurate detection and extraction of roads using remote sensing technology are crucial to the development of the transportation industry and intelligent perception tasks. Recently, in view of the advantages of CNNs in feature extraction, its related road extraction methods have been proposed successively. However, due to the limitation of kernel size, they perform less effectively at capturing long-range information and global context, which are crucial for road targets distributed over long distances and highly structured. To deal with this problem, a novel model named RoadFormer with a Swin Transformer as the backbone is developed in this paper. Firstly, to extract long-range information effectively, a Swin Transformer multi-scale encoder is adopted in our model. Secondly, to enhance the feature representation capability of the model, we design an innovative bottleneck module, in which the spatial and channel separable convolution is employed to obtain fine-grained and globe features, and then a dilated block is connected after the spatial convolution module to capture more integrated road structures. Finally, a lightweight decoder consisting of transposed convolution and skip connection generates the final extraction results. Extensive experimental results confirm the advantages of RoadFormer on the Deepglobe and Massachusetts datasets. The comparative results of visualization and quantification demonstrate that our model outperforms comparable methods. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion is to combine the information of thermal radiation and detailed texture from the two images into one informative fused image. Recently, deep learning methods have been widely applied in this task; however, those methods usually fuse multiple extracted features with the same fusion strategy, which ignores the differences in the representation of these features, resulting in the loss of information in the fusion process. To address this issue, we propose a novel method named multi-modal feature self-adaptive transformer (MFST) to preserve more significant information about the source images. Firstly, multi-modal features are extracted from the input images by a convolutional neural network (CNN). Then, these features are fused by the focal transformer blocks that can be trained through an adaptive fusion strategy according to the characteristics of different features. Finally, the fused features and saliency information of the infrared image are considered to obtain the fused image. The proposed fusion framework is evaluated on TNO, LLVIP, and FLIR datasets with various scenes. Experimental results demonstrate that our method outperforms several state-of-the-art methods in terms of subjective and objective evaluation. [ABSTRACT FROM AUTHOR]

Fine‐grained visual classification is challenging due to the inherently subtle intra‐class object variations. To solve this issue, a novel framework named channel attention and progressive multi‐granularity training network, is proposed. It first exploits meaningful feature maps through the channel attention module and captures multi‐granularity features by the progressive multi‐granularity training module. For each feature map, the channel attention module is proposed to explore channel‐wise correlation. This allows the model to re‐weight the channels of the feature map according to the impact of their semantic information on performance. Furthermore, the progressive multi‐granularity training module is introduced to fuse features cross multi‐granularity. And the fused features pay more attention to the subtle differences between images. The model can be trained efficiently in an end‐to‐end manner without bounding box or part annotations. Finally, comprehensive experiments are conducted to show that the method achieves state‐of‐the‐art performances on the CUB‐200‐2011, Stanford Cars, and FGVC‐Aircraft datasets. Ablation studies demonstrate the effectiveness of each part in our module. [ABSTRACT FROM AUTHOR]

To deal with the registration of infrared and visible image with significant difference in contrast and large geometric distortion in electro-optical pod, a robust and fast registration method by using common structural features is proposed in this paper. First, to adjust the scales of input images, a geometric transformation model is built based on parameters of the cameras. Then, a global structural feature extraction strategy is developed by using phase congruency and significance ranking space. Those features are not only invariant to the scale and contrast but also embody the structural of the input images maximally and uniformly. Finally, to enhance the robustness to contrast, blurring, and incompleteness of structures, an adaptive feature matching method with kernelized correlation filter is presented. Experimental results demonstrate that the proposed method has better performance than several state-of-the-art methods in robustness and precision, and also confirm its validity in the real environment. [ABSTRACT FROM AUTHOR]

Multimodal remote sensing image registration is a prerequisite for comprehensive application of remote sensing image data. However, inconsistent imaging environment and conditions often lead to obvious geometric deformations and significant contrast differences between multimodal remote sensing images, which makes the common feature extraction extremely difficult, resulting in their registration still being a challenging task. To address this issue, a robust local statistics-based registration framework is proposed, and the constructed descriptors are invariant to contrast changes and geometric transformations induced by imaging conditions. Firstly, maximum phase congruency of local frequency information is performed by optimizing the control parameters. Then, salient feature points are located according to the phase congruency response map. Subsequently, the geometric and contrast invariant descriptors are constructed based on a joint local frequency information map that combines Log-Gabor filter responses over multiple scales and orientations. Finally, image matching is achieved by finding the corresponding descriptors; image registration is further completed by calculating the transformation between the corresponding feature points. The proposed registration framework was evaluated on four different multimodal image datasets with varying degrees of contrast differences and geometric deformations. Experimental results demonstrated that our method outperformed several state-of-the-art methods in terms of robustness and precision, confirming its effectiveness. [ABSTRACT FROM AUTHOR]

In this paper, we propose a manifold geometry based projective nonnegative matrix factorization linear dimensionality reduction method, called neighborhood preserving orthogonal projective nonnegative matrix factorization (NPOPNMF), for feature extraction of hyperspectral image. By adding constraints on projective nonnegative matrix factorization (PNMF) that each data point can be represented as a linear combination of its neighbors, NPOPNMF preserves local neighborhood geometrical structure of hyperspectral data in the reduced space, and overcomes the Euclidean limitation of PNMF. The metric structure of original high-dimensional hyperspectral data space is preserved due to the orthogonality of projection matrix. NPOPNMF can be performed in either supervised or unsupervised mode according to the construction of adjacency graph and it can improve the discriminant performance of PNMF. Theoretical analysis and experimental results on hyperspectral data sets demonstrate that the proposed method is an effective and promising method for hyperspectral image feature extraction. [ABSTRACT FROM PUBLISHER]