Search

While fusion of hyperspectral images (HSIs) with low spatial resolution and multispectral images (MSIs) with high spatial resolution has achieved significant success, high-quality fusion between noisy images has always been challenging. In this article, we propose a domain transform model driven by deep learning for anti-noise hyperspectral and multispectral image fusion (DTAFN). This marks the first time that wavelet decomposition theory is combined with deep learning for noise reduction in hyperspectral and MSI fusion. DTAFN initially decomposes hyperspectral and MSIs into frequency components and constructs a novel feature interaction fusion module (FIFM). This module, while using MSIs to guide the removal of noise from HSIs, also achieves the fusion of spatial and spectral information. Furthermore, it maps the fused features to a lower dimensional subspace to enhance computational efficiency. Additionally, we introduce a spatial-spectral self-attention mechanism to optimize the reconstructed frequency components using the subspace features. In the end, the wavelet inverse transform is used to reconstruct the clean fused image. It is worth noting that the extraction of the subspace is considered a process of nonlinear low-rank component extraction, which, to a certain extent, suppresses noise signals. Numerous experiments of mixed noise image fusion are carried out, and the experimental results show that DTAFN can obtain high-quality fusion results, is robust, and superior to the state-of-the-art methods.

Spectral super-resolution (SSR) aims to improve the spectral resolution of images from multispectral imagery or even red, green, blue (RGB) images. However, the majority of existing SSR methods do not fully exploit the spatial and texture features in RGB images, which would lead to the image unreal and distort of the high-frequency details in the reconstructed SSR images. In this study, a hybrid spectral and texture attention pyramid network (STANet) is proposed to reconstruct hyperspectral images (HSIs) with RGB bands of remote sensing images as input. More specifically, a learnable texture feature extraction module is proposed, aiming to make full use of the texture features in the RGB images, which are important in the subsequent spectral reconstruction. Furthermore, to better reconstruct the correlations between various spectral channels, a spatial-spectral-constrained cross-attention module is introduced. Finally, a novel spectral-texture fusion method is proposed, which successfully alleviates the problem of insufficient deep interaction among multiple deep features. On three remote sensing datasets, STANet demonstrates state-of-the-art performance, with its peak signal-to-noise ratio (PSNR) exceeding the suboptimal methods by 0.7266, 0.6724, and 0.6 dB, respectively. The results of the land-cover classification experiment using the reconstructed HSI further demonstrated the performance of the STANet algorithm.

Due to sensor design limitations and the influence of weather factors, it is currently challenging to obtain remote sensing images with high temporal, spatial, and spectral resolution. Spatial–temporal–spectral fusion aims to integrate the temporal, spatial, and spectral information from multiple sources of remote sensing images to reconstruct a remote sensing image with high temporal, spatial, and spectral resolution. Existing methods typically require at least three types of data to achieve spatial–temporal–spectral fusion. However, acquiring remote sensing data observed at the same time poses significant difficulties. The major challenge lies in effectively utilizing hyperspectral images (HSIs) with low spatial and temporal resolution and multispectral images with high temporal and spatial resolution to reconstruct remote sensing images with high temporal, spatial, and spectral resolution. To address the aforementioned issues, we propose a novel unsupervised 3-D tensor subspace decomposition network. Our method incorporates the theory of 3-D tensor subspace decomposition, utilizing a 3-D hyperspectral/multispectral tensor subspace extraction network to predict the hyperspectral tensor subspace features with low spatial resolution missing at other times (to better understand, the missing moment is defined as time 2). Subsequently, the 3-D hyperspectral tensor subspace reconstruction network is employed along with the time 2 hyperspectral tensor subspace features with low spatial resolution and the time 2 MSI to reconstruct the time 2 HSI with high spatial resolution. In the experiment, we utilize three simulated datasets and two real datasets to evaluate the fusion performance of our proposed method. The results demonstrate that our method achieves high-quality fusion results, exhibits comparable performance, and has robustness and practicality.