Search

A significant challenge of spectral reconstruction (SR) task is the lower performance reconstructed in foreground regions compared to background regions, which can be attributed to the marked difference in diversity of objects and disparity of adjacent scene characteristics. Moreover, the reconstruction of edge regions is often fraught with substantial errors due to the transitional nature of these regions, an issue conventional single convolutional neural networks (CNNs) and transformers struggle to handle. To address these challenges, we introduce the residual mask in cascaded convolutional transformer (RC2T) to iteratively improve the reconstruction of hyperspectral images (HSIs). Specifically, we propose a residual-predict mask generator (RMG) to generate a residual mask that retains band properties to separate feature with different complexities. Meanwhile, to achieve band expansion of mask features within the autoencoder, we approximate it to a Markov process and exploit the multistage spectral-aware Markov transfer (MMT) for its lightweight implementation. Next, we introduce the parallel convolutional multihead self-attention module (PSM), in which CNN runs parallel to the transformer to handle simple and complex features separately. Additionally, the residual mask loss function uses the established relationship between complexity of feature and reconstruction accuracy to generate residual mask in a self-supervised manner for providing complex high-frequency prior. We have validated our approach using three published datasets (NTIRE 2020 “Clean” track, NTIRE 2022, and CAVE). Additionally, we also conducted experiments with the proposed method on remote sensing dataset grss_dfc_2018 and a satellite-borne remote sensing dataset, achieving optimal performance. The experimental results demonstrate that our RC2T method is state-of-the-art (SOTA) in the field of SR.