Cloud removal using SAR and optical images via attention mechanism-based GAN.

Clouds often appear in remote sensing images, which seriously affect the application of remote sensing images. Therefore, cloud removal is an important preprocessing process in remote sensing image applications. In this paper, we propose a generative adversarial network-based cloud removal method for optical remote sensing images with the assistance of synthetic aperture radar (SAR) images. Our model is an end-to-end model, which consists of a translation module, an attention module, a generator, and a discriminator. We introduce the attention mechanism to accurately locate the cloud regions. With the obtained attention maps as the prior information, the proposed method can remove the clouds while preserving the cloud-free regions. In addition, we include the structural similarity index (SSIM) and the attention penalty in the loss function to improve the performance of the proposed method. Numerical experiments show that the proposed model provides improved cloud removal performance compared with the state-of-the-art methods. • An end-to-end GAN-based network for thick cloud removal in optical images. • SAR image is converted into auxiliary optical image by the translation module. • Attention map obtained to accurately locate regions contaminated by clouds. • The attention mechanism makes the generator focus on recovering the cloudy regions. • Proposal greatly improves cloud removal accuracy compared with competing methods. [ABSTRACT FROM AUTHOR]