Efficient Pyramidal GAN for Versatile Missing Data Reconstruction in Remote Sensing Images

Missing data reconstruction is a classical, yet challenging problem in remote sensing (RS) image processing due to the complex atmospheric environment and variability of satellite sensors. Most of the contemporary reconstruction methods either handle only one specific task or require supplementary data, while the single input for multitask reconstruction has not been explored yet. In this article, we propose a novel generative adversarial network-based unified framework for missing RS image reconstruction, which is capable of various reconstruction tasks given only single-source data as input. Specifically, we first propose a mask extraction network (MEN) to obtain a united soft mask, which represents the intrinsic prior under various scenarios and indicates not only location, but also context information. The versatility of mask extraction enables the multitask reconstruction of RS images. Besides, we propose a unified inpainting network (UIN) to repair diverse degraded images. Being specifically tailored for RS images, dilated pyramidal convolutions (DPCs) and an attention fusion mechanism (AFM) are introduced to further improve the feature extraction ability and thus exhaustly leveraging the single-input information. Extensive experiments demonstrate the uncompromising performance of the proposed method against state-of-the-art multiinput methods on diverse missing restoration. Moreover, further exploration shows the potential of the proposed method to utilize joint spatial-spectral-temporal information, which is evaluated to outperform existing competitors on remote sense images.