Change-aware multi-temporal cloud removal
摘要
Cloud coverage varies significantly across different acquisitions, enabling the temporal complementarity of observations to recover surface details obscured by clouds. However, existing multi-temporal cloud removal methods fail to account for inconsistencies in data collected at different times (e.g., seasonal variations, human activities, meteorological events, etc.), resulting in severe performance degradation when surface changes occur. In this paper, we propose a change-aware multi-temporal cloud removal method, CA-MTCR, which utilizes SAR images, capable of penetrating clouds to capture terrain information beneath, to detect changes and optimize the usage of multi-temporal data. Specifically, we measure the similarity between SAR features to derive the change information for each auxiliary time step relative to the target time step, and then integrate this information into a spatial attention mechanism that highlights the regions requiring emphasis, thereby refining the multi-temporal fusion process. Furthermore, we propose a region-selective optical encoder to aggregate non-local information exclusively from partial cloud-free regions, enhancing the fidelity of the reconstructed images. Extensive evaluation demonstrates that the proposed algorithm outperforms state-of-the-art cloud removal algorithms with a gain of about 2.0 dB in terms of PSNR on the SEN12MS-CR-TS dataset and 0.8 dB on the Allclear dataset, and exhibits robustness to variations in the length of auxiliary time steps and their offset relative to the target time step.