Spatial-Frequency Complementary Transformer and Multi-Scale Attention Refinement for Image Inpainting
摘要
The ability to capture long-range contextual information is crucial for image inpainting networks when dealing with complex semantic scenes and large areas of damage. Therefore, Transformers and frequency-based learning methods have gained wide attention due to their powerful global modeling capabilities. However, these methods only learn global features within a single domain, ignoring the complementarity of modeling global dependencies in both spatial and frequency domains. To address this issue, a spatial-frequency complementary Transformer is proposed, which combines the advantages of spatial global learning for global structure generation and frequency global learning for repetitive texture inpainting. This solution enhances the network’s performance in complex semantic scenes and large areas of damage. For spatial global learning, deformable convolution v4 (DCNv4) is introduced to image inpainting for the first time. By utilizing its sparse and efficient global modeling capabilities, a more lightweight inpainting is realized, and the occurrence of blurring and artifacts in the inpainting results is minimized. Additionally, a multi-scale attention refinement module is proposed to enhance the network’s ability to inpaint local detail. Extensive experiments on three benchmark datasets demonstrate that the proposed method outperforms existing advanced inpainting models while maintaining lower parameter counts and a faster inference speed.