InpTRGAN: Bridging Local and Global Contexts with a Hybrid CNN-Transformer Network for High-Fidelity Image Inpainting
摘要
Image inpainting requires the completion of damaged areas with content that is both plausible and seamlessly integrated with the surrounding environment. Currently, convolutional neural network (CNN)-based inpainting models often produce artifacts that fail to blend coherently with their surroundings, particularly when dealing with extensively corrupted regions. This issue stems from the intrinsic locality and weight-sharing properties of convolutions, which limit CNNs’ ability to explicitly capture relationships between corrupted content and distant contextual information. In contrast, models based on Transformer architectures excel at modeling long-range dependencies but frequently sacrifice local detail due to their inherent lack of locality. An effective inpainting model should therefore combine the strengths of both CNNs and Transformers. To achieve this, we propose a new Image inpainting transformed GAN-based method (InpTRGAN) to generate a reconstruction image from a corrupted image. The proposed method consists of a Generative Adversarial Network-based model, while the generator network is composed of a transformer-based encoder and an upsampling-based decoder. The proposed method is evaluated and compared with existing image inpainting approaches using the CelebA-HQ dataset, demonstrating superior performance metrics and image quality compared to state-of-the-art methods.