Contrastive local constraint for irregular image reconstruction and editability

GAN inversion aims to invert a real image back into the latent space of a pre-trained GAN model, showing great potential in image reconstruction and editing. However, existing methods suffer from a loss of detail and texture when inverting irregular images with complex structural features. This is because they only focus on global reconstruction while ignoring local constraints. In this work, we propose a contrastive local constraint inversion framework, named CLC, to improve reconstruction on irregular images without decreasing the editability. Specifically, we introduce patch-wise multilayer contrastive learning (MCL) to strengthen the local correspondence in the iterative correction of the global reconstruction process, aiming to reconstruct detailed information of the target image. Meanwhile, to further refine this local correspondence between the target and reconstructed images, we push the network to learn more distinguishing representations by using hard negative samples obtained from a negative sample generator. This allows the model to further pull in the distance of locally corresponding positions in feature space, achieving better detail and texture reconstruction. Extensive experiments on irregular images demonstrate that our CLC exhibits considerable performance, significantly improving the reconstruction and editing quality of irregular images. Code is available at https://github.com/harmonic-lqw/CLC.