Selection and guidance: high-dimensional identity consistency preservation for face inpainting

This paper primarily investigates the issue of identity inconsistency resulting from facial inpainting. In the current facial image inpainting process, most low-frequency information is learned directly from the uncovered areas, while high-frequency information, including texture details, is generated from the dataset. The averageness of low-frequency features and the unreliability of high-frequency features result in identity deviations in the inpainted faces. To address this problem, a two-stage generative model is proposed, which includes structural prior guidance and multi-dimensional consistency guidance. Firstly, a parsing inpainting module was designed to predict structural information. Subsequently, a multi-dimensional information guidance module was constructed, which, based on semantic consistency guidance, extracts low-frequency confidence information of the inpainted image to suppress the averageness of facial inpainting. Then, frequency domain information is extracted from the inpainted image, combined with a facial recognition network, and the frequency domain identity feature consistency loss is calculated to aid in the inpainting of high-frequency information, including identity features, thus achieving identity consistency in facial inpainting. Experiments were conducted on the CelebAMask-HQ dataset and the FFHQ dataset. Both qualitative and quantitative experimental results demonstrate that the performance of PI-MFO-GAN surpasses that of state-of-the-art models. Furthermore, to assess the consistency of restored facial identity information with that of real faces, the identity features of the faces generated by the restoration network were extracted, and the distribution of these features was used as an indicator of restoration quality. Comparative results indicate that PI-MFO-GAN is better at reconstructing facial identity features, especially when dealing with critical area losses, further validating the model’s superiority and effectiveness in the field of facial restoration.