Quality assessment of identity inpainting based on multidimensional discrimination

With the development of generation technology, inpainted faces exhibit a high degree of visual authenticity. However, the inherent biases of the network can reduce the accuracy of face recognition, and there is also a lack of a method to evaluate facial restoration at the pixel level from the perspective of identity characteristics. Therefore, a multidimensional discriminant feature extraction model (MDFEM) is proposed to assess the impact of repair features on recognition. Firstly, a frequency domain channel interaction attention module was designed. This module uses a multi-spectral channel attention representation for retaining more valuable information, and adopts adaptive 1D convolution to better capture the spectral correlation between channels. Secondly, a multi-dimensional center loss function introduces a wider distance between the inpainted face and the ground truth in the embedded space. This enables the evaluation network to distinguish between the ground truth and the inpainted face highly accurately. Finally, the multi-dimensional discriminant model is used to extract the identity features of the inpainted face and the ground truth, and the quality of the inpainted model is evaluated by quantifying the differences. Based on the face image quality score and feature extraction model, a regression model is built to extract the gradient change of identity feature inpainting quality at each pixel point, thereby generating an evaluation interpretation map. The validity of the framework was verified through multiple experimental results on a benchmark dataset. The results show that our network can objectively evaluate the restoration effect of different models on facial identity. Based on this evaluation, the network can then select high-quality restoration models to assist with face recognition tasks.