MFI3D: masked face identification with 3D face reconstruction and deep learning

Masked face identification (MFI) aims to identify human faces that are obscured by masks or occluding objects. Today’s common practice of wearing masks imposes significant barriers to facial recognition systems that rely on an unobstructed view of the face. To address this challenge, this paper introduces MFI 3D-based deep learning model (MFI3D) to identify occluded faces. Three main architectural components provide a set of discriminant features to decide the face identity, which are synthetic face masking, face unmasking with generator and discriminator, and 3D face reconstruction. The MFI3D pipeline begins by creating a syntactic mask for the face image, simulating real-world situations where the face is partially covered by the mask. Then, effective face detection is applied using a generator that learns to generate unmasked images that are indistinguishable from true unmasked images, and a discriminator that learns to distinguish real images from fake images. As a result, the MFI model can learn to reconstruct facial features from partially masked faces. The use of 3D face reconstruction techniques to generate a detailed model of faces leverages 3D geometry to extract facial features that are not visible in 2D image, providing a superior visual facial representation. Finally, the reconstructed face is matched against a collection of known people to determine their identity. Extensive experiments were conducted on facial datasets reconstructed orderly to build a diverse collection of 3D reconstructed facial images with a benchmarking ground truth. The experimental results show the superiority of the proposed MFI3D model in identifying people with occluded faces, achieving a precision of 83.50%.