Mining Temporal Inconsistency with 3D Face Model for Deepfake Video Detection

Recently, the abuse of face swapping technique including Deepfakes has garnered widespread attention. Facial manipulation techniques arise serious problem, including information confusion and erosion of trust. Previous methods for detecting deepfakes have primarily focused on identifying forged traces in independent frames or from two-dimensional face. As a result, the challenge of detecting forged traces from video sequence and higher dimensions has emerged as a critical research area. In this paper, we approach face swapping detection from a novel perspective by converting face frames into three-dimensional space. We argue that the Euclidean distance between landmarks in 2D faces and 3D facial reprojections, along with the shape and texture feature of the 3D face model, serve as clues to discern authenticity. Our framework incorporates a Recurrent Neural Network (RNN) to effectively exploit temporal features from videos. Extensive experiments conducted on several datasets demonstrate the effectiveness of our method. Particularly in cross-manipulation experiments, our approach outperforms state-of-the-art competitors, highlighting its potential as a robust solution for detecting deepfakes.