Integration Model of Deep Forgery Video Detection Based on rPPG and Spatiotemporal Signal

With the development of deep learning, video forgery technology is becoming more and more mature, which may bring security risk and the further development of forgery detection is urgently needed. Most of the existing forgery detection technique are based on artifacts and detail features, which are greatly affected by the resolution, and its generalization ability needs to be improved. In this paper, a multi-modal fusion forgery detection model architecture based on the inherent biological signals and spatio-temporal signals in videos is proposed. In the process of forgery detection, the model first recognizes the face of the video. Subsequently, video frame extraction and rPPG signal extraction based on Green channel are performed on the video, respectively. These two data are later input into 3D and 2D convolutional neural networks to train the base learner respectively. Finally, the integration model is constructed based on stacking strategy. Sufficient experiments show that the established fusion model can cope well with low-resolution cases and has good generalization performance, achieving 93.38% and 91.57% accuracy on FF++ c23 and celeb-DF-v2 data set, respectively.