A Comparative Study of Hybrid Deepfake Detection Models Using EfficientNet–LSTM and ViT–TCM
摘要
The new developments in machine learning, deep learning, and artificial intelligence are the driving forces behind the extremely realistic deepfake video productions. Deepfakes technology can be used for face swapping, as well as expression manipulation and also for other uses based on the abilities of the state-of-the-art models including autoencoders and Generative Adversarial Networks (GANs). It has rendered detection of original content and imitation content much more challenging. Deepfakes can cause chaos in the shape of spreading of false information, blackmailing and political manipulation. Due to this, internet security and public trust have never been more crucial than they are at present. Two hybrid deepfake detectors are proposed as a countermeasure to address this new threat. The first one learns spatial features via EfficientNet and temporal anomalies with the use of Long Short-Term Memory (LSTM) networks. The second is a Vision Transformer (ViT) with an added Temporal Contrastive Module (TCM) to learn spatiotemporal features in parallel through self-attention mechanisms. The two models are used for motion pattern analysis and visual artifact analysis for detection robustness and accuracy enhancement. The experimental results conducted on the benchmark data Deepfake Detection Challenge (DFDC) demonstrated the insensitiveness of the two methods towards identifying genuine and forged videos. The predicted models add to the research contributions towards the area of deepfake detection with scalable efficient solutions to computer security.