Bid-Swires and Deep-AttCTrans: A spatio-temporal attention-based framework for detecting deepfake videos
摘要
This research suggests a novel deepfake detection method that takes these concerns into account and incorporates an attention mechanism. Initially, convert the films to frames. Next, pick the keyframes and apply adaptive temporal frame filtering (Ad-tempFilter) to pre-process the frames, which minimizes noise and blurred images. An enhanced attention scale fusion YOLOV9 model (Ad-ScYOLO) performs face detection after pre-processing. After identification, a Bidirectional Swish gated residual network (Bid-Swires) extracts the essential collection of important features from detected face images, including local facial areas and facial landmarks. A Dual-Stream Attention (DSA) module is used to extract spatiotemporal features. In order to detect deepfake video, the features were finally fused and fed into the Deep-AttCTrans model, which uses a deep fake convolutional transformer encoder to assist with dual attention. To help human reviewers reliably confirm the judgment, an Explainable AI-Grad-CAM has also been employed to highlight facial pattern inconsistencies. Two datasets, LAV-DF, and Face Forensic++, were used to evaluate the experimental performance of the proposed model. The outcomes showed improved accuracy, with values of 99.54% and 99.43%, respectively.