Spatial and frequency feature fusion using multi-scale cross attention for enhancing deepfake face detection
摘要
Deepfake technology, which enables the creation and manipulation of images in videos has attracted significant attention in the field of image forgery detection. The challenge of detecting altered content intensifies as these tools evolve and become more widely accessible. Many studies indicate that integrating frequency and spatial features from images generated by Deepfake Algorithms yields superior detection results compared to using these features in isolation, particularly when implemented within vision transformer architectures. However, a robust method for effectively combining these features remains an unresolved issue, often resulting in limited performance across varied datasets. This paper presents a framework that utilizes multi-scale cross-attention along with the Discrete Cosine Transform (DCT) and the Xception network to address this challenge. Our approach seeks to address these problems by merging spatial and frequency-based features through a multi-scale cross-attention module to incorporate diverse feature representation which enables more effective deepfake identification. We introduce a multi-scale frequency filter fusion technique with convolution based on DCT for extracting frequency details, which are then integrated with spatial information obtained from the modified Xception network. Extensive evaluations on the FaceForensics++ (FF++) and Celeb-DF datasets demonstrate that our method outperforms previous state-of-the art approaches utilizing vision transformers and CNN models, achieving an impressive F1 score of 98.91%,99.89% across two datasets.