Multi-modal Deepfake Detection: A Hybrid Approach Utilizing Video, Audio, and Textual Features
摘要
Deepfake media, which belongs to the creation of manipulated fake content, poses serious risks like spreading false information, invading privacy, and identity theft. Many current methods for detecting deepfakes focus on just one type of data, such as analyzing either video or audio. This makes them less effective against more advanced fakes that combine multiple forms of manipulation. In this study, we introduce a new approach that uses video, audio, and text together to improve detection. Our method analyzes video using 3D Convolutional Neural Networks (3D CNNs). The proposed scheme processes audio and text using Recurrent Neural Networks (RNNs) and combines these features with advanced techniques to detect inconsistencies across all three types of data. Tested on the DFDC, FaceForensics + + , and FakeAVCeleb datasets, our approach achieved an accuracy of 98.15, 97.73, and 97.77%, respectively, proving its ability to effectively identify deepfakes through a multi-modal approach.