Deepfake technology has emerged as a double-edged sword enabling creative innovation while posing serious risks to digital security and public trust. Powered by advanced generative models like GANs and diffusion networks, deepfakes have become increasingly difficult to detect. This paper explores the evolution of these techniques with a focus on audio-visual forgeries and their detection. We highlight lip-speech synchronization analysis as a powerful yet underutilized tool, capable of revealing subtle desynchronization artifacts, such as phoneme-lip mismatches and unnatural motion, often missed by traditional visual methods. A comprehensive review of detection strategies across visual, audio-visual, and text-visual modalities is presented. Benchmark datasets (FaceForensics++, DFDC, FakeAVCeleb, WildDeepfake) are examined. Finally, we discuss future research directions. By focusing on lip–speech synchronization inconsistencies, an area where generative models still struggle to achieve perfection, this work identifies promising approaches for enhancing the overall effectiveness of deepfake detection.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Survey on the Integration of Multimodal Techniques for Visual Artifact Detection: Advancing Lip–Speech Synchronization and Facial Recognition

  • Pamela Kirui,
  • Bing Zhou

摘要

Deepfake technology has emerged as a double-edged sword enabling creative innovation while posing serious risks to digital security and public trust. Powered by advanced generative models like GANs and diffusion networks, deepfakes have become increasingly difficult to detect. This paper explores the evolution of these techniques with a focus on audio-visual forgeries and their detection. We highlight lip-speech synchronization analysis as a powerful yet underutilized tool, capable of revealing subtle desynchronization artifacts, such as phoneme-lip mismatches and unnatural motion, often missed by traditional visual methods. A comprehensive review of detection strategies across visual, audio-visual, and text-visual modalities is presented. Benchmark datasets (FaceForensics++, DFDC, FakeAVCeleb, WildDeepfake) are examined. Finally, we discuss future research directions. By focusing on lip–speech synchronization inconsistencies, an area where generative models still struggle to achieve perfection, this work identifies promising approaches for enhancing the overall effectiveness of deepfake detection.