Data-Driven Techniques for Speech and Multimodal Deepfake Detection
摘要
Recent advancements in deep learning and generative models have simplified the creation and manipulation of synthetic media. Today, even inexperienced users can produce highly realistic content with minimal effort. While these technologies offer exciting opportunities, they also pose serious risks. When misused, they can facilitate fraud, blackmail, and the spread of disinformation. An example of this phenomenon is deepfakes, synthetic multimedia content generated through deep learning techniques that depict individuals in actions and behaviors that do not belong to them. Using only a few images or an audio recording of a target victim, an attacker can utilize deepfake technology to produce synthetic data that impersonates the victim and discredits their reputation. Detecting such content is essential to prevent misuse. This chapter addresses the problem of deepfake detection, beginning with a monomodal focus on synthetic speech and then extending the analysis to audio-video multimodal deepfakes. We propose multiple detection methods and discuss broader solutions to related challenges. We view this work as a foundational but meaningful step forward in multimedia forensics. While the results are encouraging, the landscape is evolving rapidly, with emerging threats demanding continuous innovation. We believe our findings can support future research and help strengthen defenses against synthetic media misuse.