With the rapid progress in neural speech synthesis and voice conversion, audio deepfakes have become a growing concern. Most existing research focuses on detecting clips that are either fully real or fully fake. However, real-world threats are often more subtle, where fake segments are quietly inserted into real audio, making them much harder to detect. To tackle this, we present a new multilingual dataset that simulates these realistic “half-truth” scenarios. By alternating short segments of real speech from the VCTK corpus with deepfake audio generated using YourTTS and X-TTS, we create interleaved clips that better reflect how deepfakes might be used in practice. We evaluate three deep learning models CNN, Bi-LSTM, and ResNet-18 across four versions of our dataset, each increasing in complexity. On the simpler versions ( \(V_1\) and \(V_2\) ), CNN performs the best, achieving up to 84% accuracy when detecting fully real or fake clips. But in the more challenging interleaved versions ( \(V_3\) and \(V_4\) ), ResNet-18 shows stronger performance, handling mixed audio more effectively. These results highlight the limits of standard binary classifiers and the need for more advanced, segment-aware detection systems. Our dataset and benchmarks aim to push forward research in building practical and robust solutions for detecting deepfake audio in the wild.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MHTVDD: Multilingual Half-Truth Voices for Audio Deepfake Detection

  • Satyam R. Tiwari,
  • Jayraj S. Lakkad,
  • Hemant A. Patil

摘要

With the rapid progress in neural speech synthesis and voice conversion, audio deepfakes have become a growing concern. Most existing research focuses on detecting clips that are either fully real or fully fake. However, real-world threats are often more subtle, where fake segments are quietly inserted into real audio, making them much harder to detect. To tackle this, we present a new multilingual dataset that simulates these realistic “half-truth” scenarios. By alternating short segments of real speech from the VCTK corpus with deepfake audio generated using YourTTS and X-TTS, we create interleaved clips that better reflect how deepfakes might be used in practice. We evaluate three deep learning models CNN, Bi-LSTM, and ResNet-18 across four versions of our dataset, each increasing in complexity. On the simpler versions ( \(V_1\) and \(V_2\) ), CNN performs the best, achieving up to 84% accuracy when detecting fully real or fake clips. But in the more challenging interleaved versions ( \(V_3\) and \(V_4\) ), ResNet-18 shows stronger performance, handling mixed audio more effectively. These results highlight the limits of standard binary classifiers and the need for more advanced, segment-aware detection systems. Our dataset and benchmarks aim to push forward research in building practical and robust solutions for detecting deepfake audio in the wild.