Vdd: voice deepfake detection with three-channel acoustic representations and advanced split-attention networks
摘要
Voice deepfake is an increasingly serious problem as speech synthesis technologies develop with great sophistication, requiring increasingly effective detection methods. Although there have been several studies combining different audio features to meet this requirement, the potential of using multiple features has not yet been fully exploited. Furthermore, previous research on speech deepfake detection using image classification models has primarily focused on earlier architectures, overlooking the capabilities of more advanced, modern models in computer vision. Therefore, this paper aims to address the challenges by introducing a novel approach for voice deepfake detection (VDD). The core novelty of VDD lies in extracting and combining three complementary acoustic features— Short-time Fourier transform (STFT), Mel frequency cepstral coefficients (MFCC), and Constant-Q cepstral coefficients (CQCC)—into a unified multi-channel representation, providing richer information for detection than using individual features alone. This representation is then processed by the Residual networks with split-attention (ResNeSt) architecture, which leverages split-attention mechanisms to focus on the most relevant patterns in the data. When trained on the logical access subset of ASVSpoof2019, VDD achieves a remarkable EER of 0.33%, surpassing the performance of previous approaches. With only 118K parameters and 0.078ms inference time, the model is well-suited for real-time use in speech-based security applications, especially for integration into portable devices and smartphones.