Using Vocoder Artifacts For Audio Deepfakes Detection

Recent advancements in AI-synthesized human voices have increased the threat of impersonation and disinformation. Detecting synthetic human voices has become crucial to combat these challenges. In this work, we propose a novel approach for detecting synthetic human voices by leveraging the identification of artifacts generated by neural vocoders in audio signals. Neural vocoders are specialized neural networks synthesizing waveforms using temporal-frequency representations such as Mel-spectrograms. These vocoders form a critical component in most deepfake audio synthesis models. Therefore, identifying the presence of neural vocoder processing suggests that an audio sample may have been artificially generated. To harness the potential of vocoder artifacts for synthetic human voice detection, we introduce a binary-class RawNet2 model. This model shares the same front-end feature extractor with the vocoder identification system. We employ a self-supervised representation learning (SSRL) approach, treating vocoder identification as a pretext task. Doing so ensures that the front-end feature extraction module is constrained and optimized to build the final binary classifier for synthetic human voice detection. Through extensive experiments, we demonstrate that the RawNet2 model, trained using SSRL based on vocoder identification, achieves a high overall classification performance on the binary task of detecting synthetic human voices. This approach holds promise in effectively combating the growing threat posed by AI-generated human voice impersonation and disinformation.