Improving Voice Style Conversion via Self-attention VAE with Feature Disentanglement

Voice conversion (VC) is a widely used technique in intelligent speech processing, that aims to modify the speaker’s information while preserving the underlying linguistic content. Speech can be divided into five parts: linguistic content, timbre, rhythm, pitch, and accent. Obtaining the disentanglement representation of these components is very useful in many speech analysis and generation applications. Recently, state-of-the-art voice conversion systems have been able to decompose speech into content feature representation and timbre representations. While information such as accent, pitch, and rhythm are still mixed with the content. It is a challenging problem to separate components from speech without explicit annotated information. This paper proposes a voice style conversion model Disentangle-VSC, which can decompose speech into five components without any text labels. We employed an encoder-decoder architecture, utilizing extractors with distinct bottlenecks to extract different features. Then these features are fed as input to the decoder to generate transformed speech. We also apply the self-attention mechanism to capture the relationships between different speech features, thereby further improving the quality of the generated converted speech. The experimental results demonstrate that our model achieves feature disentanglement, and the quality of converted speech is better than that of AutoVC and SpeechSplit. The repository can be found at https://github.com/sanena/Disentangle-VSC.