Emotional Speech Synthesis Approach Using Prosody-Based Clustering
摘要
Recent advancements in neural text-to-speech (TTS) have enabled near-human naturalness, yet achieving expressive and controllable emotional speech remains a challenge. Current models struggle with prosodic control due to label dependency, style entanglement, and limited granularity in emotion manipulation. To address these limitations, we propose a novel methodology that integrates unsupervised clustering with multi-modal representations, combining neural style embeddings from StyleTTS 2 with acoustic-prosodic features extracted via ‘librosa’ and ‘emotion2vec’. By leveraging a hierarchical clustering approach using Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN), we achieve improved separation of emotional styles without requiring predefined labels. Experiments on the EmoV-DB and Hi-Fi TTS datasets confirm that our approach effectively captures nuanced emotional variations in speech synthesis, leading to an 200% increase in overall clusters, a reduction of 21.4% of unclustered data samples and an 25.14% increase of homogeneous cluster groups compared to the baseline.