Progressive Enhancement for Emotional Speech Captioning
摘要
Speech emotion understanding has garnered significant attention in recent years due to advances in audio large language models (ALLMs). While existing methods perform well on emotion classification tasks, the inherent dynamic nature of emotions and their reliance on subtle acoustic cues present a core challenge: how to simulate the human reasoning process for understanding complex and evolving emotional states. To address this, we propose a novel Progressive Emotion Reasoning Framework, in which the reasoning process is guided by both paralinguistic and linguistic information. This framework enhances the emotional comprehension capability of ALLMs through a progressive strategy, centered around a three-stage fine-tuning process: first, injecting linguistically and paralinguistically rich emotional cues prior to model reasoning; second, performing emotion-based curriculum fine-tuning; and finally, generating high-quality speech emotion descriptions through interpretable reasoning steps. Furthermore, we have established a foundational data infrastructure and introduced the first paralinguistic-centric speech emotion dataset, which includes multi-dimensional annotations such as paralinguistic features, emotion labels, and text transcripts, providing a critical resource for advancing speech emotion understanding research. Experimental results demonstrate that our method achieves leading and competitive performance on speech emotion understanding and description tasks, offering a novel perspective for the field.