Scholarlink

Emotion recognition is essential for social robots to engage empathetically with humans. We propose a multimodal approach that integrates GPT-based textual analysis and Wav2Vec2-based acoustic processing to predict continuous emotional dimensions, valence and arousal, from speech. Our GRU-based neural ensemble achieves Concordance Correlation Coefficients of 0.715 for valence and 0.674 for arousal, significantly outperforming unimodal approaches. This method enables robots to more effectively interpret nuanced emotional states in real-world human-robot interactions.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal Prediction of Valence and Arousal from Speech for Emotion-Aware Interaction Systems

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Safal Dhungana,
  • Maria Pinto-Bernal,
  • Tony Belpaeme

摘要

Emotion recognition is essential for social robots to engage empathetically with humans. We propose a multimodal approach that integrates GPT-based textual analysis and Wav2Vec2-based acoustic processing to predict continuous emotional dimensions, valence and arousal, from speech. Our GRU-based neural ensemble achieves Concordance Correlation Coefficients of 0.715 for valence and 0.674 for arousal, significantly outperforming unimodal approaches. This method enables robots to more effectively interpret nuanced emotional states in real-world human-robot interactions.