Scholarlink

<p>Multimodal Emotion Recognition in Conversations (MERC) focuses on identifying emotions by jointly analyzing textual, acoustic, and visual cues. However, existing MERC models suffer from three significant drawbacks: (i) weak inter-modal alignment, (ii) inadequate modelling of speaker-dependent emotional expressions, and (iii) limited ability to capture long-range contextual information in multiple utterances in a conversation. To address these limitations, we propose SAMER-Net, a Speaker-Aware Multimodal Emotion Recognition framework that explicitly integrates contextual modeling, speaker information, and cross-modal reasoning. The proposed model employs a Context-Aware Modulated Attention (CAMA) mechanism that aligns modalities by conditioning attention weights on both conversational context and speaker identity, enabling more reliable extraction of emotionally salient cues. In addition, a lightweight GATv2-based graph network is used to capture inter-utterance dependencies across modalities while preserving computational efficiency. To further handle the severe class imbalance commonly observed in emotion-annotated conversational datasets, we incorporate focal loss, which emphasizes learning from minority emotion classes. Experiments are performed on two benchmark datasets, IEMOCAP and MELD, achieving an average accuracy of 72.45% and 68.38% outperforming state-of-the-art methods and offering a robust framework for speaker-awareness and context-aligned modality-sensitive emotion recognition.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SAMER-Net: speaker-aware multimodal emotion recognition with context-aware attention and graph convolutional networks

  • 期刊论文

  • 发表日期：

    2026年3月18日

查看全文
  • Bhavana Verma,
  • Priyanka Meel,
  • Dinesh Kumar Vishwakarma

摘要

Multimodal Emotion Recognition in Conversations (MERC) focuses on identifying emotions by jointly analyzing textual, acoustic, and visual cues. However, existing MERC models suffer from three significant drawbacks: (i) weak inter-modal alignment, (ii) inadequate modelling of speaker-dependent emotional expressions, and (iii) limited ability to capture long-range contextual information in multiple utterances in a conversation. To address these limitations, we propose SAMER-Net, a Speaker-Aware Multimodal Emotion Recognition framework that explicitly integrates contextual modeling, speaker information, and cross-modal reasoning. The proposed model employs a Context-Aware Modulated Attention (CAMA) mechanism that aligns modalities by conditioning attention weights on both conversational context and speaker identity, enabling more reliable extraction of emotionally salient cues. In addition, a lightweight GATv2-based graph network is used to capture inter-utterance dependencies across modalities while preserving computational efficiency. To further handle the severe class imbalance commonly observed in emotion-annotated conversational datasets, we incorporate focal loss, which emphasizes learning from minority emotion classes. Experiments are performed on two benchmark datasets, IEMOCAP and MELD, achieving an average accuracy of 72.45% and 68.38% outperforming state-of-the-art methods and offering a robust framework for speaker-awareness and context-aligned modality-sensitive emotion recognition.