<p>The growing influence of social media has transformed online public opinion into a complex multimodal environment, where textual expressions, vocal cues, and visual behaviors jointly convey rich affective information. Accurate multimodal sentiment analysis and emotion recognition are therefore essential for understanding public opinion dynamics and emotionally driven interactions on online platforms. However, existing multimodal models often rely on shallow fusion strategies and limited global reasoning, which can lead to incomplete emotional representations and unstable cross-modal interactions. To address these challenges, we propose DF-EMII, a dual-level multimodal fusion framework for sentiment analysis and emotion recognition in public opinion scenarios. DF-EMII integrates three complementary components: (1) ENSA, which enhances intra-modal representations through neuron-level saliency calibration and group-aware attention; (2) DACF, a two-stage adaptive fusion mechanism that aligns modalities under textual semantics and performs reciprocal cross-modal refinement; and (3) GMED, which incorporates memory-guided global context to preserve long-range semantic and emotional coherence. Extensive experiments on CMU-MOSI, CMU-MOSEI, and IEMOCAP demonstrate that DF-EMII consistently outperforms state-of-the-art approaches on both sentiment regression and emotion classification tasks. These results indicate that DF-EMII effectively captures subtle affective cues across modalities, providing a robust computational framework for multimodal sentiment analysis and emotion understanding in public opinion monitoring applications.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Df-emii: a dual-level multimodal fusion framework for sentiment analysis and emotion recognition with applications in public opinion monitoring

  • Xiaoyu He,
  • Fan Liu,
  • Yindong Li,
  • Xuanhe Chen

摘要

The growing influence of social media has transformed online public opinion into a complex multimodal environment, where textual expressions, vocal cues, and visual behaviors jointly convey rich affective information. Accurate multimodal sentiment analysis and emotion recognition are therefore essential for understanding public opinion dynamics and emotionally driven interactions on online platforms. However, existing multimodal models often rely on shallow fusion strategies and limited global reasoning, which can lead to incomplete emotional representations and unstable cross-modal interactions. To address these challenges, we propose DF-EMII, a dual-level multimodal fusion framework for sentiment analysis and emotion recognition in public opinion scenarios. DF-EMII integrates three complementary components: (1) ENSA, which enhances intra-modal representations through neuron-level saliency calibration and group-aware attention; (2) DACF, a two-stage adaptive fusion mechanism that aligns modalities under textual semantics and performs reciprocal cross-modal refinement; and (3) GMED, which incorporates memory-guided global context to preserve long-range semantic and emotional coherence. Extensive experiments on CMU-MOSI, CMU-MOSEI, and IEMOCAP demonstrate that DF-EMII consistently outperforms state-of-the-art approaches on both sentiment regression and emotion classification tasks. These results indicate that DF-EMII effectively captures subtle affective cues across modalities, providing a robust computational framework for multimodal sentiment analysis and emotion understanding in public opinion monitoring applications.