<p>Multimodal sentiment analysis (MSA) has gained significant attention due to its ability to leverage complementary information from multiple modalities, such as text, audio, and video, to improve sentiment understanding. However, existing approaches often focus on either global or local features, neglecting the complementary nature of these perspectives. To address this limitation, we propose a Dual-view Perspective Modeling (DPM) framework that integrates both global and local features for enhanced sentiment analysis. DPM employs a sparse mixture-of-experts network to selectively extract local features and a gated selection module to fuse global and local representations effectively. The model is evaluated on four benchmark datasets: CMU-MOSI, CMU-MOSEI, CH-SIMS, and UR-FUNNY. Experimental results demonstrate that DPM outperforms state-of-the-art baselines across all datasets, achieving significant improvements in classification accuracy, F1 score, and regression metrics. Ablation studies further validate the contributions of each module, highlighting the importance of dual-view modeling in capturing both holistic and fine-grained sentiment cues. The proposed framework offers a robust and efficient solution for multimodal sentiment analysis, advancing the field toward more accurate and interpretable emotion recognition.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal sentiment analysis based on dual perspective modeling

  • Tuerhong Gulanbaier,
  • Wang Hao,
  • Wushouer Mairidan

摘要

Multimodal sentiment analysis (MSA) has gained significant attention due to its ability to leverage complementary information from multiple modalities, such as text, audio, and video, to improve sentiment understanding. However, existing approaches often focus on either global or local features, neglecting the complementary nature of these perspectives. To address this limitation, we propose a Dual-view Perspective Modeling (DPM) framework that integrates both global and local features for enhanced sentiment analysis. DPM employs a sparse mixture-of-experts network to selectively extract local features and a gated selection module to fuse global and local representations effectively. The model is evaluated on four benchmark datasets: CMU-MOSI, CMU-MOSEI, CH-SIMS, and UR-FUNNY. Experimental results demonstrate that DPM outperforms state-of-the-art baselines across all datasets, achieving significant improvements in classification accuracy, F1 score, and regression metrics. Ablation studies further validate the contributions of each module, highlighting the importance of dual-view modeling in capturing both holistic and fine-grained sentiment cues. The proposed framework offers a robust and efficient solution for multimodal sentiment analysis, advancing the field toward more accurate and interpretable emotion recognition.