Scholarlink

<p>Social media platforms are often flooded with short videos that contain emotional interpretation segments. These videos possess the potential to go viral, significantly influencing public opinion. However, short video creators often intentionally introduce inconsistencies in sentiment semantics across different modalities to attract viewers, significantly degrading the efficacy of conventional multimodal sentiment analysis frameworks. To address this challenge, we propose the Multimodal Semantics Consistency Enhancement network via Hybrid Attention Fusion (MSCE-HAF), which integrates cross-attention and multi-head attention mechanisms to enhance semantic consistency while preserving raw modality-specific information. Our approach incorporates a distribution-aware alignment strategy leveraging CMD and MMD to effectively align cross-modal distributions. Experiments conducted on the CMU-MOSI, CMU-MOSEI, and CH-SIMS datasets demonstrate that MSCE-HAF achieves an average accuracy improvement of 2.3%, exhibiting superior performance when processing samples with inconsistent sentiment semantics across modalities. Real-world case studies further validate its effectiveness in detecting public opinion risks within short videos.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing semantics consistency via hybrid attention fusion in multimodal sentiment analysis of short videos

  • 期刊论文

  • 发表日期：

    2025年7月7日

查看全文
  • Xuanchi Gong,
  • Ziyang Xue,
  • Tengjun Liu,
  • Yubao Sun,
  • Yifan Zhang

摘要

Social media platforms are often flooded with short videos that contain emotional interpretation segments. These videos possess the potential to go viral, significantly influencing public opinion. However, short video creators often intentionally introduce inconsistencies in sentiment semantics across different modalities to attract viewers, significantly degrading the efficacy of conventional multimodal sentiment analysis frameworks. To address this challenge, we propose the Multimodal Semantics Consistency Enhancement network via Hybrid Attention Fusion (MSCE-HAF), which integrates cross-attention and multi-head attention mechanisms to enhance semantic consistency while preserving raw modality-specific information. Our approach incorporates a distribution-aware alignment strategy leveraging CMD and MMD to effectively align cross-modal distributions. Experiments conducted on the CMU-MOSI, CMU-MOSEI, and CH-SIMS datasets demonstrate that MSCE-HAF achieves an average accuracy improvement of 2.3%, exhibiting superior performance when processing samples with inconsistent sentiment semantics across modalities. Real-world case studies further validate its effectiveness in detecting public opinion risks within short videos.