<p>Multimodal sentiment analysis aims to analyze and recognize users' emotional expressions through multiple modalities such as vision, acoustic, and text. Integrating complementary information from various modalities can enhance the accuracy and precision of sentiment recognition tasks. However, previous studies mainly focus on inter-modal interactions while overlooking the fact that different modalities contribute unequally to sentiment analysis. In this paper, we propose a priority-guided multi-scale adaptive attention network (PMAAN), which introduces the concept of modality priority and incorporates a complex hierarchical structure to learn multi-scale information, enhancing the performance of multimodal sentiment analysis. First, we efficiently fuse local multimodal information through interactions between high-priority and low-priority modalities. Then, using a global attention mechanism, we retain key information from high-priority modalities while interacting with low-priority modalities to extract multi-scale features. Finally, we dynamically adjust modality weights and remove redundant information to achieve final feature fusion. We evaluate our method on two challenging multimodal sentiment analysis datasets: CMU-MOSI and CMU-MOSEI. Experimental results demonstrate that PMAAN achieves competitive performance compared with state-of-the-art models while maintaining cost-effective computational complexity.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PMAAN: a priority-guided multi-scale adaptive attention network for multimodal sentiment analysis

  • Fei Xu,
  • Shuo An,
  • Daipeng Guo,
  • Xintong Zhang

摘要

Multimodal sentiment analysis aims to analyze and recognize users' emotional expressions through multiple modalities such as vision, acoustic, and text. Integrating complementary information from various modalities can enhance the accuracy and precision of sentiment recognition tasks. However, previous studies mainly focus on inter-modal interactions while overlooking the fact that different modalities contribute unequally to sentiment analysis. In this paper, we propose a priority-guided multi-scale adaptive attention network (PMAAN), which introduces the concept of modality priority and incorporates a complex hierarchical structure to learn multi-scale information, enhancing the performance of multimodal sentiment analysis. First, we efficiently fuse local multimodal information through interactions between high-priority and low-priority modalities. Then, using a global attention mechanism, we retain key information from high-priority modalities while interacting with low-priority modalities to extract multi-scale features. Finally, we dynamically adjust modality weights and remove redundant information to achieve final feature fusion. We evaluate our method on two challenging multimodal sentiment analysis datasets: CMU-MOSI and CMU-MOSEI. Experimental results demonstrate that PMAAN achieves competitive performance compared with state-of-the-art models while maintaining cost-effective computational complexity.