Multimodal sentiment analysis faces significant challenges when processing incomplete data, which is a common scenario in real-world applications due to sensor failures or transmission errors. In this paper, we propose a Prompt-Enhanced Multimodal Learning (PEML) framework that mimics human cognitive process for handling incomplete information. It comprises three core components: (1) Modality-Specific Prompt Encoder (MSPE) that activates prior knowledge through learnable prompt templates, providing adaptive enhancement for different missing patterns; (2) Cross-Modal Adaptive Alignment (CMAA) that establishes inter-modal information exchange channels through a dynamic gating mechanism; (3) Quality-Aware Fusion (QAF) that dynamically fuses high-quality features based on multi-level quality assessment, achieving confidence-based information integration. Extensive experiments across various missing data scenarios demonstrate that PEML outperforms existing state-of-the-art methods, validating the effectiveness of modeling human cognitive processes for robust multimodal learning.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Prompt-Enhanced Multimodal Learning for Robust Sentiment Analysis with Incomplete Data

  • Yuhao Sun,
  • Peng Zhang,
  • Wei Zhao,
  • Fuqiang Wang,
  • Xiangzhi Liu,
  • Xiaoming Wu

摘要

Multimodal sentiment analysis faces significant challenges when processing incomplete data, which is a common scenario in real-world applications due to sensor failures or transmission errors. In this paper, we propose a Prompt-Enhanced Multimodal Learning (PEML) framework that mimics human cognitive process for handling incomplete information. It comprises three core components: (1) Modality-Specific Prompt Encoder (MSPE) that activates prior knowledge through learnable prompt templates, providing adaptive enhancement for different missing patterns; (2) Cross-Modal Adaptive Alignment (CMAA) that establishes inter-modal information exchange channels through a dynamic gating mechanism; (3) Quality-Aware Fusion (QAF) that dynamically fuses high-quality features based on multi-level quality assessment, achieving confidence-based information integration. Extensive experiments across various missing data scenarios demonstrate that PEML outperforms existing state-of-the-art methods, validating the effectiveness of modeling human cognitive processes for robust multimodal learning.