Multimodal Sentiment Analysis (MSA) requires robust representations that capture both cross-modal consistency and intra-modal distinctions. Existing fusion methods often fail to adapt to diverse sentiment cues and neglect inter-modal correlations, while contrastive learning approaches insufficiently consider pair distribution and loss design. We propose an Adaptive Multi-scale Convolution fusion network with Contrastive Learning for multimodal sentiment analysis (AMCCL), which dynamically fuses multimodal information using an Adaptive Multi-scale Convolution (AMC) module. The AMC module dynamically fuses features through multi-scale convolutions with adaptive weighting and squeeze-and-excitation block to enhance salient channels. Our fine-grained contrastive learning leverages sentiment polarity and intensity, with tailored loss functions to strengthen the positive pairs and balance the inter-modal and intra-modal relations. Extensive evaluations on the MOSI and MOSEI datasets confirm that AMCCL delivers superior performance relative to state-of-the-art approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AMCCL: Adaptive Multi-scale Convolution Fusion Network with Contrastive Learning for Multimodal Sentiment Analysis

  • Jiakang Yu,
  • Mingxin Li,
  • Hongtao Deng,
  • Wang Gao,
  • Xun Zhu

摘要

Multimodal Sentiment Analysis (MSA) requires robust representations that capture both cross-modal consistency and intra-modal distinctions. Existing fusion methods often fail to adapt to diverse sentiment cues and neglect inter-modal correlations, while contrastive learning approaches insufficiently consider pair distribution and loss design. We propose an Adaptive Multi-scale Convolution fusion network with Contrastive Learning for multimodal sentiment analysis (AMCCL), which dynamically fuses multimodal information using an Adaptive Multi-scale Convolution (AMC) module. The AMC module dynamically fuses features through multi-scale convolutions with adaptive weighting and squeeze-and-excitation block to enhance salient channels. Our fine-grained contrastive learning leverages sentiment polarity and intensity, with tailored loss functions to strengthen the positive pairs and balance the inter-modal and intra-modal relations. Extensive evaluations on the MOSI and MOSEI datasets confirm that AMCCL delivers superior performance relative to state-of-the-art approaches.