BiGMF: Multimodal Sentiment Analysis By Bidirectional Cross-Modal Attention with Geometric Volume Regularization
摘要
Multimodal Sentiment Analysis (MSA) aims to integrate text, audio, and visual information to better recognize human emotions. The reciprocal exchange of semantic information across text, audio, and video modalities often lacks a structured mechanism in previous works. This difficulty in modeling complex cross-modal dependencies consequently restricts their capacity to capture detailed semantic correlations. Moreover, previous methods often fail to align disparate modalities, causing semantic inconsistencies and redundancy during fusion. To address these issues, this paper introduces a novel bidirectional cross-modal fusion framework named BiGMF. The method is built upon a hierarchical cross-modal interaction architecture that enables bidirectional information exchange at multiple levels, enhancing the modeling capacity for cross-modal interactions. In addition, a geometric volume regularization strategy is introduced to reinforce semantic consistency. This strategy explicitly promotes the alignment of modality-specific features by constraining the geometric volume of their joint distribution in a shared embedding space. Extensive experiments on two MSA benchmarks demonstrate the effectiveness of the proposed method.