Multimodal Sentiment Analysis via Spatio-Temporal Decoupling and Language-Focused Fusion
摘要
Multimodal Sentiment Analysis (MSA) aims to infer human emotions by integrating data from multiple modalities, such as linguistic, visual, and acoustic signals. It has found broad applications in domains such as social media opinion mining and human-computer interaction. Existing MSA methods typically focus on extracting cross-modal shared features or directly fusing heterogeneous modalities. However, processing or fusing all modalities indiscriminately may amplify irrelevant noise and introduce redundant information, thus undermining the effectiveness of the learned representations. To address these challenges, we propose a novel framework, Multimodal Sentiment Analysis via Spatio-Temporal Decoupling and Language-Focused Fusion (ST-DLF), specifically designed for the MSA task. In the feature decoupling stage, ST-DLF incorporates a spatio-temporal dual-attention mechanism to enhance the representational quality of each modality. Meanwhile, a quadruple loss function is introduced to suppress redundant and overlapping information across modalities. In the fusion stage, we design a Language-Focused Attractor, which employs a language-guided cross-attention mechanism to selectively incorporate complementary modality-specific information, thereby enriching sentiment-discriminative representations. Furthermore, by integrating low-level shared features, medium-level cross-modal features, and high-level decoupled features, ST-DLF further enhances the discriminative capacity of the fused representations. Extensive experiments on two benchmark datasets demonstrate the effectiveness of our approach, with consistent improvements over representative state-of-the-art baselines. Our code is available at https://github.com/sakitma-r/ST-DLF .