Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations
摘要
Emotion Recognition in Conversations (ERCs) is a vital area within multimodal interaction research, aiming to accurately identify and classify emotions expressed by speakers. Traditional ERC approaches mainly rely on unimodal cues—such as text, audio, or visual data—leading to limited performance. These methods face two key challenges: 1) Consistency in Multimodal Information. Effective integration requires well-aligned and coherent data across modalities. 2) Contextual Information Capture. Capturing evolving emotional dynamics in long conversations is essential. To tackle these issues, we propose a novel Mamba-enhanced Text-Audio-Video alignment network (MaTAV) for the ERC task. MaTAV aligns unimodal features to ensure cross-modal consistency and effectively handles long input sequences to better model contextual emotional flow. Extensive experiments on the MELD and IEMOCAP datasets demonstrate that MaTAV significantly outperforms existing state-of-the-art methods on the ERC task.