<p>With the rapid spread of social networks, understanding the emotional dynamics of tourists becomes a decisive topic of research in the field of affective computation and social network analysis. However, most current approaches are largely based on unimodal text data that allows the spirit of rich multimodal information, this limits the accuracy of forecasts and time modeling capability. To address these limitations, this paper proposes DyStaFusion (Dynamic State-Space and Transformer Fusion for Multimodal Emotion Tracking), a multimodal model of emotional and dynamic prediction, which is a multimodal fusion module, an integrated enhanced transformer with long-term dependency improvement, and a Mamba-State spatial module. Thanks to the combination of text, acoustics and visual characteristics, DyStaFusion effectively detects short-term fluctuations and long-term trends in emotional states. The advanced transformer module models time dependencies through modularity, while the Mamba state space module reinforces dynamic sequence representation. These results, which were performed on the MELD and Sentiment140 data sets, show that DyStaFusion achieves consecutive improvements over several ultra-modern baselines, with an overall productivity increase of 3 to 5% in emotion classification accuracy, emotion intensity prediction, and dynamic emotion modeling. The ablation studies continue to confirm the additional contribution of each module, and highlight the importance of multimodal fusion and the overall construction of the state-space transformer. These results show that DyStaFusion offers a reliable and widespread system for multimodal dynamic prediction of emotions, with significant potential for practical applications and future research.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DyStaFusion: Dynamic State-Space fusion network for multimodal tourist emotion dynamics prediction in social media

  • Enming Zhang,
  • Lin Zhou

摘要

With the rapid spread of social networks, understanding the emotional dynamics of tourists becomes a decisive topic of research in the field of affective computation and social network analysis. However, most current approaches are largely based on unimodal text data that allows the spirit of rich multimodal information, this limits the accuracy of forecasts and time modeling capability. To address these limitations, this paper proposes DyStaFusion (Dynamic State-Space and Transformer Fusion for Multimodal Emotion Tracking), a multimodal model of emotional and dynamic prediction, which is a multimodal fusion module, an integrated enhanced transformer with long-term dependency improvement, and a Mamba-State spatial module. Thanks to the combination of text, acoustics and visual characteristics, DyStaFusion effectively detects short-term fluctuations and long-term trends in emotional states. The advanced transformer module models time dependencies through modularity, while the Mamba state space module reinforces dynamic sequence representation. These results, which were performed on the MELD and Sentiment140 data sets, show that DyStaFusion achieves consecutive improvements over several ultra-modern baselines, with an overall productivity increase of 3 to 5% in emotion classification accuracy, emotion intensity prediction, and dynamic emotion modeling. The ablation studies continue to confirm the additional contribution of each module, and highlight the importance of multimodal fusion and the overall construction of the state-space transformer. These results show that DyStaFusion offers a reliable and widespread system for multimodal dynamic prediction of emotions, with significant potential for practical applications and future research.