Depression, a widespread mental health condition, presents significant challenges due to stigma, concealed symptoms, and high treatment costs, underscoring the need for effective Automatic Depression Detection (ADD) systems. However, existing methods often face difficulties in modeling long-term dependencies and directional information. To address these limitations, we propose DepMambaformer, a novel multimodal depression detection framework. Our model integrates CNNs for local feature extraction, the designed DepBiMamba2 module to capture global dependencies and directional information, and multi-head attention for multimodal fusion of audio and video features. The combination of state space duality module with attention mechanisms demonstrates superior performance. Extensive experiments validate the effectiveness and scalability of DepMambaformer, outperforming state-of-the-art methods. Furthermore, ablation studies and visualizations provide deeper insights into the model’s functionality. This work highlights the potential of advanced global feature modeling in enhancing ADD systems for early and efficient depression screening.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DepMambaformer: Integrating Bidirectional State Space Duality Model with Multimodal Attention for Depression Detection

  • Changfeng He,
  • Yutao Dou,
  • Fan Liu,
  • Xiaoping Wang,
  • Shaoliang Peng

摘要

Depression, a widespread mental health condition, presents significant challenges due to stigma, concealed symptoms, and high treatment costs, underscoring the need for effective Automatic Depression Detection (ADD) systems. However, existing methods often face difficulties in modeling long-term dependencies and directional information. To address these limitations, we propose DepMambaformer, a novel multimodal depression detection framework. Our model integrates CNNs for local feature extraction, the designed DepBiMamba2 module to capture global dependencies and directional information, and multi-head attention for multimodal fusion of audio and video features. The combination of state space duality module with attention mechanisms demonstrates superior performance. Extensive experiments validate the effectiveness and scalability of DepMambaformer, outperforming state-of-the-art methods. Furthermore, ablation studies and visualizations provide deeper insights into the model’s functionality. This work highlights the potential of advanced global feature modeling in enhancing ADD systems for early and efficient depression screening.