Scholarlink

<p>Depression detection using multimodal data, such as facial images and speech signals, has emerged as an important research direction in affective computing. However, the task remains highly challenging due to difficulties in extracting high-quality features and achieving effective cross-modal fusion, which often limit model performance. To address these issues, we propose IAEC-DepressNet, an Identity-Adaptive and Emotionally Consistent Network for multimodal depression detection. Our model employs a two-level feature fusion strategy to integrate visual and auditory information effectively. Specifically, we pre-train the key video encoder using adversarial learning, enabling it to extract identity-invariant emotion representations. Audio features are extracted with existing tools, and a frame-level cross-modal emotional consistency mechanism is introduced to align temporal dynamics between the two modalities. This design encourages the model to capture time-varying emotional patterns consistently across modalities. We evaluate IAEC-DepressNet on three benchmark datasets: AVEC2014, AVEC2017, and AVEC2019. Experimental results demonstrate that our approach consistently achieves competitive performance across all datasets and establishes new state-of-the-art results on AVEC2017. The code is available at <a href="https://github.com/zhangxy0183-netizen/IAEC.">https://github.com/zhangxy0183-netizen/IAEC.</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

IAEC-DepressNet: Identity-Adaptive and Emotionally Consistent Multimodal Depression Detection Network

  • 期刊论文

  • 发表日期：

    2026年7月16日

查看全文
  • Xingyu Zhang,
  • Qiyang Li,
  • Jiangxin Gao,
  • Yanchun Liang,
  • Xiaohu Shi

摘要

Depression detection using multimodal data, such as facial images and speech signals, has emerged as an important research direction in affective computing. However, the task remains highly challenging due to difficulties in extracting high-quality features and achieving effective cross-modal fusion, which often limit model performance. To address these issues, we propose IAEC-DepressNet, an Identity-Adaptive and Emotionally Consistent Network for multimodal depression detection. Our model employs a two-level feature fusion strategy to integrate visual and auditory information effectively. Specifically, we pre-train the key video encoder using adversarial learning, enabling it to extract identity-invariant emotion representations. Audio features are extracted with existing tools, and a frame-level cross-modal emotional consistency mechanism is introduced to align temporal dynamics between the two modalities. This design encourages the model to capture time-varying emotional patterns consistently across modalities. We evaluate IAEC-DepressNet on three benchmark datasets: AVEC2014, AVEC2017, and AVEC2019. Experimental results demonstrate that our approach consistently achieves competitive performance across all datasets and establishes new state-of-the-art results on AVEC2017. The code is available at https://github.com/zhangxy0183-netizen/IAEC.