Scholarlink

<p>3D human pose estimation is a key task in computer vision with wide-ranging applications. Although 2D-to-3D pose lifting methods have made significant progress, challenges remain in handling complex movements and local occlusions. In this paper, we propose a spatio-temporal dual-stream fusion (D2SF) method, which integrates a spatial interaction module (SIM) and a temporal interaction module (TIM). SIM incorporates depthwise convolution and a cross-channel encoding mechanism, while TIM adopts bilinear pooling to accurately model cross-joint and cross-frame interactions, respectively, yielding enhanced spatio-temporal representations. We also introduce an adaptive gated residual fusion mechanism to effectively integrate complementary features from both the Transformer and GCN streams. A learnable implicit topology matrix is incorporated into the GCN stream to improve its capacity for modeling complex motion patterns. Extensive experiments on the Human3.6M and MPI-INF-3DHP datasets demonstrate the effectiveness of our method, achieving highly competitive performance. Notably, our method achieves P1 errors of 37.6 mm and 14.3 mm with 243 frames on these two datasets, respectively. The source code is publicly available at: <a href="https://github.com/Cuinanc/D2SF">https://github.com/Cuinanc/D2SF</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing 3D human pose estimation via spatio-temporal dual-stream fusion

  • 期刊论文

  • 发表日期：

    2025年9月20日

查看全文
  • Junfen Chen,
  • Shuhan Cui,
  • Jie Zhu,
  • Bojun Xie

摘要

3D human pose estimation is a key task in computer vision with wide-ranging applications. Although 2D-to-3D pose lifting methods have made significant progress, challenges remain in handling complex movements and local occlusions. In this paper, we propose a spatio-temporal dual-stream fusion (D2SF) method, which integrates a spatial interaction module (SIM) and a temporal interaction module (TIM). SIM incorporates depthwise convolution and a cross-channel encoding mechanism, while TIM adopts bilinear pooling to accurately model cross-joint and cross-frame interactions, respectively, yielding enhanced spatio-temporal representations. We also introduce an adaptive gated residual fusion mechanism to effectively integrate complementary features from both the Transformer and GCN streams. A learnable implicit topology matrix is incorporated into the GCN stream to improve its capacity for modeling complex motion patterns. Extensive experiments on the Human3.6M and MPI-INF-3DHP datasets demonstrate the effectiveness of our method, achieving highly competitive performance. Notably, our method achieves P1 errors of 37.6 mm and 14.3 mm with 243 frames on these two datasets, respectively. The source code is publicly available at: https://github.com/Cuinanc/D2SF.