Scholarlink

<p>Human action recognition from videos is a cornerstone in multimedia and computer vision, with applications spanning human behavior analysis, video surveillance, and intelligent transportation systems. Traditional RGB-based methods suffer from low robustness to lighting changes and high computational costs. In contrast, skeleton-based approaches exhibit enhanced robustness by excluding background noise. However, existing skeleton-based methods often adopt a two-stage pipeline, involving separate pose estimation and action recognition models, leading to error propagation and amplification. To address these challenges, we propose an end-to-end deep framework that unifies pose estimation and skeleton action recognition within a joint optimization architecture. Our framework eliminates explicit intermediate pose decoding by preserving implicit pose representations through Gaussian probability encoding, thereby avoiding quantization errors and enabling differentiable optimization. Building upon STGCN++, we introduce a multi-scale network, MS-STGCN++, to enhance multi-scale feature exchange. Additionally, a lightweight visual semantic branch integrates discarded low-level features with skeleton motion patterns via cross-attention, providing complementary scene context. Extensive experiments on NTU-60, NTU-120, and N-UCLA datasets demonstrate that our framework outperforms the two-stage baseline in both pose estimation (achieving an mAP improvement of 2.2% on NTU-60 Xview) and action recognition accuracy (increasing accuracy by 0.8% on NTU-60 Xview), offering a practical solution for integrated pose-action recognition. Code has been made available at <a href="https://github.com/YijieYang23/End2end">https://github.com/YijieYang23/End2end</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

End-to-end pose-action recognition via implicit pose encoding and multi-scale skeleton modeling

  • 期刊论文

  • 发表日期：

    2025年6月9日

查看全文
  • Yijie Yang,
  • Jianlin Zhou,
  • Wei Hu,
  • Zhigang Tu

摘要

Human action recognition from videos is a cornerstone in multimedia and computer vision, with applications spanning human behavior analysis, video surveillance, and intelligent transportation systems. Traditional RGB-based methods suffer from low robustness to lighting changes and high computational costs. In contrast, skeleton-based approaches exhibit enhanced robustness by excluding background noise. However, existing skeleton-based methods often adopt a two-stage pipeline, involving separate pose estimation and action recognition models, leading to error propagation and amplification. To address these challenges, we propose an end-to-end deep framework that unifies pose estimation and skeleton action recognition within a joint optimization architecture. Our framework eliminates explicit intermediate pose decoding by preserving implicit pose representations through Gaussian probability encoding, thereby avoiding quantization errors and enabling differentiable optimization. Building upon STGCN++, we introduce a multi-scale network, MS-STGCN++, to enhance multi-scale feature exchange. Additionally, a lightweight visual semantic branch integrates discarded low-level features with skeleton motion patterns via cross-attention, providing complementary scene context. Extensive experiments on NTU-60, NTU-120, and N-UCLA datasets demonstrate that our framework outperforms the two-stage baseline in both pose estimation (achieving an mAP improvement of 2.2% on NTU-60 Xview) and action recognition accuracy (increasing accuracy by 0.8% on NTU-60 Xview), offering a practical solution for integrated pose-action recognition. Code has been made available at https://github.com/YijieYang23/End2end.