Scholarlink

<b>Purpose</b> <p>Accurate 3D hand pose estimation supports surgical applications such as skill assessment and robot-assisted interventions. However, surgical environments pose severe challenges, including intense and localized lighting, frequent occlusions by instruments or staff, and uniform hand appearance due to gloves, combined with a scarcity of annotated datasets for reliable model training.</p> <b>Method</b> <p>We propose a multi-view pipeline for 3D hand pose estimation in surgical contexts that requires no domain-specific fine-tuning and relies on off-the-shelf pretrained models. The pipeline combines person detection, whole-body pose estimation, and 2D hand keypoint prediction on tracked hand crops, followed by constrained 3D optimization. We also introduce a benchmark dataset with over 68,000 frames and 3,000 manually annotated 2D hand poses with triangulated 3D ground truth, recorded in a high-fidelity physical operating room replica.</p> <b>Results</b> <p>Quantitative evaluation on the proposed dataset shows that the pipeline consistently outperforms state-of-the-art baselines for both two- and three-dimensional hand pose estimation. The method substantially improves joint localization accuracy, achieving up to a 31% reduction in 2D joint error and a 76% reduction in 3D joint position error. The approach remains robust across increasing levels of scene complexity, including motion and partial occlusions.</p> <b>Conclusion</b> <p>The proposed multi-view pipeline demonstrates the potential of combining robust detection, tracking, and geometric optimization for three-dimensional hand pose estimation in surgical environments without domain-specific retraining. Together with the introduced dataset, this work provides a baseline framework and benchmarking resource to support future research on surgical motion analysis and objective skill assessment.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A multi-view pipeline and benchmark dataset for 3D hand pose estimation in surgery

  • 期刊论文

  • 发表日期：

    2026年5月11日

查看全文
  • Valery Fischer,
  • Alan Magdaleno,
  • Anna-Katharina Calek,
  • Nicola Cavalcanti,
  • Nathan Hoffman,
  • Christoph Germann,
  • Joschua Wüthrich,
  • Max Krähenmann,
  • Mazda Farshad,
  • Philipp Fürnstahl,
  • Lilian Calvet

摘要

Purpose

Accurate 3D hand pose estimation supports surgical applications such as skill assessment and robot-assisted interventions. However, surgical environments pose severe challenges, including intense and localized lighting, frequent occlusions by instruments or staff, and uniform hand appearance due to gloves, combined with a scarcity of annotated datasets for reliable model training.

Method

We propose a multi-view pipeline for 3D hand pose estimation in surgical contexts that requires no domain-specific fine-tuning and relies on off-the-shelf pretrained models. The pipeline combines person detection, whole-body pose estimation, and 2D hand keypoint prediction on tracked hand crops, followed by constrained 3D optimization. We also introduce a benchmark dataset with over 68,000 frames and 3,000 manually annotated 2D hand poses with triangulated 3D ground truth, recorded in a high-fidelity physical operating room replica.

Results

Quantitative evaluation on the proposed dataset shows that the pipeline consistently outperforms state-of-the-art baselines for both two- and three-dimensional hand pose estimation. The method substantially improves joint localization accuracy, achieving up to a 31% reduction in 2D joint error and a 76% reduction in 3D joint position error. The approach remains robust across increasing levels of scene complexity, including motion and partial occlusions.

Conclusion

The proposed multi-view pipeline demonstrates the potential of combining robust detection, tracking, and geometric optimization for three-dimensional hand pose estimation in surgical environments without domain-specific retraining. Together with the introduced dataset, this work provides a baseline framework and benchmarking resource to support future research on surgical motion analysis and objective skill assessment.