Scholarlink

<p>Digital dance and theater archives are difficult to analyse and understand because few tools can accurately extract 3D motion from monocular images, which are often low-quality in heritage collections. We present two cloud-ready pipelines that transform low-quality videos into temporally dense SMPL-X reconstructions, per-frame segmentation masks, depth maps, and estimated camera trajectories. The PREMIERE pipeline targets archival material, while the MultiPerson pipeline is tuned for high-frame-rate smartphone or action camera recordings with up to five performers. Both pipelines couple Segment Anything v2.1, Neural Localizer Fields pose recovery, MoGe-based depth cues, WiLoR hand refinement, and VGGT for camera parameters estimation. This is followed by scale-aware optimization and RBF smoothing. Extensive testing on the AIST++ dance data set confirms that the resulting 3D assets surpass earlier monocular baselines in pose fidelity and temporal stability, while remaining robust to motion blur, extreme lighting, and human occlusion on stage film. Outputs conform to a unified geometry schema that feeds the Horizon PREMIERE project tools and two open-source WebGL viewers, enabling browser-based playback, VR/MR immersion, and high-resolution render export with no additional recording hardware. All codes, evaluation scripts, and viewers will be released under an open licence, offering a reproducible, extensible foundation for large-scale motion mining and the preservation of intangible cultural heritage. See interactive results on our project page: <a href="https://www.couleur.org/PREMIERE/JMTA/">https://www.couleur.org/PREMIERE/JMTA/</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

End-to-end pipelines for scalable 3D motion mining in dance archives from monocular footage

  • 期刊论文

  • 发表日期：

    2026年7月27日

查看全文
  • Philippe Colantoni,
  • Rafique Ahmed,
  • Prashant Ghimire,
  • Damien Muselet,
  • Alain Trémeau

摘要

Digital dance and theater archives are difficult to analyse and understand because few tools can accurately extract 3D motion from monocular images, which are often low-quality in heritage collections. We present two cloud-ready pipelines that transform low-quality videos into temporally dense SMPL-X reconstructions, per-frame segmentation masks, depth maps, and estimated camera trajectories. The PREMIERE pipeline targets archival material, while the MultiPerson pipeline is tuned for high-frame-rate smartphone or action camera recordings with up to five performers. Both pipelines couple Segment Anything v2.1, Neural Localizer Fields pose recovery, MoGe-based depth cues, WiLoR hand refinement, and VGGT for camera parameters estimation. This is followed by scale-aware optimization and RBF smoothing. Extensive testing on the AIST++ dance data set confirms that the resulting 3D assets surpass earlier monocular baselines in pose fidelity and temporal stability, while remaining robust to motion blur, extreme lighting, and human occlusion on stage film. Outputs conform to a unified geometry schema that feeds the Horizon PREMIERE project tools and two open-source WebGL viewers, enabling browser-based playback, VR/MR immersion, and high-resolution render export with no additional recording hardware. All codes, evaluation scripts, and viewers will be released under an open licence, offering a reproducible, extensible foundation for large-scale motion mining and the preservation of intangible cultural heritage. See interactive results on our project page: https://www.couleur.org/PREMIERE/JMTA/.