This paper presents a semantic, platform-agnostic motion–data transfer framework for enabling expressive robotic conducting in real performance environments. Building on high-precision multimodal motion capture, we construct a four-dimensional Intermediate Representation (IR) that integrates spatial trajectories, timestamps, musical-event labels, and expressive intensity. This IR serves as the bridge between human conducting semantics and robot-executable motion. A three-stage pipeline—motion capture, semantics-aware data processing and retargeting, and stage-level temporal synchronization—is implemented on the Unitree H1 humanoid. Through rame-skipping delay compensation and joint-limit–aware amplitude correction, the system achieves millisecond-scale visual alignment and preserves key musical intentions despite kinematic differences. The framework has been validated through multiple public concerts and television productions, and further extended to robotic percussion, demonstrating both robustness and cross-modal generality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Robot Conducting Performance System Based on Motion Data Transfer

  • Yan Gao,
  • Xiaoqing Wang,
  • Di Lu,
  • Weiliao Deng,
  • Xinyu Guo,
  • Baoyin Cheng

摘要

This paper presents a semantic, platform-agnostic motion–data transfer framework for enabling expressive robotic conducting in real performance environments. Building on high-precision multimodal motion capture, we construct a four-dimensional Intermediate Representation (IR) that integrates spatial trajectories, timestamps, musical-event labels, and expressive intensity. This IR serves as the bridge between human conducting semantics and robot-executable motion. A three-stage pipeline—motion capture, semantics-aware data processing and retargeting, and stage-level temporal synchronization—is implemented on the Unitree H1 humanoid. Through rame-skipping delay compensation and joint-limit–aware amplitude correction, the system achieves millisecond-scale visual alignment and preserves key musical intentions despite kinematic differences. The framework has been validated through multiple public concerts and television productions, and further extended to robotic percussion, demonstrating both robustness and cross-modal generality.