Scholarlink

The integration of robotics and augmented reality (AR) presents transformative opportunities for advancing human-robot interaction (HRI) by improving usability, intuitiveness, and accessibility. This work introduces a controller-free, LLM-driven voice-commanded AR puppeteering system, enabling users to teleoperate a robot by manipulating its virtual counterpart in real-time. By leveraging natural language processing (NLP) and AR technologies, our system—prototyped using Meta Quest 3—eliminates the need for physical controllers, enhancing ease of use while minimizing potential safety risks associated with direct robot operation. A preliminary user demonstration successfully validated the system’s functionality, demonstrating its potential for safer, more intuitive, and immersive robotic control.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yuchong Zhang,
  • Bastian Orthmann,
  • Michael C. Welle,
  • Jonne Van Haastregt,
  • Danica Kragic

摘要

The integration of robotics and augmented reality (AR) presents transformative opportunities for advancing human-robot interaction (HRI) by improving usability, intuitiveness, and accessibility. This work introduces a controller-free, LLM-driven voice-commanded AR puppeteering system, enabling users to teleoperate a robot by manipulating its virtual counterpart in real-time. By leveraging natural language processing (NLP) and AR technologies, our system—prototyped using Meta Quest 3—eliminates the need for physical controllers, enhancing ease of use while minimizing potential safety risks associated with direct robot operation. A preliminary user demonstration successfully validated the system’s functionality, demonstrating its potential for safer, more intuitive, and immersive robotic control.