This study explores using Large Language Models (LLMs) for voice-controlled robotics to improve human-robot interaction (HRI) and make it more accessible to non-experts. Implemented on Boston Dynamics’ Spot robot, the framework uses state-of-the-art wake word detection, speech-to-text, and text-to-speech systems within a behavior tree architecture. Results show successful task execution and command comprehension, despite challenges with speech-to-text accuracy and LLM reliability. Experiments demonstrate the framework’s potential for handling simple to moderately complex commands, highlighting the need for improved speech recognition and safety mechanisms. This project underscores the promise of LLMs in robotics and HRI, suggesting future directions for more advanced, multimodal interaction frameworks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Large Language Model Framework for Intuitive Interaction with Autonomous Mobile Robots

  • Frederike Durow,
  • Rasmus Gjerlund,
  • Tsampikos Kounalakis,
  • Leon Bodenhagen

摘要

This study explores using Large Language Models (LLMs) for voice-controlled robotics to improve human-robot interaction (HRI) and make it more accessible to non-experts. Implemented on Boston Dynamics’ Spot robot, the framework uses state-of-the-art wake word detection, speech-to-text, and text-to-speech systems within a behavior tree architecture. Results show successful task execution and command comprehension, despite challenges with speech-to-text accuracy and LLM reliability. Experiments demonstrate the framework’s potential for handling simple to moderately complex commands, highlighting the need for improved speech recognition and safety mechanisms. This project underscores the promise of LLMs in robotics and HRI, suggesting future directions for more advanced, multimodal interaction frameworks.