In this work, we propose an effective context-aware framework for human-robot interaction (HRI) that uses large language models (LLM) and state-of-the-art object detection techniques. Our approach integrates ChatGPT as the LLM for interpreting task instructions and generating context-aware responses, while YOLOv8 is employed for robust object detection in dynamic environments. The proposed system interprets spoken task requirements, extracts contextual information, and engages with a dual-arm service robotic system for real-time task execution. We present a comprehensive evaluation of our method, demonstrating significant improvements in task understanding, response accuracy, and adaptability in various interactive scenarios. Experimental results validate the efficacy of our approach in enhancing HRI.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Context-Aware LLM-Based Human-Robot Interaction

  • Thanh Nguyen Canh,
  • Kien HoangVan,
  • Xiem HoangVan

摘要

In this work, we propose an effective context-aware framework for human-robot interaction (HRI) that uses large language models (LLM) and state-of-the-art object detection techniques. Our approach integrates ChatGPT as the LLM for interpreting task instructions and generating context-aware responses, while YOLOv8 is employed for robust object detection in dynamic environments. The proposed system interprets spoken task requirements, extracts contextual information, and engages with a dual-arm service robotic system for real-time task execution. We present a comprehensive evaluation of our method, demonstrating significant improvements in task understanding, response accuracy, and adaptability in various interactive scenarios. Experimental results validate the efficacy of our approach in enhancing HRI.