The development and integration of artificial intelligence and robotics technologies have significantly progressed. The emergence of large language models and embodied intelligence technologies has created new opportunities for various robot applications while advancing technologies related to the metaverse. Among these, spatially separated telepresence and teleoperation are key technologies that connect humans and machines in metaverse applications. A crucial area of research in this field is the development of efficient human-machine interfaces (HMIs). VR immersive teleoperation based on panoramic imaging has been proven to improve the teleoperator’s situation awareness effectively. It is a highly promising human-machine interface within the metaverse technology system. The proposed “Avatar” system is an immersive, ultra-low-latency panoramic HMI platform. The feasibility of the Avatar system on various robots with mainstream applications is tested, showing that Avatar successfully achieved immersive telepresence and teleoperation across different physical distances. This intuitive platform enhances efficiency and situational awareness in teleoperation applications. With further advancements in related technologies, the Avatar system has the potential to become a versatile tool for efficient and symbiotic cooperation across the metaverse, embodied intelligence, and human-computer interaction applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Amplifying Robotics Capacities with a Human Touch: An Immersive Human-Machine Interaction System

  • Junjie Li,
  • Kang Li,
  • Dewei Han,
  • Jian Xu,
  • Zhaoyuan Ma

摘要

The development and integration of artificial intelligence and robotics technologies have significantly progressed. The emergence of large language models and embodied intelligence technologies has created new opportunities for various robot applications while advancing technologies related to the metaverse. Among these, spatially separated telepresence and teleoperation are key technologies that connect humans and machines in metaverse applications. A crucial area of research in this field is the development of efficient human-machine interfaces (HMIs). VR immersive teleoperation based on panoramic imaging has been proven to improve the teleoperator’s situation awareness effectively. It is a highly promising human-machine interface within the metaverse technology system. The proposed “Avatar” system is an immersive, ultra-low-latency panoramic HMI platform. The feasibility of the Avatar system on various robots with mainstream applications is tested, showing that Avatar successfully achieved immersive telepresence and teleoperation across different physical distances. This intuitive platform enhances efficiency and situational awareness in teleoperation applications. With further advancements in related technologies, the Avatar system has the potential to become a versatile tool for efficient and symbiotic cooperation across the metaverse, embodied intelligence, and human-computer interaction applications.