Scholarlink

In this paper, we present a user-friendly mixed reality (MR) interface designed for robot teleoperation. To enable untrained users to effectively operate a robot, we developed an interactive application leveraging SwiftUI, RealityKit, and ARKit, and deployed it on an extended reality (XR) headset. Our application enables users to control a robot by moving a virtual sphere that serves as a dynamic target. Using RealityKit’s support for natural hand and eye gestures, users guide the robot through easy and subtle motions, eliminating the need for controllers or exaggerated gestures. This reduces physical strain, enhances comfort, and makes teleoperation more immersive. The system has been experimentally tested and statistically validated across users with varying experience levels.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Mixed Reality User-Friendly Interface for Robot Teleoperation

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Mariia Chemerys,
  • Matvei Novoselov,
  • Sofia Diniz Melo Santos,
  • Riccardo Aliotta,
  • Francesco Cufino,
  • Fabio Ruggiero

摘要

In this paper, we present a user-friendly mixed reality (MR) interface designed for robot teleoperation. To enable untrained users to effectively operate a robot, we developed an interactive application leveraging SwiftUI, RealityKit, and ARKit, and deployed it on an extended reality (XR) headset. Our application enables users to control a robot by moving a virtual sphere that serves as a dynamic target. Using RealityKit’s support for natural hand and eye gestures, users guide the robot through easy and subtle motions, eliminating the need for controllers or exaggerated gestures. This reduces physical strain, enhances comfort, and makes teleoperation more immersive. The system has been experimentally tested and statistically validated across users with varying experience levels.