Mixed Reality Platform to Support Human-Robot Bi-directional Interaction

This work proposes a Mixed Reality (MR) application, designed for HoloLens 2 which enables seamless interaction between collaborative robots and human operators, regardless of their prior experience or expertise in the field of robotics. Building on the application presented in a previous work of the authors, the novel contributions are focused on a bi-directional interaction that manages the exchange of data from the robot to the human operator and, in the meantime, the flow of commands in the opposite direction. The robot-to-human flow consists of the information about the robot state (joint positions, velocities and torques), the distance from the joint limits and the planned path of the end-effector. Moreover, the human commands to the robot the desired end-point for the end-effector, than the planned path can be directly modified by moving a set of way-points displayed in the mixed reality environment. A Franka Emika Panda collaborative robot has been used to test the application.