Towards Enabling Intuitive Interaction and Control of Mobile Robots Utilising Augmented Reality Techniques

Significant effort has been allocated recently by the research community for enabling seamless co-existence and communication between human operators and robots. In that context, the adoption of Augmented Reality (AR) has gained major attention towards supporting the operator's interaction with the robots. This paper aims towards enabling human-robot interaction (HRI) and robot control of mobile manipulators through the development of an AR application providing simple, customisable, and user-friendly interfaces. Robot data such as real-time status and planned pathways are projected to the operator’s augmented environment enhancing their safety awareness. Additionally, interaction tools designed to enable robot control functionalities and holographic indications are tailored to provide real-time information to the operators, following User Experience (UX) design principles. The designed augmented environment is easily interacted with, utilising voice commands and virtual buttons. The implemented functionalities are realised through online communication and data exchange between the AR application and the ROS framework. The proposed application is validated through an industrial scenario from the elevators’ manufacturing industry focusing on the elevator’s electrical panel assembly.