Robot-Facilitated Human–Robot Interaction with Integrated Tracking, Re-identification and Gesture Recognition

For the success in human–robot collaboration, it is essential to facilitate the readiness and maintenance of human–robot interaction (HRI). With the advance of robot intelligence, robot is expected to play a key role for such facilitation so that human can focus on the main task. In this paper, we present an HRI system, referred to here as Robot-Facilitated Interaction System (RFIS), by which an autonomous mobile robot transporting a patient facilitates interaction with an accompanying caretaker monitoring and nursing the patient. The proposed RFIS enables a transport robot to automatically recognize, track and reidentify a caretaker in a cluttered area, while understanding gestures of the caretaker, to proactively maintain readiness in real time for anytime-anywhere interaction. An emphasis is given to efficient and robust integration of interaction facilitation modules into a real time HRI system in an edge computing environment. The proposed RFIS is implemented with lightly configured yet performance optimized deep learning network modules and applied to an autonomous mobile robot developed for transporting a patient. Experimental results demonstrate the state-of-the-art performance of RFIS in terms of its operational efficacy in HRI.