Use of Robot Operating System for Remote Manipulation of Robot Arm Based on Hand Gestures

We have been investigating various robot manipulation systems based on human hand gestures. In the present study, we constructed a prototype system incorporating a robot operating system (ROS) that uses an AI-based camera to recognize hand movements for intuitive operation of a robot arm. The major difference between this and our previous work is the use of ROS communication. We developed nodes and topic messages for the ROS package, including a topic message (handtracker_topic) for human hand camera coordinate system data acquired using a DepthAI-based camera, a node (handtracker_pub) to publish this topic message, and a node (handtracker_bridge) that obtains camera coordinate system data and performs transformation processes, including adjustment of the operating range of the robot arm coordinate system. We applied our prototype system to operate a real scalar robot via the scara_controller node based on hand gestures, and at the same time emulated a gazebo robot via the scara_controller_gazebo node. Although the performance test was successful, we believe that it is still necessary to improve the correspondence between hand gestures and the movements of the robot. This can be achieved by adding sensitivity adjustment and acceleration control functions to make the movements more natural.