A Vision-Based Motion Retargeting for Teleoperation of Dexterous Robotic Hands
摘要
Teleoperation control of dexterous robotic hands can substitute for humans in performing relatively difficult tasks. To achieve low-cost teleoperation control of dexterous robotic hands, this paper designs a vision-based retargeting method (MoReNet) for dexterous robotic hand. Thus, ordinary RGB cameras can complete the teleoperation of the dexterous robotic hand. This motion retargeting method combines a visual hand pose estimation network and a deep-learning-based retargeting network. First, the hand skeleton images identified by MediaPipe which is a hand pose estimator and the joint angles of the dexterous robotic hand are used as the dataset. Then an encoder-decoder retargeting network is trained, with the hand skeleton images as input and the joint angles of the dexterous hand as output. Finally, the loss of joint angle and control effect of the dexterous hand teleoperation are analyzed through simulation. We design two motion retargeting modes for teleoperation: pose mode and grasping mode, targeting the two applications of dexterous hand pose and grasping. The results show that the loss of joint angle is stable at 0.0072 rad, and the accuracy is higher than other retargeting methods. In the simulation experiments, the dexterous hand can well imitate human hand movements. Therefore, the motion retargeting method proposed in this paper can achieve low-cost and accurate teleoperation of the dexterous hand.