As global populations age, enabling older adults to live independently becomes increasingly important. This study presents a non-invasive, multi-device dexterous hand teleoperation system designed to enhance domestic service robots’ ability to perform assistive tasks remotely. By integrating multiple visual-based sensors—specifically cameras and Leap Motion Controllers—the system effectively recognizes two common hand grasping gestures, allowing real-time, accurate control of a robotic manipulator. Unlike prior approaches that rely on single-sensor input or wearable devices, the proposed system leverages sensor redundancy to improve gesture recognition and user comfort. Performance was validated through user studies involving two daily tasks, demonstrating a high success rate (94.17%) and positive user experience, with training effects observed in task efficiency and perceived workload. Notably, the system showed consistent usability and reliability across tasks, with NASA-TLX analysis highlighting decreased cognitive and physical demand over time. Future work aims to enhance grasp recognition with additional hand features, compare with wearable alternatives, and extend control to full robotic arm kinematics for more complex tasks. This work contributes toward scalable, intuitive teleoperation technologies that support aging in place through robotic assistance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

From Vision to Grasp: A Multi-device Hand Tracking System for Robotic Teleoperation in Daily Activities

  • Alessandra Sorrentino,
  • Niccolò Alunni,
  • Filippo Cavallo

摘要

As global populations age, enabling older adults to live independently becomes increasingly important. This study presents a non-invasive, multi-device dexterous hand teleoperation system designed to enhance domestic service robots’ ability to perform assistive tasks remotely. By integrating multiple visual-based sensors—specifically cameras and Leap Motion Controllers—the system effectively recognizes two common hand grasping gestures, allowing real-time, accurate control of a robotic manipulator. Unlike prior approaches that rely on single-sensor input or wearable devices, the proposed system leverages sensor redundancy to improve gesture recognition and user comfort. Performance was validated through user studies involving two daily tasks, demonstrating a high success rate (94.17%) and positive user experience, with training effects observed in task efficiency and perceived workload. Notably, the system showed consistent usability and reliability across tasks, with NASA-TLX analysis highlighting decreased cognitive and physical demand over time. Future work aims to enhance grasp recognition with additional hand features, compare with wearable alternatives, and extend control to full robotic arm kinematics for more complex tasks. This work contributes toward scalable, intuitive teleoperation technologies that support aging in place through robotic assistance.