The prolonged use of tools emitting vibration may cause damage in fingers and hands, known as the Raynaud/White Finger Syndrome. A solution is monitoring the device use for those who work long hours with heavy tools. In this paper, we demonstrate how to hack the Rokoko smart gloves to enable them to detect and monitor the tool in use. We demonstrate how to capture data and how to train a machine learning model. The data collected stems from six 9-DoF IMUs that are placed at each finger and the back of the hand. The model is a Recurrent Neural Network (RNN) with a Long Short-Term Memory (LSTM) layer. In this work-in-progress we show the exemplary discrimination of idle from the use of scissors, knife, and hammer with a precision of 70.75%. As workers already wear gloves, we believe commercial smart gloves to be highly practical for similar applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Hacking Rokoko Smart Gloves for Tool Detection Using Recurrent Neural Networks

  • Zhouyao Yu,
  • Ruben Schlonsak,
  • Denys J. C. Matthies

摘要

The prolonged use of tools emitting vibration may cause damage in fingers and hands, known as the Raynaud/White Finger Syndrome. A solution is monitoring the device use for those who work long hours with heavy tools. In this paper, we demonstrate how to hack the Rokoko smart gloves to enable them to detect and monitor the tool in use. We demonstrate how to capture data and how to train a machine learning model. The data collected stems from six 9-DoF IMUs that are placed at each finger and the back of the hand. The model is a Recurrent Neural Network (RNN) with a Long Short-Term Memory (LSTM) layer. In this work-in-progress we show the exemplary discrimination of idle from the use of scissors, knife, and hammer with a precision of 70.75%. As workers already wear gloves, we believe commercial smart gloves to be highly practical for similar applications.