Hacking Rokoko Smart Gloves for Tool Detection Using Recurrent Neural Networks
摘要
The prolonged use of tools emitting vibration may cause damage in fingers and hands, known as the Raynaud/White Finger Syndrome. A solution is monitoring the device use for those who work long hours with heavy tools. In this paper, we demonstrate how to hack the Rokoko smart gloves to enable them to detect and monitor the tool in use. We demonstrate how to capture data and how to train a machine learning model. The data collected stems from six 9-DoF IMUs that are placed at each finger and the back of the hand. The model is a Recurrent Neural Network (RNN) with a Long Short-Term Memory (LSTM) layer. In this work-in-progress we show the exemplary discrimination of idle from the use of scissors, knife, and hammer with a precision of 70.75%. As workers already wear gloves, we believe commercial smart gloves to be highly practical for similar applications.