Hand Gesture Recognition Applied to the Interaction with Video Games

In this work, a hand gesture recognition system was created for 11 different gestures. The system employed CNN-LSTM artificial neural networks and followed the CRISP-ML(Q) process model. The aim was to incorporate software engineering practices into machine learning projects. The system uses Electromyography (EMG) and Inertial Measurement Unit (IMU) signals as input to compute a gesture label and the time of occurrence in the signal. The system is integrated with a video game that utilizes hand gestures as input. A system usability scale (SUS) survey was done by ten final users in order to measure the interaction with the video game using gestures as the main way of interaction. The complete application evaluation obtained a SUS score of 75, or a B grade.