Real-Time and on-the-Edge Multiple Channel Capacitive and Inertial Fusion-Based Glove

Human-robot interaction (HRI) has become increasingly ubiquitous and pervasive, with hand gestures serving as a common means of communication between humans and robots. We present a glove-based wearable solution for classifying hand gestures. It is specifically designed for drone control applications. Our solution is a textile-based, low-power consumption ( \(\le \) 1.15 W), privacy-conscious, real-time, and on-the-Edge alternative, ensuring user comfort and data security. With a tiny memory footprint ( \(\le \) 2 MB), the proposed approach provides an efficient and sustainable approach to hand gesture recognition. To achieve high accuracy while maintaining low power consumption, we adopt a hierarchical lightweight neural network scheme. This fusion technique not only reduces energy consumption but also improves the overall performance of the system. During the offline evaluation of nine classes, including eight hand gestures and the null class, our system achieves an F1 score of 80%. In real-time and on-the-edge evaluation with one user, our wearable solution yields an F1 score of 67%, further highlighting its practicality and effectiveness in real-world scenarios.