Real-Time Gesture Recognition Using Convolutional Neural Networks on Embedded Systems

One of the most rapidly growing areas that need real-time motion recognition to succeed is in smart technology and human–computer interaction. In this paper, we propose a novel approach that utilizes convolutional neural networks (CNNs) for real-time gesture identification on embedded systems. The ultimate aim of our research is to make interactions through gestures swift and smooth on relatively low-performance devices such as smartphones, IoT devices, and wearable technologies. CNNs are an excellent type of deep learning models specialized in processing images and videos; we have exploited this strength for our system. Using CNNs to extract and process spatial characteristics of input picture frames is an important element in getting good results in motion detection. The structure of such CNNs must be optimized not only for their performance but also for embedded systems, where memory and processing power are constrained resources. The use of network quantization and depth-wise separable convolutions is to have a system that works well with low latency while still being real-time capable without any loss of accuracy. We obtained a dataset of various hand gestures, both static and dynamic, to develop the CNN model. To enhance the reliability and generalization capabilities of the model, we applied state-of-the-art methods for data augmentation. The model can now recognize hand movements captured from videos, making it possible to use such a system for controlling devices with simple gestures. A video-based real-time motion detection system can be used on numerous embedded platforms, demonstrating its flexibility and versatility. It has been tested with the help of intelligent devices, going through user interfaces, and coping with virtual reality programs in practice. With the system having very little latency but high accuracy, it can be easily used by integrated devices to create a pleasant user experience.