Robot Control Using Hand Gestures of the Mexican Sign Language

The pursuit of process automation remains a persistent objective across various industries, including manufacturing, agriculture, and medicine. This drive compels us to continually refine and innovate tools capable of handling intricate tasks. Concurrently, as we enhance these tools, the dynamics of human-machine interaction (HMI) evolve. Hand gestures present a promising avenue for HMI due to their innate adaptability and ease of use. Integrating Hand Gesture Recognition (HGR) into the control systems of devices and robots has the potential to streamline operations and simplify complex tasks. In this study, we introduce a methodology aimed at enhancing the control of mobile robots using hand gestures to convey movement instructions. Specifically, we leverage gestures from the Mexican Sign Language alphabet as a means of communication. From the 29 alphabet signs, we selected 12 that are distinct from each other and easy to perform and consist only static signs. Each gesture corresponds to a specific movement command for the robot. Geometric features are extracted from images, and in our experiments, we implement convolutional neural networks to classify the gestures. We also use four other classifiers to compare results.