Fingerspelling Recognition in Mexican Sign Language (LSM) Using Machine Learning

Sign languages allow deaf people to express their thoughts, emotions, and opinions in a complex and complete way, just like oral languages. Each sign language is unique and has its own grammar, syntax, and vocabulary. Mexican Sign Language (LSM) is characterized by rich gestural and facial expression that gives it a great communicative and linguistic capacity. In the study of LSM, two main components have been identified: (i) fingerspelling, and (ii) ideograms. The first is similar to spelling in oral languages, and is used to communicate proper names, technical terms or words for which there are no specific signs or which are little known to the deaf community. In this paper, we propose a method for recognizing the LSM alphabet by using machine learning-based techniques capable of classifying the signs made by 10 test subjects. 21-keypoints of the hands were extracted from the MediaPipe library, in order to have a better representation to feed the classification models. The results when classifying the 21 letters exceeded an F1-score of 0.98 with 3 of the 4 trained classifiers, and scoring values below 0.95 for less than 3 letters. Tools such as those proposed in this work can facilitate seamless communication by translating Spanish into LSM and vice versa, allowing both communities to engage effectively in various settings.