A Software for Learning Thai Fingerspelling Sign Language into Thai Consonants with Machine Learning

Children with hearing impairment or deafness often face difficulties in spelling and reading. This research aims to develop a program that translates Thai fingerspelling sign language into Thai consonants. The objective is to use this program as a teaching tool to help children with hearing impairment understand and learn the Thai language more easily. The program created is a utilization of Artificial Intelligence (AI) techniques, where the program captures real-time hand gesture images in fingerspelling sign language and then translates the results into Thai consonants. It enables them to accurately create hand signs and makes the learning process more engaging and interesting. The program development begins with Hand Detection algorithm, which involves detecting the positions of 21 landmarks on the hand after using image processing techniques. After that, the hand gesture images for all the Thai consonants are captured and saved to be used as ground truth datasets. Next, the ground truth datasets are fed into the machine learning process to create a trained model. This enables the program to match the hand gesture images accurately with the correct consonant flashcards, using Google’s Teachable Machine. The results of testing the program revealed that the program has an average accuracy of 79.50%. The program was deployed with 12 users, consisting of 2 teachers and 10 elementary school students from Setsatian School for the Deaf under the Royal Patronage of His Royal Highness Crown Prince Maha Vajiralongkorn. After conducting the user satisfaction survey and applying statistical analysis methods, the results revealed an overall satisfaction rating of “very satisfactory,” scoring 83.80% across all aspects. The program developed can be used as a teaching aid for children with hearing impairments to facilitate their learning and academic progress.