Cloud-Based Real-Time Sign Language Detection and Voice Synthesis for Impaired Individuals Using Machine Learning

Individuals with hearing and speech impairments find it harder to communicate in real time as they can primarily speak sign language and the general public usually requires a translator for them to understand. Therefore, there is a requirement for a system that translates the sign language gestures into audible speech in real-time, ensuring them an opportunity to communicate. This research paper presents a cloud-based solution that offers real-time sign language recognition and text-to-speech conversion websites leveraging a Random Forest classifier. The user can turn on their webcam to detect sign language in real-time and display each letter recognized from hand landmarks detected by the cloud-deployed Random Forest model and store it in a text file. Once the recording of the video is done, the user is immediately provided with an audio file of the converted text file, through an on-screen MP3 player, that is natural and audible enough for enabling smooth communication. Through extensive testing and experimentation, the Random Forest model was found to have an accuracy of 88.2% on testing data and made quite accurate predictions. In conclusion, the paper demonstrates an intuitive and reliable tool for effective communication for impaired individuals, by aiming to empower them with the means to participate fully in the digital age.