Design and Development of a Deep Learning-Based Sign Language Learning Aid for Deaf Teenagers

In this paper, we developed HearUsNow, a deep learning-based system designed to assist deaf teenagers in correcting speech order and learning sign language, thereby improving their communication skills. This interactive system incorporates a deep learning-based back-end with a user-friendly front-end interface, aiming to bridge the gap between these individuals and their real-world communication needs. In this system, deaf teenagers receive corrections on sentence structure and accompanying sign language instructional images after entering sentences, and sign language practice is facilitated through a camera interface that provides real-time feedback. The results of a pilot study (n = 6) showed high usability, interaction fluency, and accuracy of correction and recognition of the interactive system. Over a seven-day period, the participants mastered more than 30 gesture combinations for everyday communication, and parents noted improved self-confidence and parent-child relationships.