Multimodal Social Interaction for Speaking Disabilities Care
摘要
This chapter presents an innovative web application designed to improve communication for individuals with speech disabilities, particularly within the Sinhala-speaking community. The application addresses the communication barriers faced by the deaf-mute and mutism populations, which hinder their engagement on digital and social platforms. Traditional methods, such as sign and miming languages, are vital but often misunderstood by those unfamiliar with them, increasing isolation for these individuals. The research offers an AI-driven solution to overcome these barriers, promoting inclusivity and understanding. The system comprises two main components. The first is a meeting platform with modules for Miming Language to Sinhala Text transcription and Signing Language to Sinhala Text transcription, enabling individuals to convey messages in written Sinhala. Additionally, a Sinhala Audio Conversion to Miming Language module provides avatar-based visual representation and text transcription for the deaf-mute community to participate in digital conversations. The second component is an AI-assisted Sign Language Trainer that offers an interactive environment for learning or enhancing proficiency in Sinhala sign language.