AI-Based Services to Support Language-Learning for Deaf and Hearing Individuals in Immersive XR Settings
摘要
This study presents a suite of AI-based services, including speech-to-text, text-to-speech, and text-to-sign translation, designed to support language learning for both deaf and hearing individuals in immersive Extended Reality (XR) environments. In particular, it explores and develops speech-to-text, text-to-speech, and text-to-sign translation technologies, to enable inclusive, multimodal communication tailored to diverse user needs. These services become available via APIs with the goal to be explored by 3D avatars within Extended Reality (XR) settings, delivering engaging, self-directed learning experiences. The paper emphasizes the importance of modular and reusable design, and proposes a framework to ensure interoperability and scalability across various XR platforms, addressing critical challenges in accessibility and multilingual interaction, opening new opportunities in the field for equitable educational solutions in immersive environments.