This study presents a suite of AI-based services, including speech-to-text, text-to-speech, and text-to-sign translation, designed to support language learning for both deaf and hearing individuals in immersive Extended Reality (XR) environments. In particular, it explores and develops speech-to-text, text-to-speech, and text-to-sign translation technologies, to enable inclusive, multimodal communication tailored to diverse user needs. These services become available via APIs with the goal to be explored by 3D avatars within Extended Reality (XR) settings, delivering engaging, self-directed learning experiences. The paper emphasizes the importance of modular and reusable design, and proposes a framework to ensure interoperability and scalability across various XR platforms, addressing critical challenges in accessibility and multilingual interaction, opening new opportunities in the field for equitable educational solutions in immersive environments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AI-Based Services to Support Language-Learning for Deaf and Hearing Individuals in Immersive XR Settings

  • N. D. Tantaroudas,
  • A. J. McCracken,
  • I. Karachalios,
  • E. Papatheou

摘要

This study presents a suite of AI-based services, including speech-to-text, text-to-speech, and text-to-sign translation, designed to support language learning for both deaf and hearing individuals in immersive Extended Reality (XR) environments. In particular, it explores and develops speech-to-text, text-to-speech, and text-to-sign translation technologies, to enable inclusive, multimodal communication tailored to diverse user needs. These services become available via APIs with the goal to be explored by 3D avatars within Extended Reality (XR) settings, delivering engaging, self-directed learning experiences. The paper emphasizes the importance of modular and reusable design, and proposes a framework to ensure interoperability and scalability across various XR platforms, addressing critical challenges in accessibility and multilingual interaction, opening new opportunities in the field for equitable educational solutions in immersive environments.