Individuals with speech impairments often face significant communication challenges, requiring innovative solutions. With over 300+ sign languages globally, only about 3% are widely recognized, such as American, British, and Indian Sign Language. This research introduces a pioneering Augmentative and Alternative Communication (AAC) device designed for easy wearability around the neck, integrating a camera, Raspberry Pi, and Google’s Text-to-Speech technology. This device enables users to create and utilize their own sign languages, enhancing communication with family and the broader community without needing to learn standardized sign languages. The device is complemented by a user-friendly mobile application that simplifies device management and empowers users to create and customize their own sign languages. Additionally, it contributes to filling a significant research gap by generating a unique dataset of first-person perspective sign language images, supporting more accurate and personalized communication solutions. The effectiveness of this device has been validated using machine learning models, including a random forest classifier and a convolutional neural network (CNN), achieving high accuracy levels (84–99%) in translating personalized sign language gestures into speech. This approach marks a substantial advancement over existing methods, offering a novel communication aid for individuals with speech impairments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Customizable AAC Device: ML-Enhanced Speech Aid for Impairment

  • Daksh Kitukale,
  • Gresey Patidar,
  • Vandana Kate,
  • Chanchal Bansal,
  • Mukund Solanki,
  • Deepansh Jain

摘要

Individuals with speech impairments often face significant communication challenges, requiring innovative solutions. With over 300+ sign languages globally, only about 3% are widely recognized, such as American, British, and Indian Sign Language. This research introduces a pioneering Augmentative and Alternative Communication (AAC) device designed for easy wearability around the neck, integrating a camera, Raspberry Pi, and Google’s Text-to-Speech technology. This device enables users to create and utilize their own sign languages, enhancing communication with family and the broader community without needing to learn standardized sign languages. The device is complemented by a user-friendly mobile application that simplifies device management and empowers users to create and customize their own sign languages. Additionally, it contributes to filling a significant research gap by generating a unique dataset of first-person perspective sign language images, supporting more accurate and personalized communication solutions. The effectiveness of this device has been validated using machine learning models, including a random forest classifier and a convolutional neural network (CNN), achieving high accuracy levels (84–99%) in translating personalized sign language gestures into speech. This approach marks a substantial advancement over existing methods, offering a novel communication aid for individuals with speech impairments.