Scholarlink

This research aims to enhance sustainable communication accessibility for individuals who are Deaf or Hard of Hearing (DHH) by developing an IoT-based Sign-to-Text (STT) system. The system’s primary objective is to provide a real-time, low-energy solution for translating sign language into text, fostering inclusive and equitable communication between Deaf and hearing individuals. The methodology employs Convolutional Neural Networks (CNNs) for hand gesture recognition and Natural Language Processing (NLP) to convert recognized gestures into coherent text. The system is built on a Raspberry Pi platform, using the MediaPipe library to detect facial and hand landmarks and Long Short-Term Memory (LSTM) networks to predict corresponding text. Testing with deaf users yielded an 80% accuracy rate, although challenges remain in differentiating similar gestures. The system’s potential to bridge communication gaps aligns with the UAE’s inclusive initiatives. Future research will focus on expanding gesture datasets and integrating multi-language support to improve adaptability.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

IoT-Driven Sign-to-Text Systems for Sustainable Accessibility in Deaf and Hard of Hearing Communities

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Samar Mouti,
  • Samer Rihawi,
  • Mohammed Abushohada,
  • Hani Al Chalabi,
  • Sulafa Abdalla

摘要

This research aims to enhance sustainable communication accessibility for individuals who are Deaf or Hard of Hearing (DHH) by developing an IoT-based Sign-to-Text (STT) system. The system’s primary objective is to provide a real-time, low-energy solution for translating sign language into text, fostering inclusive and equitable communication between Deaf and hearing individuals. The methodology employs Convolutional Neural Networks (CNNs) for hand gesture recognition and Natural Language Processing (NLP) to convert recognized gestures into coherent text. The system is built on a Raspberry Pi platform, using the MediaPipe library to detect facial and hand landmarks and Long Short-Term Memory (LSTM) networks to predict corresponding text. Testing with deaf users yielded an 80% accuracy rate, although challenges remain in differentiating similar gestures. The system’s potential to bridge communication gaps aligns with the UAE’s inclusive initiatives. Future research will focus on expanding gesture datasets and integrating multi-language support to improve adaptability.