This study offers a hands-on solution to help bridge communication gaps for individuals who are Deaf or Hard of Hearing. DIST (Dynamic IoT-Enabled Sign-to-Text) is designed, which interprets sign language into text in real time. It runs on small, affordable devices like the Raspberry Pi, allowing gesture processing to happen directly on the device without needing internet access. To capture and understand gestures accurately, MediaPipe is used to track hand and facial movements. DIST dataset is built, called DSS, which focuses on dynamic signs rather than static ones—this makes recognition more accurate. The system works by using a camera to record gestures, then it analyzes the movement and converts it into readable text. This is shown on a DIST interface, and users can also choose to hear the text using a built-in text-to-speech function. An important feature is the user feedback, which lets users correct mistakes. These corrections help improve the system over time. Because the model continues to learn and adapt, it performs better in real-life situations. This work supports broader efforts in the UAE to promote inclusive technology and communication for all.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dynamic IoT-Enabled Sign-to-Text System (DIST) for Inclusive and Sustainable Communication in Deaf Communities

  • Samar Mouti,
  • Hani Alchalabi,
  • Sulafa Abdalla,
  • Samer Rihawi,
  • Mohamed Abushohada

摘要

This study offers a hands-on solution to help bridge communication gaps for individuals who are Deaf or Hard of Hearing. DIST (Dynamic IoT-Enabled Sign-to-Text) is designed, which interprets sign language into text in real time. It runs on small, affordable devices like the Raspberry Pi, allowing gesture processing to happen directly on the device without needing internet access. To capture and understand gestures accurately, MediaPipe is used to track hand and facial movements. DIST dataset is built, called DSS, which focuses on dynamic signs rather than static ones—this makes recognition more accurate. The system works by using a camera to record gestures, then it analyzes the movement and converts it into readable text. This is shown on a DIST interface, and users can also choose to hear the text using a built-in text-to-speech function. An important feature is the user feedback, which lets users correct mistakes. These corrections help improve the system over time. Because the model continues to learn and adapt, it performs better in real-life situations. This work supports broader efforts in the UAE to promote inclusive technology and communication for all.