Sign language represents an essential communication tool that benefits the Deaf and hard-of-hearing community, yet the lack of effective communication with non-signers often leads to social isolation. This research emphasizes on the development of an application using Tkinter for recognizing words in American Sign Language (ASL) through fingerspelling and turning them into sentences. It presents a novel approach to converting American Sign Language into text in real time, utilizing cutting-edge computer vision and deep learning techniques. The proposed method uses a webcam-based self-made dataset and a custom model with the InceptionV3 architecture as the basic model to categorize the letters via transfer learning. The recognized letters were then progressively combined to form words, which were further computed into meaningful sentences. Our approach focuses on improving recognition accuracy while maintaining real-time processing for minimizing the communication gap and contributing to promote social inclusivity of the Deaf community.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Bridging the Gap: Real-Time ASL Fingerspelling to Sentence Translation

  • Manya Chandna,
  • Abantika Dasgupta,
  • Bhumika Gupta,
  • M. Ravinder,
  • S. R. N. Reddy,
  • Rishika Anand

摘要

Sign language represents an essential communication tool that benefits the Deaf and hard-of-hearing community, yet the lack of effective communication with non-signers often leads to social isolation. This research emphasizes on the development of an application using Tkinter for recognizing words in American Sign Language (ASL) through fingerspelling and turning them into sentences. It presents a novel approach to converting American Sign Language into text in real time, utilizing cutting-edge computer vision and deep learning techniques. The proposed method uses a webcam-based self-made dataset and a custom model with the InceptionV3 architecture as the basic model to categorize the letters via transfer learning. The recognized letters were then progressively combined to form words, which were further computed into meaningful sentences. Our approach focuses on improving recognition accuracy while maintaining real-time processing for minimizing the communication gap and contributing to promote social inclusivity of the Deaf community.