Scholarlink

Communication barriers have a major influence on the deaf and mute society in India, resulting in social isolation and restricted access to education, employment, and everyday interactions. Indian Sign Language (ISL) is the primary mode of communication, but its lack of widespread understanding restricts integration with the larger society. This study presents a real-time ISL recognition and translation system that integrates deep learning, spatio-temporal analysis, and natural language processing (NLP) to overcome this communication barrier. This study proposes a real-time ISL gesture recognition and translation system utilizing Long Short-Term Memory (LSTM) networks, which are ideal for sequential gesture recognition so that accurate mapping of static and dynamic ISL gestures into text and speech can be done. A spatio-temporal feature extraction pipeline is incorporated using MediaPipe-based skeletal keypoint detection to guarantee strong recognition through capturing hand, facial, and body landmarks. The dataset, created with deaf and mute people’s inputs, provides regional gesture diversity and sign diversity. It has been engineered to operate effectively in real-world environments, with adaptations to lighting changes, background noise, and the complexity of gestures. This work contributes to assistive technology, accessibility, and human computer interaction, fostering social inclusion through facilitating effective communication between the hearing and non-hearing populations. This paper is a step towards a more inclusive digital communication environment, empowering the deaf community in various aspects of life.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Advancing Educational Inclusion: Integrating Indian Sign Language with Spoken Language Through LSTM Neural Networks

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Aaron Mendonca,
  • Arya Gawde,
  • Nikki Mehta,
  • Nilay Koul,
  • Rohit Parmar,
  • Nikita Raichada

摘要

Communication barriers have a major influence on the deaf and mute society in India, resulting in social isolation and restricted access to education, employment, and everyday interactions. Indian Sign Language (ISL) is the primary mode of communication, but its lack of widespread understanding restricts integration with the larger society. This study presents a real-time ISL recognition and translation system that integrates deep learning, spatio-temporal analysis, and natural language processing (NLP) to overcome this communication barrier. This study proposes a real-time ISL gesture recognition and translation system utilizing Long Short-Term Memory (LSTM) networks, which are ideal for sequential gesture recognition so that accurate mapping of static and dynamic ISL gestures into text and speech can be done. A spatio-temporal feature extraction pipeline is incorporated using MediaPipe-based skeletal keypoint detection to guarantee strong recognition through capturing hand, facial, and body landmarks. The dataset, created with deaf and mute people’s inputs, provides regional gesture diversity and sign diversity. It has been engineered to operate effectively in real-world environments, with adaptations to lighting changes, background noise, and the complexity of gestures. This work contributes to assistive technology, accessibility, and human computer interaction, fostering social inclusion through facilitating effective communication between the hearing and non-hearing populations. This paper is a step towards a more inclusive digital communication environment, empowering the deaf community in various aspects of life.