Scholarlink

<p>This paper presents a vision transformer-based, context-aware, real-time Indian Sign Language (ISL) conversational agent designed to enhance digital accessibility for India’s Deaf and Hard-of-Hearing community within e-governance services. The system supports continuous sign language recognition, from isolated words to full ISL sentences, and delivers information and services directly in sign language. A domain-specific ISL video dataset, incorporating diverse signing styles and environments, addresses ISL’s low-resource challenges, enabling robust, scalable real-world deployment. The hybrid architecture, combining convolutional neural networks with vision transformer models, effectively handles ISL’s spatial–temporal complexities, maintaining reliability even with complex queries. A dynamic context-response mapping engine uses contextual data to increase accuracy, particularly for ambiguous inputs. The modular design ensures efficient scalability, facilitating seamless integration of new services. System evaluations, including stress testing and usability studies, confirmed its effectiveness in enabling real-time, inclusive digital interactions. Under optimized conditions, the system achieved 97.5% accuracy, a mean response time of 6.53&#xa0;s, and an average system usability score of 71.83, significantly advancing digital inclusion in India.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Vision transformer-powered conversational agent for real-time Indian Sign Language e-governance accessibility

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年9月26日

查看全文
  • Prema Nedungadi,
  • Gautham Dileep,
  • M. Geetha,
  • Raghu Raman

摘要

This paper presents a vision transformer-based, context-aware, real-time Indian Sign Language (ISL) conversational agent designed to enhance digital accessibility for India’s Deaf and Hard-of-Hearing community within e-governance services. The system supports continuous sign language recognition, from isolated words to full ISL sentences, and delivers information and services directly in sign language. A domain-specific ISL video dataset, incorporating diverse signing styles and environments, addresses ISL’s low-resource challenges, enabling robust, scalable real-world deployment. The hybrid architecture, combining convolutional neural networks with vision transformer models, effectively handles ISL’s spatial–temporal complexities, maintaining reliability even with complex queries. A dynamic context-response mapping engine uses contextual data to increase accuracy, particularly for ambiguous inputs. The modular design ensures efficient scalability, facilitating seamless integration of new services. System evaluations, including stress testing and usability studies, confirmed its effectiveness in enabling real-time, inclusive digital interactions. Under optimized conditions, the system achieved 97.5% accuracy, a mean response time of 6.53 s, and an average system usability score of 71.83, significantly advancing digital inclusion in India.