In this paper, a comprehensive study on the development and implementation of Augmented Reality (AR) glasses designed to provide real-time text conversion of live speech for individuals with hearing impairments is presented. The work aims to break communication barriers, enhance accessibility, and promote inclusivity by merging AR technology with speech recognition and natural language processing. These glasses offer live subtitles, improving communication access and confidence for the deaf and hard-of-hearing community. The technical implementation involves utilizing a speech recognition AI model Whisper to transcribe audio into text and displaying onto the glasses via specular reflection from an OLED display through NodeMCU (ESP32), which is reflected on the mirror in front of the glass. The system effectively converts speech to text, enhancing communication and accessibility. The novelty of this work lies in designing the live audio transcription glasses to show the text clearly on the glass and including offline functionality for transcription.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Assistive Live Audio Transcription Glasses for Individuals Suffering Auditory Impairment

  • Siddharth Menon,
  • Aparna Padma Balaji,
  • Jayant Sasikumar,
  • Thazhai Mugunthan,
  • V. Ravikumar Pandi,
  • Soumya Sathyan,
  • Vipina Valsan,
  • Kavya Suresh

摘要

In this paper, a comprehensive study on the development and implementation of Augmented Reality (AR) glasses designed to provide real-time text conversion of live speech for individuals with hearing impairments is presented. The work aims to break communication barriers, enhance accessibility, and promote inclusivity by merging AR technology with speech recognition and natural language processing. These glasses offer live subtitles, improving communication access and confidence for the deaf and hard-of-hearing community. The technical implementation involves utilizing a speech recognition AI model Whisper to transcribe audio into text and displaying onto the glasses via specular reflection from an OLED display through NodeMCU (ESP32), which is reflected on the mirror in front of the glass. The system effectively converts speech to text, enhancing communication and accessibility. The novelty of this work lies in designing the live audio transcription glasses to show the text clearly on the glass and including offline functionality for transcription.