This paper proposes an advanced architecture for the seamless inclusion of deaf people in virtual meeting seamlessly, coupled with growing attention towards inclusivity in digital communication. The novel architecture uses audio processing along with its transcription, translation and text summarization in real time with a sign language translator for completing a two-way communication channel between the participants which are deaf and hearing individuals in a virtual meeting. The system employs advance machine learning to provide accessible and scalable interaction in a digital environment. The proposed architecture is divided up into two stages, which are different and integrated efficiently. The first stage features actual time sounds from a hearing individual which undergoes noise filtration for clearer signals, the audio is then transcribed into text using pre-trained machine learning model, trained on a vast dataset featuring different dialect and speech pattern. This ensures complete accessibility, the system then translates, transcribe and summarizes the text for concise comprehension into a user-preferred language for the deaf participants to provide a personalized experience. The second part uses a camera module to register and recognize the gesture of Sign languages performed by the deaf participants using a LSTM-based neural network model. Here, the model provides an accurate interpretation of the gesture in textual format, thus completing a bidirectional and seamless communication bridge between the hearing and deaf participants. The key objective of this architecture is successfully enabling deaf individuals to participate in a virtual meeting environment promoting their inclusion in digital environments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-Time Virtual Meeting Architecture for Deaf People: Audio Transcription, Noise Processing and Sign Language Translation Using LSTM Neural Networks

  • Aaryan Chaurasia,
  • Sivakumar Rajagopal

摘要

This paper proposes an advanced architecture for the seamless inclusion of deaf people in virtual meeting seamlessly, coupled with growing attention towards inclusivity in digital communication. The novel architecture uses audio processing along with its transcription, translation and text summarization in real time with a sign language translator for completing a two-way communication channel between the participants which are deaf and hearing individuals in a virtual meeting. The system employs advance machine learning to provide accessible and scalable interaction in a digital environment. The proposed architecture is divided up into two stages, which are different and integrated efficiently. The first stage features actual time sounds from a hearing individual which undergoes noise filtration for clearer signals, the audio is then transcribed into text using pre-trained machine learning model, trained on a vast dataset featuring different dialect and speech pattern. This ensures complete accessibility, the system then translates, transcribe and summarizes the text for concise comprehension into a user-preferred language for the deaf participants to provide a personalized experience. The second part uses a camera module to register and recognize the gesture of Sign languages performed by the deaf participants using a LSTM-based neural network model. Here, the model provides an accurate interpretation of the gesture in textual format, thus completing a bidirectional and seamless communication bridge between the hearing and deaf participants. The key objective of this architecture is successfully enabling deaf individuals to participate in a virtual meeting environment promoting their inclusion in digital environments.