EmpowerSpeak: “A Breakthrough Model for Speech Synthesis from Videos for the Differently Abled”
摘要
Living things can get information by using a variety of senses to perceive their environment. Humans, for instance, interpret what they perceive and utilize that knowledge to make sense of their environment and interact with one another. This made us investigate how computers might be used to accomplish this and how it might benefit those who are unable to sense their surroundings. Therefore, our goal is to develop a system that can provide text and audio descriptions of videos for individuals with disabilities. Existing systems rely on neural networks like LSTM. On the other hand, LSTMs have difficulty comprehending lengthy sequences, which is necessary for creating descriptions for videos. Transformers have also gained popularity recently because of their ability to process data in parallel; nonetheless, they have memory and temporal problems. Furthermore, existing systems do not prioritize speech from an application-oriented standpoint, which limits the accessibility of these technologies. Therefore, we provide a system that generates textual and audio descriptions by utilizing several transformer models. Additionally, we offer a web interface via which you may upload films and create textual and voice descriptions for them. Taking things a step further, this work can be expanded to provide an interactive user interface that leverages the above-mentioned models to generate and read out loud descriptions of live video captured from a camera.