VisualVoice: Real-World Implementation of a Deep Learning-Based Assistive App for Visually Impaired Individuals
摘要
The digital age, in which we live today, has not done much by way of helping a visually impaired person understand or access images. With this challenge in context, this paper, VisualVoice: An Image captioning app with audio descriptions for visually impaired people, whose production focuses on addressing the current scenario prevalent. This solution we are building will translate images into direct and easily understood audio descriptors. The image processing in the project is carried out by OpenCV and the system is trained using the Flickr dataset. Convolutional Neural Networks (CNNs), a deep learning model employed for image features extraction, and Long Short-Term Memory networks, used to create valuable captions. The models are built using TensorFlow for robustness and they are trained on Google Colab. At the backend, Flask in Python is used which ensures smooth integration. It takes the support of NLTK for encoding to extend a library that produces captions correctly, Also, SQLAlchemy is used to perform database operations. The User Interface of the frontend is created using ReactJs and HTML & CSS, for a better user-friendly experience. One of the most prominent features makes use of Google Text-to-Speech (gTTS) to convert text, so that the blind individuals can get a detailed description about the surrounding environments. The project illustrates how with the advancement of deep learning, technology can be used more seamlessly by persons who are visually impaired to interact better in their world.