Towards a Real-Time Webcam Feed Narrator Using Multimodal Language Models and Speech Synthesis
摘要
This work introduces the design and development of a real-time system that records webcam video frames, translates them with large multimodal language models (LLMs), and renders the translated content as expressive speech. The system utilizes vision-language models like LLaVA and adds a stylized personality filter, to produce witty, engaging commentary. Core pieces of the framework are frame capture through OpenCV, visual-to-text conversion using LLMs, and voice synthesis through pyttsx3. We also investigate the use of this framework in assistive technologies for the visually impaired by modifying the personality tone and enhancing descriptive fidelity. Testing proves successful end-to-end narration within five seconds per frame cycle. Ethical and social impacts of anthropomorphized AI interfaces are also briefly taken into account.