DiagramVoice: Automatic Lecture Video Commentator for Visually Impaired Students Supporting Diagram Commentary

Recently, due to the COVID-19 pandemic, the utilization of non-face-to-face lectures has increased. Online lectures use a lot of visual materials, and blind college students experience difficulties in understanding the lecture content. This causes them to lose concentration and interest in the class. This paper proposes an automatic lecture video commentary system, “DiagramVoice”, to ensure an independent learning environment for visually impaired students. It is designed as a mobile application to be easily accessible to the visually impaired. It can provide analysis of text and nature photographs among visual materials in lecture videos and also provides commentary on diagrams through the diagram commentary generation algorithm developed in this paper. To verify the practical usability of our DiagramVoice, we conducted user tests on the results of the diagram commentary generation algorithm. https://github.com/nan0silver/DiagramAnalysisGenerationAlgorithm .