Scholarlink

<p>Video lectures and tutorials have become increasingly popular worldwide with the widespread availability of the internet. While online captions are commonly provided for spoken content, significant challenges arise when processing mathematical expressions (MEs). Captions typically convert spoken equations into linear text, losing the essential 2D structure and spatial relationships of handwritten MEs. This limitation significantly affects understanding, especially in academic and technical learning environments. The proposed work focuses on detecting and recognizing handwritten mathematical equations in video lectures, where instructors write equations during their presentations. YOLOv8 is employed to accurately detect handwritten equations within complex video frames that may contain images, text, and graphs. Detected equations are recognized using a deep learning architecture comprising a CNN encoder and an attention-based RNN decoder, which converts handwritten MEs into LaTeX format to preserve their structural and semantic integrity. To enhance processing efficiency, redundant frames are filtered using perceptual hashing combined with deep similarity models, ensuring that unique frames are processed. YOLOv8 is further utilized for tracking equations across consecutive frames, promoting temporal coherence and minimizing redundant recognition. The system is evaluated on benchmark datasets such as CROHME 2014 and CROHME 2019, with performance measured using word error rate (WER). Additional evaluations on real-world YouTube videos containing mathematical content demonstrate the robustness and practical applicability of the approach in educational scenarios. The method offers a scalable and accurate solution for enhancing the accessibility and clarity of mathematical content in video-based learning platforms.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Bridging captioning gaps in educational videos: a deep learning approach for handwritten mathematical expression recognition

  • 期刊论文

  • 发表日期：

    2026年4月8日

查看全文
  • Sabthami J,
  • Ramani Boothalingam,
  • Vijayalakshmi P

摘要

Video lectures and tutorials have become increasingly popular worldwide with the widespread availability of the internet. While online captions are commonly provided for spoken content, significant challenges arise when processing mathematical expressions (MEs). Captions typically convert spoken equations into linear text, losing the essential 2D structure and spatial relationships of handwritten MEs. This limitation significantly affects understanding, especially in academic and technical learning environments. The proposed work focuses on detecting and recognizing handwritten mathematical equations in video lectures, where instructors write equations during their presentations. YOLOv8 is employed to accurately detect handwritten equations within complex video frames that may contain images, text, and graphs. Detected equations are recognized using a deep learning architecture comprising a CNN encoder and an attention-based RNN decoder, which converts handwritten MEs into LaTeX format to preserve their structural and semantic integrity. To enhance processing efficiency, redundant frames are filtered using perceptual hashing combined with deep similarity models, ensuring that unique frames are processed. YOLOv8 is further utilized for tracking equations across consecutive frames, promoting temporal coherence and minimizing redundant recognition. The system is evaluated on benchmark datasets such as CROHME 2014 and CROHME 2019, with performance measured using word error rate (WER). Additional evaluations on real-world YouTube videos containing mathematical content demonstrate the robustness and practical applicability of the approach in educational scenarios. The method offers a scalable and accurate solution for enhancing the accessibility and clarity of mathematical content in video-based learning platforms.