We present an open-source presentation video retrieval system, which performs automatic video content analysis of a presentation video (e.g., a lecture or talk) using state-of-the-art transformers for OCR and ASR and a specific shot detection approach. It automatically recognizes text and speech in the video and indexes it with the corresponding keyframe, enabling content-based retrieval. By combining OCR and ASR with a simplistic, efficient UI, we provide an effective and easy-to-use solution to retrieve specific video segments from large collections of lecture videos. Users can query for specific video segments displayed by representative keyframes. In this paper, we compare our system to related work and give an in-depth description of the functionalities and frameworks used to build the system. Additionally, we discuss the systems’ architecture and present a preliminary evaluation of our system, which shows promising results for different types of presentation videos. The entire system is released as an open-source tool and can be freely used by all colleagues who want to make their presentation videos searchable.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Open-Source Presentation Video Retrieval System Using Transformers

  • Mario Leopold,
  • Klaus Schoeffmann

摘要

We present an open-source presentation video retrieval system, which performs automatic video content analysis of a presentation video (e.g., a lecture or talk) using state-of-the-art transformers for OCR and ASR and a specific shot detection approach. It automatically recognizes text and speech in the video and indexes it with the corresponding keyframe, enabling content-based retrieval. By combining OCR and ASR with a simplistic, efficient UI, we provide an effective and easy-to-use solution to retrieve specific video segments from large collections of lecture videos. Users can query for specific video segments displayed by representative keyframes. In this paper, we compare our system to related work and give an in-depth description of the functionalities and frameworks used to build the system. Additionally, we discuss the systems’ architecture and present a preliminary evaluation of our system, which shows promising results for different types of presentation videos. The entire system is released as an open-source tool and can be freely used by all colleagues who want to make their presentation videos searchable.