An Open-Source Presentation Video Retrieval System Using Transformers
摘要
We present an open-source presentation video retrieval system, which performs automatic video content analysis of a presentation video (e.g., a lecture or talk) using state-of-the-art transformers for OCR and ASR and a specific shot detection approach. It automatically recognizes text and speech in the video and indexes it with the corresponding keyframe, enabling content-based retrieval. By combining OCR and ASR with a simplistic, efficient UI, we provide an effective and easy-to-use solution to retrieve specific video segments from large collections of lecture videos. Users can query for specific video segments displayed by representative keyframes. In this paper, we compare our system to related work and give an in-depth description of the functionalities and frameworks used to build the system. Additionally, we discuss the systems’ architecture and present a preliminary evaluation of our system, which shows promising results for different types of presentation videos. The entire system is released as an open-source tool and can be freely used by all colleagues who want to make their presentation videos searchable.