CESE: A Clip-Based Event Search Engine for AI Challenge HCMC 2024
摘要
In the context of video event retrieval, identifying specific events in vast visual data collections is a critical challenge. This paper presents CESE, our solution for the AI Challenge HCMC 2024, which focuses on retrieving video clips based on multiple queries rather than individual frames based on a single query. CESE employs a dual-module architecture: FrameClipping and TextClipping, designed to address visual and textual data, respectively. FrameClipping uses a Visual Semantic Clipping algorithm based on keyframe extraction and visual embeddings, while TextClipping introduces both Textual Semantic Clipping for sentence-based queries and Keywords Clipping for keyword-only queries. The system integrates a lightweight keyframe extraction pipeline, optimizing the processing of large video datasets. We also leverage Gemini 1.5 Flash for context-aware caption generation, ensuring detailed and coherent event descriptions. Additionally, CESE offers a keyboard-driven interface for faster, more efficient retrieval by minimizing mouse interactions. Our approach significantly improves retrieval accuracy and user experience by returning meaningful video segments that encapsulate the described events. CESE represents a scalable and efficient solution for multimodal event search in competitive settings.