As the volume of video data grows, efficient retrieval has become crucial for applications in security and multimedia management. Our system, ReViMM, is designed to address this challenge by employing a novel reweighting mechanism that enhances query accuracy. This mechanism dynamically prioritizes essential elements within each query, significantly improving relevance and precision in search results. ReViMM integrates FAISS for similarity search, ElasticSearch for optimized indexing, and Whisper for robust speech transcription, allowing it to process complex multimedia inputs, including text, images, and audio. By combining these tools with large language models to generate precise descriptions, our system delivers a comprehensive solution for accurate, context-sensitive video and multimedia retrieval.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ReViMM: Enhanced Video Retrieval with Reweighting Mechanism for Multi-modal Queries

  • To Anh Phat,
  • Truong Thanh Minh,
  • Doan Nguyen Tran Hoan,
  • Khanh-Duy Nguyen

摘要

As the volume of video data grows, efficient retrieval has become crucial for applications in security and multimedia management. Our system, ReViMM, is designed to address this challenge by employing a novel reweighting mechanism that enhances query accuracy. This mechanism dynamically prioritizes essential elements within each query, significantly improving relevance and precision in search results. ReViMM integrates FAISS for similarity search, ElasticSearch for optimized indexing, and Whisper for robust speech transcription, allowing it to process complex multimedia inputs, including text, images, and audio. By combining these tools with large language models to generate precise descriptions, our system delivers a comprehensive solution for accurate, context-sensitive video and multimedia retrieval.