Fine-Grained Cross-Modal Contrast Learning for Video-Text Retrieval

Video-sharing platforms emphasize video-text retrieval in multimodal information retrieval. Existing methods often overlook video text intricacies and redundancy, focusing mainly on single-granularity information. To address this, we propose Fine-grained Cross-modal Contrast Learning (FCCL), an end-to-end framework. FCCL includes a frame enhancement module to reduce data complexity by discerning key features from each video frame. Additionally, we introduce a multimodal attention model to identify text-similar video sub-regions accurately. We also intro-duce a multi-granularity discrepancy analysis model to capture cross-modal similarity across different levels, including video-sentence, frame-sentence, and frame-word perspectives. Experimental results on MSR-VTT and MSVD datasets demonstrate FCCL's superiority in video-text retrieval. Code is available at: https://github.com/LHlh917/FCCL .