<p>Natural language video localization (NLVL), a crucial task in video understanding, localizes target moments in videos corresponding to a language description. The point-supervised paradigm, which requires only a single annotated frame instead of complete temporal boundaries, offers a favorable balance between performance and annotation cost. However, the absence of full annotations impedes effective video-language alignment, thereby reducing prediction accuracy. To overcome this limitation, we propose COTEL (COllaborative Temporal consistEncy Learning), a framework that enhances alignment by leveraging the synergy between saliency detection and moment localization. COTEL integrates frame-level and segment-level consistency learning branches, which are trained to reinforce each other via a cross-consistency guidance scheme. Furthermore, we introduce a hierarchical contrastive alignment loss that combines intra-video positive alignment with inter-video negative mining to achieve comprehensive video-text matching. Extensive experiments on three widely used benchmarks demonstrate that our method performs favorably against state-of-the-art approaches. All source code will be released.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Collaborative Temporal Consistency Learning for Point-supervised Natural Language Video Localization

  • Zhuo Tao,
  • Liang Li,
  • Qi Chen,
  • Yunbin Tu,
  • Zheng-Jun Zha,
  • Amin Beheshti,
  • Qingming Huang,
  • Yuankai Qi,
  • Ming-Hsuan Yang

摘要

Natural language video localization (NLVL), a crucial task in video understanding, localizes target moments in videos corresponding to a language description. The point-supervised paradigm, which requires only a single annotated frame instead of complete temporal boundaries, offers a favorable balance between performance and annotation cost. However, the absence of full annotations impedes effective video-language alignment, thereby reducing prediction accuracy. To overcome this limitation, we propose COTEL (COllaborative Temporal consistEncy Learning), a framework that enhances alignment by leveraging the synergy between saliency detection and moment localization. COTEL integrates frame-level and segment-level consistency learning branches, which are trained to reinforce each other via a cross-consistency guidance scheme. Furthermore, we introduce a hierarchical contrastive alignment loss that combines intra-video positive alignment with inter-video negative mining to achieve comprehensive video-text matching. Extensive experiments on three widely used benchmarks demonstrate that our method performs favorably against state-of-the-art approaches. All source code will be released.