Scholarlink

<p>Adapting large-scale vision–language models (VLMs) to real-time video action recognition is challenging under strict latency constraints and limited tuning budgets. Most parameter-efficient fine-tuning (PEFT) methods apply time-agnostic, frame-wise adaptations that underexploit motion cues, while explicit temporal modeling often relies on single-tower adaptations that entangle appearance and motion or dual-tower adaptations that add redundant computation with weak cross-stream interaction. In this paper, we propose <i>HyBridge</i>, a light-weight adaptation framework that performs <i>shallow spatial-temporal decoupling</i> and <i>deep semantic recoupling</i> for efficient video temporal modeling. Specifically, HyBridge splits early layers into a spatial RGB stream and a temporal RGB-diff stream, and progressively fuses them through lightweight fusion adaptation layers to encourage hierarchical interaction. Both streams employ learnable visual prompts and adapters, and we further introduce learnable text adaptation to improve video-text alignment while keeping all original VLM parameters frozen. Experiments on four benchmarks demonstrate that HyBridge achieves a strong accuracy–efficiency trade-off for practical real-time video understanding.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HyBridge: hybrid decoupling-to-recoupling adaptation of vision-language models for real-time video action recognition

  • 期刊论文

  • 发表日期：

    2026年5月28日

查看全文
  • Mengyu Yang,
  • Ye Tian,
  • Lanshan Zhang,
  • Gongli Xi,
  • Wendong Wang

摘要

Adapting large-scale vision–language models (VLMs) to real-time video action recognition is challenging under strict latency constraints and limited tuning budgets. Most parameter-efficient fine-tuning (PEFT) methods apply time-agnostic, frame-wise adaptations that underexploit motion cues, while explicit temporal modeling often relies on single-tower adaptations that entangle appearance and motion or dual-tower adaptations that add redundant computation with weak cross-stream interaction. In this paper, we propose HyBridge, a light-weight adaptation framework that performs shallow spatial-temporal decoupling and deep semantic recoupling for efficient video temporal modeling. Specifically, HyBridge splits early layers into a spatial RGB stream and a temporal RGB-diff stream, and progressively fuses them through lightweight fusion adaptation layers to encourage hierarchical interaction. Both streams employ learnable visual prompts and adapters, and we further introduce learnable text adaptation to improve video-text alignment while keeping all original VLM parameters frozen. Experiments on four benchmarks demonstrate that HyBridge achieves a strong accuracy–efficiency trade-off for practical real-time video understanding.