<p>Open-vocabulary object detection (OVD) requires detectors to align image regions with text-defined categories while remaining computationally practical. In existing systems, stronger cross-modal reasoning often improves recognition of rare or fine-grained categories, but typically increases inference cost. We present TG-OverLoCK, an OverLoCK-based extension for OVD that selectively concentrates language-conditioned refinement capacity in an overview-to-focus pathway, rather than inserting uniformly dense cross-modal fusion throughout the backbone. The model combines a lightweight overview stage with a deeper focus stage. The overview stage produces a prompt-conditioned spatial Context–Text Prior (CTP), which is propagated top-down and updated across focus blocks to guide selective refinement. Within the focus stage, multi-head cross-attention and ContMix-MLP provide text-conditioned feature adaptation and dynamic spatial mixing. TG-OverLoCK does not introduce a new vision–language pre-training objective or a trainable text encoder; instead, it isolates the architectural effect of coarse-to-fine text guidance under a standard frozen-CLIP OVD pipeline. On LVIS rare-to-novel and COCO OVD zero-shot evaluation, TG-OverLoCK-B achieves 38.2 AP and 29.0 Novel AP, respectively, while running at 35 FPS on a single RTX 4090 under our in-house timing protocol. Relative to the matched OverLoCK-B baseline, TG-OverLoCK-B improves LVIS AP from 35.9 to 38.2 while increasing end-to-end latency from 24.5 ms to 28.6 ms and peak VRAM from 4.8 GB to 6.2 GB, indicating an accuracy gain with moderate overhead rather than a free improvement.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TG-OverLoCK for open-vocabulary detection with text-guided coarse-to-fine refinement

  • Yue Li,
  • Hongqiang Huo,
  • Hengjie Su,
  • Siyi Yan,
  • Ruijuan Chen

摘要

Open-vocabulary object detection (OVD) requires detectors to align image regions with text-defined categories while remaining computationally practical. In existing systems, stronger cross-modal reasoning often improves recognition of rare or fine-grained categories, but typically increases inference cost. We present TG-OverLoCK, an OverLoCK-based extension for OVD that selectively concentrates language-conditioned refinement capacity in an overview-to-focus pathway, rather than inserting uniformly dense cross-modal fusion throughout the backbone. The model combines a lightweight overview stage with a deeper focus stage. The overview stage produces a prompt-conditioned spatial Context–Text Prior (CTP), which is propagated top-down and updated across focus blocks to guide selective refinement. Within the focus stage, multi-head cross-attention and ContMix-MLP provide text-conditioned feature adaptation and dynamic spatial mixing. TG-OverLoCK does not introduce a new vision–language pre-training objective or a trainable text encoder; instead, it isolates the architectural effect of coarse-to-fine text guidance under a standard frozen-CLIP OVD pipeline. On LVIS rare-to-novel and COCO OVD zero-shot evaluation, TG-OverLoCK-B achieves 38.2 AP and 29.0 Novel AP, respectively, while running at 35 FPS on a single RTX 4090 under our in-house timing protocol. Relative to the matched OverLoCK-B baseline, TG-OverLoCK-B improves LVIS AP from 35.9 to 38.2 while increasing end-to-end latency from 24.5 ms to 28.6 ms and peak VRAM from 4.8 GB to 6.2 GB, indicating an accuracy gain with moderate overhead rather than a free improvement.