Scholarlink

<p>Video instance segmentation is a challenging computer vision task that involves detecting, classifying, segmenting, and tracking object instances in videos. Traditional methods often struggle with modeling complex spatio-temporal dynamics and producing high-quality segmentation masks. This study introduces TADeFormer, a novel framework that addresses these limitations by enhancing cross-frame feature modeling through a temporal multi-scale deformable attention module. This module dynamically aggregates sparse features across a learnable temporal window and multiple spatial scales, guided by a novel 3D sinusoidal spatio-temporal position encoding. To further refine the output, TADeFormer incorporates an instance-aware attention mechanism in the decoder to ensure temporal consistency and a conditional PatchGAN discriminator during training to enforce sharper and more structured masks. Experiments on YouTube-VIS 2019 and 2021 benchmarks demonstrate the superior performance of TADeFormer, achieving 50.7% mask AP with ResNet-50 backbone and 51.8% AP with ResNet-101 backbone, significantly exceeding the baseline state-of-the-art performance by 3.3% and 2.8%, respectively.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

TADeFormer: Harnessing temporal-spatial deformable attention for enhanced video instance segmentation

  • 期刊论文

  • 发表日期：

    2026年3月10日

查看全文
  • Jing Li,
  • Xiaoping Huang,
  • Mei Xue

摘要

Video instance segmentation is a challenging computer vision task that involves detecting, classifying, segmenting, and tracking object instances in videos. Traditional methods often struggle with modeling complex spatio-temporal dynamics and producing high-quality segmentation masks. This study introduces TADeFormer, a novel framework that addresses these limitations by enhancing cross-frame feature modeling through a temporal multi-scale deformable attention module. This module dynamically aggregates sparse features across a learnable temporal window and multiple spatial scales, guided by a novel 3D sinusoidal spatio-temporal position encoding. To further refine the output, TADeFormer incorporates an instance-aware attention mechanism in the decoder to ensure temporal consistency and a conditional PatchGAN discriminator during training to enforce sharper and more structured masks. Experiments on YouTube-VIS 2019 and 2021 benchmarks demonstrate the superior performance of TADeFormer, achieving 50.7% mask AP with ResNet-50 backbone and 51.8% AP with ResNet-101 backbone, significantly exceeding the baseline state-of-the-art performance by 3.3% and 2.8%, respectively.