Scholarlink

<p>In real-world images, low-resolution regions caused by downsampling or compression often exhibit severe pixelation and detail loss, which substantially degrades visual relationship detection (VRD). Most existing VRD approaches primarily rely on fine-grained appearance cues and therefore perform poorly when entities are small or visually ambiguous. To address this limitation, we propose a Segmentation-Guided Transformer Network (SGTN) designed for robust detection of subtle visual relationships under low-resolution conditions. SGTN explicitly incorporates segmentation information and leverages pixel-level semantics, such as entity masks, to compensate for detail loss in low-resolution areas, enabling joint entity and relation prediction within a single-stage Transformer architecture. The framework consists of three modules: a two-stream feature extraction module employing visual and textual encoders, a multimodal feature fusion module based on an encoder-decoder structure that deeply integrates visual and textual features to generate enhanced entity, relation, and mask features, and an entity-relation prediction module that uses the fused features for entity detection and relationship classification. Experiments on downsampled <i>VG</i> benchmarks <i>VG_2</i>, <i>VG_5</i>, and <i>VG_10</i> show that SGTN achieves the best performance in most settings, with particularly notable gains under severe downsampling.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Segmentation-guided transformer network for subtle visual relationship detection

  • 期刊论文

  • 发表日期：

    2026年7月11日

查看全文
  • Fan Yu,
  • Hanxi Cao,
  • Ruichao Hou,
  • Tongwei Ren

摘要

In real-world images, low-resolution regions caused by downsampling or compression often exhibit severe pixelation and detail loss, which substantially degrades visual relationship detection (VRD). Most existing VRD approaches primarily rely on fine-grained appearance cues and therefore perform poorly when entities are small or visually ambiguous. To address this limitation, we propose a Segmentation-Guided Transformer Network (SGTN) designed for robust detection of subtle visual relationships under low-resolution conditions. SGTN explicitly incorporates segmentation information and leverages pixel-level semantics, such as entity masks, to compensate for detail loss in low-resolution areas, enabling joint entity and relation prediction within a single-stage Transformer architecture. The framework consists of three modules: a two-stream feature extraction module employing visual and textual encoders, a multimodal feature fusion module based on an encoder-decoder structure that deeply integrates visual and textual features to generate enhanced entity, relation, and mask features, and an entity-relation prediction module that uses the fused features for entity detection and relationship classification. Experiments on downsampled VG benchmarks VG_2, VG_5, and VG_10 show that SGTN achieves the best performance in most settings, with particularly notable gains under severe downsampling.