Scholarlink

<p>Scene graph generation aims to detect visual objects and their relationships in images, providing structured representations for visual scene understanding. However, it faces fundamental challenges from extreme long-tail distribution and semantic ambiguities between related predicates. The severe class imbalance—where head classes dominate with abundant samples while tail classes have scarce training instances—creates distinct learning dynamics that existing re-balancing methods fail to address by treating all classes uniformly. We propose Frequency-Adaptive Prototype Learning (FAPL), a unified framework that introduces three key innovations: (1) frequency-aware momentum scheduling that enables head prototypes to stabilize early as knowledge sources while tail prototypes maintain aggressive adaptation to maximize learning from limited samples; (2) pattern-aware supervision that learns semantic confusion patterns between related predicates and adaptively reduces penalties for plausible errors while maintaining full supervision for implausible ones; (3) curriculum learning integration that progressively shifts training emphasis from frequent to rare relationships as the model accumulates semantic understanding. These components form a mutually reinforcing system where early head learning establishes reliable knowledge bases, pattern-aware supervision reduces confusion noise, and curriculum scheduling maximizes tail exploitation. Extensive experiments on Visual Genome, GQA, and Open Images V6 datasets demonstrate that FAPL achieves state-of-the-art performance with substantial improvements over previous best methods: up to 30.3% on Visual Genome, 28.8% on GQA, and 19.9% on Open Images V6, while showing superior generalization capability on recognizing unseen relationship combinations.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Frequency-adaptive prototype learning for long-tail scene graph generation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月14日

查看全文
  • Funing Xie,
  • YaHong Wu,
  • Feng Liu

摘要

Scene graph generation aims to detect visual objects and their relationships in images, providing structured representations for visual scene understanding. However, it faces fundamental challenges from extreme long-tail distribution and semantic ambiguities between related predicates. The severe class imbalance—where head classes dominate with abundant samples while tail classes have scarce training instances—creates distinct learning dynamics that existing re-balancing methods fail to address by treating all classes uniformly. We propose Frequency-Adaptive Prototype Learning (FAPL), a unified framework that introduces three key innovations: (1) frequency-aware momentum scheduling that enables head prototypes to stabilize early as knowledge sources while tail prototypes maintain aggressive adaptation to maximize learning from limited samples; (2) pattern-aware supervision that learns semantic confusion patterns between related predicates and adaptively reduces penalties for plausible errors while maintaining full supervision for implausible ones; (3) curriculum learning integration that progressively shifts training emphasis from frequent to rare relationships as the model accumulates semantic understanding. These components form a mutually reinforcing system where early head learning establishes reliable knowledge bases, pattern-aware supervision reduces confusion noise, and curriculum scheduling maximizes tail exploitation. Extensive experiments on Visual Genome, GQA, and Open Images V6 datasets demonstrate that FAPL achieves state-of-the-art performance with substantial improvements over previous best methods: up to 30.3% on Visual Genome, 28.8% on GQA, and 19.9% on Open Images V6, while showing superior generalization capability on recognizing unseen relationship combinations.