Semantic segmentation is defined by a fundamental conflict between accuracy and coherence. Dense prediction models excel at pixel-level precision but often shatter objects into fragmented masks. In contrast, generative approaches maintain object integrity, producing coherent masks at the expense of spatial accuracy and practical efficiency. We find that generative models encode valuable shape priors, which, if transferred efficiently, could resolve this conflict. We introduce Guided-Mask2Former, a novel framework that injects this generative knowledge during training and deploys it at inference without expensive generative support. Our approach introduces a Soft-Guidance Query mechanism that learns these priors and injects them into the decoder via a guidance attention bias. To ensure the effective transfer of shape priors, the model aligns the predicted and generative mask distributions using the Wasserstein distance. Evaluated on Cityscapes, ADE20K and Hypersim, Guided-Mask2Former reduces fragmentation by 54.5% while simultaneously improving accuracy by +1.9% mIoU. This performance is achieved with a minimal 3.9% parameter increase, a result of our ‘training-time learning, inference-time guidance’ paradigm that effectively unites the strengths of both segmentation approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Coherent Without Cost: Learning Generative Shape Priors for Fragment-Free Semantic Segmentation

  • Shengye Yang,
  • Jie Liu,
  • Qingyang Zhang

摘要

Semantic segmentation is defined by a fundamental conflict between accuracy and coherence. Dense prediction models excel at pixel-level precision but often shatter objects into fragmented masks. In contrast, generative approaches maintain object integrity, producing coherent masks at the expense of spatial accuracy and practical efficiency. We find that generative models encode valuable shape priors, which, if transferred efficiently, could resolve this conflict. We introduce Guided-Mask2Former, a novel framework that injects this generative knowledge during training and deploys it at inference without expensive generative support. Our approach introduces a Soft-Guidance Query mechanism that learns these priors and injects them into the decoder via a guidance attention bias. To ensure the effective transfer of shape priors, the model aligns the predicted and generative mask distributions using the Wasserstein distance. Evaluated on Cityscapes, ADE20K and Hypersim, Guided-Mask2Former reduces fragmentation by 54.5% while simultaneously improving accuracy by +1.9% mIoU. This performance is achieved with a minimal 3.9% parameter increase, a result of our ‘training-time learning, inference-time guidance’ paradigm that effectively unites the strengths of both segmentation approaches.