GUIDE: Generative Understanding Injection for Dense Estimation via Enhanced Shape Coherence
摘要
Semantic segmentation models face a critical dilemma: dense prediction approaches achieve high accuracy but produce visually disruptive fragmented masks, while generative methods create coherent masks but sacrifice spatial precision and incur prohibitive computational costs. Examining this trade-off, we find that generative models encode valuable shape priors that could transform semantic segmentation if efficiently transferred. Building on this insight, we present GUIDE (Generative Understanding Injection for Dense Estimation), a novel framework that injects generative coherence knowledge through a layer specific Soft-Guidance Query (SG-query) mechanism during training, then deploys this knowledge at inference time without the generative branch. The key innovations are a layer adaptive guidance attention bias that modulates cross-attention operations within each decoder layer, enhancing focus on regions conforming to learned shape priors while suppressing fragmentation, complemented by multi-scale Wasserstein distribution alignment that enables comprehensive shape-level knowledge transfer. Experiments across Cityscapes, ADE20K, and Hypersim datasets demonstrate that GUIDE reduces fragmentation by 59.4% while improving accuracy (+1.9% mIoU) with minimal computational overhead (4% parameter increase), achieving fragment-free segmentation without the substantial costs of existing generative approaches. This training-time learning, inference-time guidance paradigm efficiently combines the complementary strengths of both segmentation approaches.