Scholarlink

<p>The Mixture-of-Experts (MoE) paradigm is a promising approach to efficiently scale neural networks, especially for large-scale Transformers. This paper investigates integrating MoE into the state-of-the-art mask-based semantic segmentation models, MaskFormer and Mask2Former. We hypothesize that routing inputs to specialized experts will enhance performance in terms of accuracy and domain generalization. We present two novel architectures, Mask-MoE and Mask2-MoE, which embed the MoE paradigm within the transformer decoder to foster specialization among experts for different semantic concepts. Our experiments show that these architectures significantly improve in both in-domain accuracy and cross-domain generalization performance. On the Cityscapes benchmark, our models achieve a better accuracy-efficiency trade-off by reducing confusion between visually similar classes. Furthermore, in a sim-to-real zero-shot domain generalization experiments, our MoE-enhanced models outperform baselines. Our in-depth investigation shows that this improvement stems from a learned, domain-agnostic routing policy in which the experts consistently divide their labor across domains.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

On the effectiveness of MoE-enhanced transformer for accurate and generalizable mask-based semantic segmentation

  • 期刊论文

  • 发表日期：

    2026年6月17日

查看全文
  • Dahye Jung,
  • Youji Sohn,
  • Sang In Lee,
  • Jihun Park

摘要

The Mixture-of-Experts (MoE) paradigm is a promising approach to efficiently scale neural networks, especially for large-scale Transformers. This paper investigates integrating MoE into the state-of-the-art mask-based semantic segmentation models, MaskFormer and Mask2Former. We hypothesize that routing inputs to specialized experts will enhance performance in terms of accuracy and domain generalization. We present two novel architectures, Mask-MoE and Mask2-MoE, which embed the MoE paradigm within the transformer decoder to foster specialization among experts for different semantic concepts. Our experiments show that these architectures significantly improve in both in-domain accuracy and cross-domain generalization performance. On the Cityscapes benchmark, our models achieve a better accuracy-efficiency trade-off by reducing confusion between visually similar classes. Furthermore, in a sim-to-real zero-shot domain generalization experiments, our MoE-enhanced models outperform baselines. Our in-depth investigation shows that this improvement stems from a learned, domain-agnostic routing policy in which the experts consistently divide their labor across domains.