Scholarlink

<p>Knowledge distillation typically requires additional distillation costs to improve model performance. In this paper, our focus lies in the straightforward construction of task-level losses by mimicking the knowledge transfer mechanism embedded in the existing logits-based knowledge distillation. Firstly, we put forward a method that enables direct knowledge transfer from the ground truth, with the aim of eliminating the supplementary costs linked to traditional distillation methods. Furthermore, we introduce a strategy to address the issue of overconfident softmax predictions that may emerge from this direct transfer. By applying a linear mapping to the ground truth, we can effectively regulate the model’s outputs and thus enhance the reliability of predictions. We carry out extensive experiments on the Cityscapes dataset, the Pascal Context dataset, ADE20K, and COCO Stuff164k, respectively. Both the experimental and visualization results illustrate that our proposed methods surpass the state-of-the-art KD methods in terms of training efficiency and segmentation performance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Ground truth is the best teacher: supervised semantic segmentation inspired by knowledge transfer mechanisms

  • 期刊论文

  • 发表日期：

    2025年1月3日

查看全文
  • Xiangchun Yu,
  • Huofa Liu,
  • Dingwen Zhang,
  • Miaomiao Liang,
  • Lingjuan Yu,
  • Jian Zheng

摘要

Knowledge distillation typically requires additional distillation costs to improve model performance. In this paper, our focus lies in the straightforward construction of task-level losses by mimicking the knowledge transfer mechanism embedded in the existing logits-based knowledge distillation. Firstly, we put forward a method that enables direct knowledge transfer from the ground truth, with the aim of eliminating the supplementary costs linked to traditional distillation methods. Furthermore, we introduce a strategy to address the issue of overconfident softmax predictions that may emerge from this direct transfer. By applying a linear mapping to the ground truth, we can effectively regulate the model’s outputs and thus enhance the reliability of predictions. We carry out extensive experiments on the Cityscapes dataset, the Pascal Context dataset, ADE20K, and COCO Stuff164k, respectively. Both the experimental and visualization results illustrate that our proposed methods surpass the state-of-the-art KD methods in terms of training efficiency and segmentation performance.