For medical image segmentation, some distillation methods have yielded impressive results, but in these approaches student models normally fail to obtain the focus-needed knowledge during the feature distillation procedure. Therefore, in this paper, we propose Spatial Position Augmentation cRoss-guided Knowledge distillation called SPARK . Specifically, we first focus on the enhanced intermediate features by spatial position, allowing student models to acquire knowledge that is of more concern to segmentation, thus, segmenting to a more precise location. Furthermore, we design a novel Cross-Guided Distillation (CGD) in which the student model can acquire old knowledge to avoid oblivion of knowledge, and acquire new knowledge to obtain learning direction from the teacher model. Thanks to that, student models can segment more precisely, especially for small targets. Besides, by transferring knowledge from well-trained but heavy teacher models to another lightweight model, we address the problem that most existing segmentation models depend on massive storage and complex computations and cannot be used in current clinical settings. To validate the effectiveness of the proposed method, we conduct experiments on two widely used CT datasets LiTS17 and KiTS19. The results demonstrate that our approach significantly improves the segmentation performance of lightweight models, with improvements of up to 32.69% in the dice coefficient score.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SPARK: Cross-Guided Knowledge Distillation with Spatial Position Augmentation for Medical Image Segmentation

  • Lingbing Xu,
  • Zhiyuan Wang,
  • Weitao Song,
  • Yi Ji,
  • Chunping Liu

摘要

For medical image segmentation, some distillation methods have yielded impressive results, but in these approaches student models normally fail to obtain the focus-needed knowledge during the feature distillation procedure. Therefore, in this paper, we propose Spatial Position Augmentation cRoss-guided Knowledge distillation called SPARK . Specifically, we first focus on the enhanced intermediate features by spatial position, allowing student models to acquire knowledge that is of more concern to segmentation, thus, segmenting to a more precise location. Furthermore, we design a novel Cross-Guided Distillation (CGD) in which the student model can acquire old knowledge to avoid oblivion of knowledge, and acquire new knowledge to obtain learning direction from the teacher model. Thanks to that, student models can segment more precisely, especially for small targets. Besides, by transferring knowledge from well-trained but heavy teacher models to another lightweight model, we address the problem that most existing segmentation models depend on massive storage and complex computations and cannot be used in current clinical settings. To validate the effectiveness of the proposed method, we conduct experiments on two widely used CT datasets LiTS17 and KiTS19. The results demonstrate that our approach significantly improves the segmentation performance of lightweight models, with improvements of up to 32.69% in the dice coefficient score.