Semantic Segmentation of Medical Images Based on Knowledge Distillation Algorithm

(1) Background: Since the advent of convolutional neural networks, they have been employed in various fields. Nevertheless, the calculation cost must be considered for the automatic deep learning analysis. A novel strategy was proposed to improve the segmentation performance without adding additional parameters by transferring pre-trained prior knowledge information in medical scenes. (2) Methods: we accessed knowledge distillation to train a teacher and student network simultaneously, then introduced the best weight of the teacher to distill the student network, contributing to the distillation loss function. The effectiveness was verified with two segmentation convolutional neural network structures. (3) Result: the mean Acc, mIoU, Dice, and Kappa of the distilled network in three medical datasets were increased by 0.18 percent, 1.25 percent, 1.62 percent, and 0.79 percent, respectively. (4) Conclusions: the dark knowledge was transformed from a deeper network to a lightweight one that can enhance the segmentation significantly without additional elements.