Progressively Correcting Soft Labels via Teacher Team for Knowledge Distillation in Medical Image Segmentation

State-of-the-art knowledge distillation (KD) methods aim to capture the underlying information within the teacher and explore effective strategies for knowledge transfer. However, due to challenges such as blurriness, noise, and low contrast inherent in medical images, the teacher’s predictions (soft labels) may also include false information, thus potentially misguiding the student’s learning process. Addressing this, we pioneer a novel correction-based KD approach (PLC-KD) and introduce two assistants for perceiving and correcting the false soft labels. More specifically, the false-pixel-aware assistant targets global error correction, while the boundary-aware assistant focuses on lesion boundary errors. Additionally, a similarity-based correction scheme is designed to forcefully rectify the remaining hard false pixels. Through this collaborative effort, the teacher team (comprising a teacher and two assistants) progressively generates more accurate soft labels, ensuring the “all-correct” final soft labels for student guidance during KD. Extensive experimental results demonstrate that the proposed PLC-KD framework attains superior performance to state-of-the-art methods on three challenging medical segmentation tasks.