CS-KD: Confused Sample Knowledge Distillation for Semantic Segmentation of Aerial Imagery

Currently, semantic segmentation methods based on knowledge distillation (KD) mainly focus on transferring various structured knowledge to the student network and designing corresponding optimization goals to encourage the student network to imitate the output of the teacher network. However, these methods do not consider the impact of sample quality on model training. Especially for aerial images of complex scenes, problems such as object occlusion and boundary blur caused by factors such as illumination and imaging angle will introduce many confused samples. These confused samples can lead to labeling bias or incorrect predictions. Therefore, we propose a confused sample knowledge distillation method (CS-KD) and design an adaptive sample screening strategy. During the training process, CS-KD makes full use of the prediction capabilities of the teacher and student networks at all stages to screen confused samples pixel by pixel and adjust the importance of different training samples. Experiment results verify that, based on the Potsdam and Vaihingen benchmarks, CS-KD can achieve competitive performance compared with other state-of-the art KD methods. Additionally, our research showcases that CS-KD can integrate with existing KD methods to improve their upper performance bound.