Anti-backdoor Distillation: Towards Secure Student Models in Autonomous Unmanned Systems

Under the constraints of resources, large neural networks are difficult to deploy directly in autonomous unmanned systems, making knowledge distillation a key technology for lightweight model deployment. However, existing researches have shown that the distillation process is vulnerable to backdoor attacks, potentially transferring backdoors from the teacher model to the student model. To address this critical issue, this paper proposes an innovative Anti-Backdoor Distillation (ABD) framework, which can distill and obtain a secure and reliable student model from a backdoor teacher model and poisoned data. The ABD framework consists of three main stages: samples detection, normal distillation, and reverse distillation. First, we innovatively leverage prediction entropy to identify backdoor samples based on their overfitting behavior, and extract a noisy clean set and a high-purity poisoned set from the original backdoor dataset. Second, the student model undergoes normal distillation training on the noisy clean set. Third, the high-purity poisoned set guides the student model to unlearn any residual backdoor knowledge. Experimental results demonstrate that ABD efficiently achieves secure distillation of lightweight student models without requiring any prior knowledge, ensuring the safe deployment of neural networks in autonomous unmanned systems. This research holds significant theoretical and practical value in the co-optimization of model lightweighting and security.