Resisting Catastrophic Recall: Persistent Unlearning via Knowledge Distillation with Feature Suppression
摘要
Machine learning, as a key supporting technology for AI, has greatly contributed to the rapid development of AI and provided a strong impetus for improving productivity. Meanwhile, machine unlearning has emerged as an important area as data privacy and security concerns become more prominent. It helps to remove certain data knowledge from a trained model without retraining it from scratch. Existing research on machine unlearning methods has primarily focused on improving the efficiency of unlearning algorithms and the effectiveness of data removal. However, it largely ignores whether real-world models can maintain unlearned performance during incremental learning. Motivated by this, we incorporate a knowledge distillation-based feature suppression mechanism to prevent the model from re-learning the removed class representations when the unlearned model undergoes successive incremental learning. By leveraging knowledge distillation, we effectively constrain the feature space, ensuring that the unlearned knowledge does not resurface in subsequent learning stages. Furthermore, we design class-specific unlearning methods to validate the proposed approach and provide a new perspective on class unlearning.