FADE: Progressive Unlearning for Language Models via Adversarial Disruption and Editing
摘要
As large language models (LLMs) are increasingly deployed in real-world systems, the ability to selectively remove specific training data, known as machine unlearning, has become essential for privacy protection and compliance. Existing unlearning methods often fall short in this setting: gradient-based approaches risk destabilizing generation, preference-based methods struggle with incomplete forgetting, and parameter-space editing lacks behavioral control. To address these limitations, we propose Forgetting through Adversarial Disruption and Editing (FADE), a progressive framework that decomposes unlearning into three stages. FADE first introduces distributional noise to weaken memorization, then redirects output probabilities away from forgotten targets, at last FADE recovers performance on unaffected data. Experiments on the MUSE benchmark demonstrate that FADE achieves strong forgetting and privacy protection without compromising utility, outperforming prior methods. These results highlight the importance of staged, behavior-level interventions for reliable and controllable unlearning in LLMs.