As large language models (LLMs) are increasingly deployed in real-world systems, the ability to selectively remove specific training data, known as machine unlearning, has become essential for privacy protection and compliance. Existing unlearning methods often fall short in this setting: gradient-based approaches risk destabilizing generation, preference-based methods struggle with incomplete forgetting, and parameter-space editing lacks behavioral control. To address these limitations, we propose Forgetting through Adversarial Disruption and Editing (FADE), a progressive framework that decomposes unlearning into three stages. FADE first introduces distributional noise to weaken memorization, then redirects output probabilities away from forgotten targets, at last FADE recovers performance on unaffected data. Experiments on the MUSE benchmark demonstrate that FADE achieves strong forgetting and privacy protection without compromising utility, outperforming prior methods. These results highlight the importance of staged, behavior-level interventions for reliable and controllable unlearning in LLMs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FADE: Progressive Unlearning for Language Models via Adversarial Disruption and Editing

  • Yang Chen,
  • Zhiwen Tang

摘要

As large language models (LLMs) are increasingly deployed in real-world systems, the ability to selectively remove specific training data, known as machine unlearning, has become essential for privacy protection and compliance. Existing unlearning methods often fall short in this setting: gradient-based approaches risk destabilizing generation, preference-based methods struggle with incomplete forgetting, and parameter-space editing lacks behavioral control. To address these limitations, we propose Forgetting through Adversarial Disruption and Editing (FADE), a progressive framework that decomposes unlearning into three stages. FADE first introduces distributional noise to weaken memorization, then redirects output probabilities away from forgotten targets, at last FADE recovers performance on unaffected data. Experiments on the MUSE benchmark demonstrate that FADE achieves strong forgetting and privacy protection without compromising utility, outperforming prior methods. These results highlight the importance of staged, behavior-level interventions for reliable and controllable unlearning in LLMs.