Robust Optimization in Machine Unlearning
摘要
This chapter examines the robustness challenges in machine unlearning, focusing on two major vulnerabilities: relearning attacks, where forgotten knowledge rapidly resurfaces under targeted fine-tuning, and irrelevant fine-tuning, where even unrelated training tasks can inadvertently undo unlearning effects. It first introduces unlearning robustness against worst-case relearning attacks as a min–max optimization problem, leveraging techniques such as sharpness-aware minimization and smoothness optimization to flatten the forget loss landscape and improve resistance to adversarial parameter updates. This chapter then addresses the broader challenge of arbitrary downstream fine-tuning by extending invariant risk minimization to the unlearning setting, enforcing environment-agnostic forgetting so that unlearning effects persist across diverse fine-tuning tasks. These techniques achieve substantially higher robustness against both direct relearning attacks and arbitrary downstream fine-tuning, ensuring that forgotten knowledge remains reliably erased under a wide range of post-unlearning scenarios.