Machine unlearning has become a pivotal paradigm for mitigating the impact of unwanted data points incorporated during model training. Despite recent progress in unlearning algorithms, rigorously assessing their effectiveness remains challenging. Retraining a model from scratch—though the very process many algorithms aim to avoid—continues to serve as a widely adopted benchmark because it inherently excludes undesirable data. This chapter critically examines the role of retraining as a benchmark for evaluating machine unlearning. Through extensive experiments spanning diverse metrics, datasets, and model architectures, it reveals significant instability in retrained models. In particular, retraining can produce both the strongest and weakest unlearning performance while maintaining comparable utility. This pattern persists across multiple membership inference attacks and distance-based metrics. Although these findings do not diminish retraining’s value as an unlearning method that guarantees complete data removal, they highlight the importance of applying it with caution and nuance as an evaluation standard.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Instability of Retraining as a Benchmark for Machine Unlearning Evaluation

  • Chris Yuhao Liu,
  • Zonglin Di

摘要

Machine unlearning has become a pivotal paradigm for mitigating the impact of unwanted data points incorporated during model training. Despite recent progress in unlearning algorithms, rigorously assessing their effectiveness remains challenging. Retraining a model from scratch—though the very process many algorithms aim to avoid—continues to serve as a widely adopted benchmark because it inherently excludes undesirable data. This chapter critically examines the role of retraining as a benchmark for evaluating machine unlearning. Through extensive experiments spanning diverse metrics, datasets, and model architectures, it reveals significant instability in retrained models. In particular, retraining can produce both the strongest and weakest unlearning performance while maintaining comparable utility. This pattern persists across multiple membership inference attacks and distance-based metrics. Although these findings do not diminish retraining’s value as an unlearning method that guarantees complete data removal, they highlight the importance of applying it with caution and nuance as an evaluation standard.