A Coreset Perspective for Machine Unlearning
摘要
This chapter explores coreset perspectives in machine unlearning across efficiency, robustness, and evaluation. We first reveal a coreset effect: small random subsets of the forget set can match full-data unlearning performance, with a coreset-training trade-off showing that fewer samples require proportionally more unlearning epochs. We then examine advanced coreset selection methods, e.g., gradient-, clustering-, and likelihood-based, finding only marginal gains over random selection across standard unlearning benchmarks. Next, we assess the faithfulness of coreset-unlearned models, showing they share the same optimal basin as full-forget-set models and exhibit similar robustness to adversarial attacks and downstream finetuning. We further introduce a utility-forget trade-off perspective, where pruning high-variance outliers mitigates collateral utility loss. Finally, using bi-level optimization, we identify worst-case forget sets that are hardest to erase yet least essential for generalization, linking coresets to adversarial unlearning evaluation. Overall, coresets provide a unifying lens for understanding data importance in unlearning, motivating principled selection methods and rigorous evaluation protocols for both vision and language models.