Synthetic data generators and machine learning models can memorize their training data, posing privacy concerns. Membership inference attacks (MIAs) are a standard method of estimating their privacy risk. The risk of individual records is typically computed by evaluating MIAs in a record-specific privacy game. We analyze the privacy game commonly used for attackers under realistic assumptions (the traditional game)—particularly for synthetic tabular data—and show that it averages a record’s privacy risk across datasets. We show this implicitly assumes the dataset a record is part of has no impact on the record’s risk, providing a misleading risk estimate when a specific model or synthetic dataset is released. Instead, we propose a novel use of the leave-one-out privacy game, so far used exclusively to audit differential privacy guarantees, and call this the model-seeded game. We formalize it and show that it provides an accurate estimate of the privacy risk for a record in its specific dataset. We instantiate and evaluate the state-of-the-art MIA for synthetic data generators in both privacy games, and show across multiple datasets and models that they indeed result in different risk scores, with up to 94% of high-risk records being overlooked by the traditional game. We further show that records in smaller datasets tend to have a larger gap between risk estimates. Taken together, our results show that the model-seeded setup yields a risk estimate specific to a released synthetic dataset or model and in line with the standard notion of privacy leakage from prior work, meaningfully different from the dataset-averaged risk provided by the traditional privacy game.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Lost in the Averages: Reassessing Record-Specific Privacy Risk Evaluation

  • Nataša Krčo,
  • Florent Guépin,
  • Matthieu Meeus,
  • Bogdan Kulynych,
  • Yves-Alexandre de Montjoye

摘要

Synthetic data generators and machine learning models can memorize their training data, posing privacy concerns. Membership inference attacks (MIAs) are a standard method of estimating their privacy risk. The risk of individual records is typically computed by evaluating MIAs in a record-specific privacy game. We analyze the privacy game commonly used for attackers under realistic assumptions (the traditional game)—particularly for synthetic tabular data—and show that it averages a record’s privacy risk across datasets. We show this implicitly assumes the dataset a record is part of has no impact on the record’s risk, providing a misleading risk estimate when a specific model or synthetic dataset is released. Instead, we propose a novel use of the leave-one-out privacy game, so far used exclusively to audit differential privacy guarantees, and call this the model-seeded game. We formalize it and show that it provides an accurate estimate of the privacy risk for a record in its specific dataset. We instantiate and evaluate the state-of-the-art MIA for synthetic data generators in both privacy games, and show across multiple datasets and models that they indeed result in different risk scores, with up to 94% of high-risk records being overlooked by the traditional game. We further show that records in smaller datasets tend to have a larger gap between risk estimates. Taken together, our results show that the model-seeded setup yields a risk estimate specific to a released synthetic dataset or model and in line with the standard notion of privacy leakage from prior work, meaningfully different from the dataset-averaged risk provided by the traditional privacy game.