Navigating Differential Privacy Constraints in Machine Learning

Given a trained model and a data sample, membership inference (MI) attacks ascertain whether the sample is part of the model’s training dataset. A prevalent countermeasure against MI attacks involves employing differential privacy (DP) during model training, which constrains the impact that individual training data records exert on the machine learning model. Implementing differential privacy in machine learning necessitates that data scientists select privacy parameters (ε, δ). The selection of meaningful privacy parameters is crucial, as models trained with weak privacy parameters can result in significant privacy leakage, whereas strong privacy parameters can substantially diminish the model’s utility. However, determining appropriate privacy parameter values is challenging for two primary reasons. First, the theoretical upper bound on privacy loss (ε, δ) can be lenient, contingent on the chosen sensitivity and data distribution of real-world datasets. Second, legal requirements and societal norms for anonymization often pertain to the capability to identify individuals, which (ε, δ) only indirectly addresses. To tackle this issue, the parameters (ε, δ) were recalibrated based on a bound on the Bayesian posterior belief of the adversary, which was presumed to be due to differential privacy concerning the presence of any record in the training dataset. Ultimately, empirically implementing this differential privacy adversary enabled data scientists to audit model training and compute empirical identifiability scores (ε, δ) on the MNIST dataset.