Evaluating Contribution of Training Samples for Differentially Private Machine Learning

Differential privacy is applied to machine learning for privacy protection because of its formal privacy guarantee. Differentially private algorithm needs to inject enough noise to limit the overall privacy disclosure risk of the training set. In the case of unbalanced data sets, however, this privacy protection intensity is excessive for most datasets, leading to significant loss of utility. In this paper, we adjust the training set by assessing the risk of disclosure of data privacy and its contribution to model accuracy to improve the utility of differentially private ML. Specifically, we quantify the risk of data privacy disclosure through membership inference attacks and evaluate the contribution of data to improving model accuracy through ablation experiments. Our experiments show that when the training set of the model is unbalanced, the model accuracy can be substantially improved by adjusting the training set to our assessment method under the same budget of privacy.