Strict Differentially Private Support Vector Machines with Dimensionality Reduction

With the widespread data collection and processing, privacy-preserving machine learning has become increasingly important in addressing privacy risks related to individuals. Support vector machine (SVM) is one of the most elementary learning models of machine learning. Privacy issues surrounding SVM training classifiers have attracted increasing attention. In this paper, we propose DPDR-DPSVM which is a strict differentially private support vector machine algorithm with high data utility. Aiming at high-dimensional data, we adopt differential privacy in both the dimensionality reduction phase and SVM classifier training phase, which improves model accuracy while achieving strong privacy guarantees. Besides, we train DP-compliant SVM classifiers by adding noise to the objective function itself, thus leading to better data utility. Extensive experiments on three high-dimensional datasets demonstrate that DPDR-DPSVM can achieve high accuracy while ensuring strong privacy protection.