Quantifying the Trade-Offs Between Dimensions of Trustworthy AI - An Empirical Study on Fairness, Explainability, Privacy, and Robustness

Trustworthy AI encompasses various requirements for AI systems, including explainability, fairness, privacy, and robustness. Addressing these dimensions concurrently is challenging due to inherent tensions and trade-offs between them. Current research highlights these trade-offs, focusing on specific interactions, but comprehensive and systematic evaluations remain insufficient. This study aims to enhance the understanding of trade-offs among explainability, fairness, privacy, and robustness in AI. By conducting extensive experiments in the domain of image classification, it quantitatively assesses how methods to improve one requirement impact others. More specifically, it explores different training adaptations to enhance each requirement and measures their effects on the others on four datasets for gender classification. The experiments revealed that the Local Gradient Alignment method improved explainability and robustness but introduced trade-offs in fairness, privacy, and accuracy. Fairness-focused training adaptations only enhanced fairness for the most biased models. For all other cases, fairness, explainability and robustness are reduced. Differential Privacy improved privacy but compromised explainability, fairness, and accuracy, with varied impacts on robustness. Data augmentation techniques enhanced robustness, explainability and accuracy with minor trade-offs in privacy and fairness.