Trusting Fair Data: Leveraging Quality in Fairness-Driven Data Removal Techniques

In this paper, we explore bias mitigation techniques that use undersampling to create a fair representation of sociodemographic groups in training sets. These techniques aim to make machine learning models trained on these datasets fair. However, such approaches may exclude relevant data, reducing trustworthiness and potentially harming models’ performances. To address this, we propose two criteria in addition to fairness: (1) group coverage and (2) minimal data loss. Group coverage ensures that no group gets entirely removed, while minimal data loss aims to retain as many data points as possible. By proposing a multi-objective optimization approach, we find Pareto-optimal solutions that balance these objectives. This allows users to make informed decisions about the trade-off between fairness and data quality. Our method is distributed as a Python package via PyPI under the name FairDo ( https://github.com/mkduong-ai/fairdo ).