Item Response Theory in Sample Reweighting to Build Fairer Classifiers

Currently, one of the biggest challenges of Machine Learning (ML) is to develop fairer models that do not propagate prejudices, stereotypes, social inequalities, and other types of discrimination in their decisions. Before ML faced the problem of unfair decision-making, the field of educational testing developed several mathematical tools to decrease bias in selections made by tests. Thus, the Item Response Theory is one of these main tools, and its great power of evaluation helps make fairer selections. Therefore, in this paper, we use the concepts of Item Response Theory to propose a novel sample reweighting method named IRT-SR. The IRT-SR method aims to assign weights to the most important instances to minimize discriminatory effects in binary classification tasks. According to our results, IRT-SR guides classification algorithms to fit fairer models, improving the main group fairness notions such as demographic parity, equal opportunity, and equalized odds without significant performance loss.