<p>This article proposes FairDIF, a reference framework that leverages Item Response Theory (IRT) and Differential Item Functioning (DIF) concepts to promote group fairness in classification problems. IRT and DIF are distinct yet complementary mathematical tools widely used in educational assessments. IRT enables robust evaluations by modeling the probability of a correct response as a function of item properties and respondent ability, effectively capturing individual performance variability. In contrast, DIF identifies whether specific items systematically favor respondents from certain sociodemographic groups, regardless of their underlying ability levels, thereby revealing potential sources of bias. Our framework first assesses classifier predictions using an IRT model. Subsequently, FairDIF analyzes potential bias through the DIF area method. This process yields rich insights into classifier performance and fairness, as well as the relative difficulty of correctly classifying each example. We can apply FairDIF directly or extend it to create new fairness-aware methods. Here, we present multiple avenues for employing FairDIF in more responsible solutions and reformulate and evaluate three distinct and promising methods that use the proposed framework as their core component: (i) sample reweighting, (ii) model selection, and (iii) threshold tuning. Results demonstrate that these methods effectively balance predictive performance with key group-fairness criteria relative to their counterparts. These findings emphasize FairDIF’s potential to support fair model development across different stages of machine learning pipelines. Moreover, this work advances the integration of IRT and DIF into machine learning tasks by expanding their scope and providing a foundation for future research in this emerging area.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FairDIF: debiasing classifiers with item response theory and differential item functioning

  • Diego Minatel,
  • Antonio R. S. Parmezan,
  • Nícolas Roque dos Santos,
  • Mariana Cúri,
  • Ricardo B. C. Prudêncio,
  • Alneu de Andrade Lopes

摘要

This article proposes FairDIF, a reference framework that leverages Item Response Theory (IRT) and Differential Item Functioning (DIF) concepts to promote group fairness in classification problems. IRT and DIF are distinct yet complementary mathematical tools widely used in educational assessments. IRT enables robust evaluations by modeling the probability of a correct response as a function of item properties and respondent ability, effectively capturing individual performance variability. In contrast, DIF identifies whether specific items systematically favor respondents from certain sociodemographic groups, regardless of their underlying ability levels, thereby revealing potential sources of bias. Our framework first assesses classifier predictions using an IRT model. Subsequently, FairDIF analyzes potential bias through the DIF area method. This process yields rich insights into classifier performance and fairness, as well as the relative difficulty of correctly classifying each example. We can apply FairDIF directly or extend it to create new fairness-aware methods. Here, we present multiple avenues for employing FairDIF in more responsible solutions and reformulate and evaluate three distinct and promising methods that use the proposed framework as their core component: (i) sample reweighting, (ii) model selection, and (iii) threshold tuning. Results demonstrate that these methods effectively balance predictive performance with key group-fairness criteria relative to their counterparts. These findings emphasize FairDIF’s potential to support fair model development across different stages of machine learning pipelines. Moreover, this work advances the integration of IRT and DIF into machine learning tasks by expanding their scope and providing a foundation for future research in this emerging area.