<p>Model slicing refers to computing model-performance metrics independently for sub-groups to identify biases. As an example, a healthcare model with good overall accuracy may perform poorly for children or seniors. We extend model slicing to summarize a new type of bias: algorithmic recourse. This quantifies the ability and cost, on average, for members of sub-groups to move from an undesirable to a desirable outcome. As an example, a recourse-biased loan-approval model may show equal classification accuracy for different sub-groups; however, women whose loans were declined may need to increase their savings balance by twenty percent to receive approval, while men only require a ten-percent increase. Our solution, REACT, combines counterfactual mining, finding nearest positively labeled neighbors to a given negatively labeled example, and if-then rule mining to generate a concise summary as an explanation table of recourse bias across sub-groups. We experimentally demonstrate the value of our solution using several case studies including finance and law enforcement.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Model Slicing for Analysis of Recourse Bias with Explanation Tables

  • Anastasiia Avksientieva,
  • Andrew Chai,
  • Parke Godfrey,
  • Lukasz Golab,
  • Divesh Srivastava,
  • Jarek Szlichta

摘要

Model slicing refers to computing model-performance metrics independently for sub-groups to identify biases. As an example, a healthcare model with good overall accuracy may perform poorly for children or seniors. We extend model slicing to summarize a new type of bias: algorithmic recourse. This quantifies the ability and cost, on average, for members of sub-groups to move from an undesirable to a desirable outcome. As an example, a recourse-biased loan-approval model may show equal classification accuracy for different sub-groups; however, women whose loans were declined may need to increase their savings balance by twenty percent to receive approval, while men only require a ten-percent increase. Our solution, REACT, combines counterfactual mining, finding nearest positively labeled neighbors to a given negatively labeled example, and if-then rule mining to generate a concise summary as an explanation table of recourse bias across sub-groups. We experimentally demonstrate the value of our solution using several case studies including finance and law enforcement.