RSUTT: Robust Search Using T-Way Testing

Recent years have seen an increase of decision-making software based on Machine Learning (ML). Multiple cases have been reported where such software are discriminatory based on attributes such as race and gender. Thus, ML-based decision making software need to be tested for discrimination, or fairness testing. One popular approach to fairness testing is to find discriminatory data items by first conducting a global search, and then searching locally near the found discriminatory data items. Aequitas, CGFT, and KOSEI are three examples taking this approach. However, there are issues in terms of stability and efficiency. We thus propose an approach called Robust Search Using T-way Testing (RSUTT), which is based on CGFT for global search and KOSEI for local search. Experiments showed that RSUTT performs more efficiently compared to Aequitas, CGFT, and KOSEI.