How to Explain It to System Testers?

In the realm of explainable AI (XAI), limited research exists on user role-specific explanations. This study aims to determine the explanation needs for the user role “system tester of AI-based systems.” It investigates whether established explanation types adequately address the explainability requirements of ML-based application testers. Through a qualitative study (n = 12), we identified the explanation needs for three user tasks: test strategy determination, test case determination, and test result determination. The research yields five findings: F1) proposing a new explanation domain type, “system domain,” F2) proposing a new explanation structure, “hierarchical,” F3) identifying overlapping explanation content between two user groups, F4) considering identified inputs of a user task as explanation content candidates, and F5) highlighting the risk of combining the evaluation of assumed mental model representations with identifying explanation content in one study.