Enabling Understanding of AI Model Behavior Through Visualization

Artificial intelligence (AI) models are quickly becoming ubiquitous in systems throughout all sectors of society. As a result, decisions about which AI-enabled system (AIES) to procure are increasingly becoming the responsibility of individuals in organizations without AI expertise. Faced with this decision, a decision-maker is typically provided only summary statistics about model performance, including accuracy, precision, recall, and F1 score of the model. In addition to being nonintuitive to many decision-makers, by focusing on success, these metrics may mask model behavior including the risks associated with differences in the failure modes of the models. To understand how to communicate efficiently such that non-AI experts can make informed procurement decision, grounded in their understanding of the model behavior, an experiment was conducted to test how different information formats impact that understanding. The results, which indicate that there is a difference in the effectiveness of the tested visualizations, not only advance the conversation regarding explainable AI but also inform the procurement processes for AI models.