Navigating Transparency: The Influence of On-demand Explanations on Non-expert User Interaction with AI

Artificial intelligence (AI) has an increasingly powerful impact on our lives, influencing even non-expert users. This mismatch between an increasingly advanced technology on the one hand and non-expert users on the other hand can lead to unfounded reliance. To improve collaboration between end-users and the system, Explainable AI (XAI) is gaining momentum. However, recent studies yield mixed results on the effects of explanations, leaving uncertainty about whether increased transparency prevents or encourages unfounded reliance. Despite this uncertainty, it is common practice to call for transparency from AI. Thus, most systems display explanations immediately without considering the potential downsides. Still, it is uncertain when users actually need an explanation and how thoroughly they scrutinize them versus viewing them as a signal of competence. Since algorithms are vulnerable to biases and are often unable to identify incorrect advice, users need to think analytically about explanations to avoid misjudgments. If users proactively demand an explanation of the recommendation, it becomes increasingly likely that they cognitively evaluate it. Therefore, this study investigates how the accessibility of explanations—provided immediately or on-demand—affects non-expert users’ willingness to rely on AI recommendations. In addition, this research analyzes whether users request an explanation in the first place and whether personal-driven or contextual factors influence the decision for or against an explanation. By examining the interaction between an AI recommendation system and non-expert users, this experiment reveals that many non-expert users abstain from requesting verbal explanations despite performing significantly better with them. It sheds light on the circumstances prompting users to seek explanations, demonstrating that personal factors drive general explanation demands, while contextual factors influence the number of explanations requested. Surprisingly, participants attribute similar importance to on-demand and immediately provided explanations, indicating that increased transparency through verbal explanations results in unfounded reliance.