Exploring the Impact of Explainability on Trust and Acceptance of Conversational Agents – A Wizard of Oz Study

With advancements in natural language processing and understanding, conversational agents (CAs) have become one of the fundamental modes of human-computer interaction. However, the black-box problem of AI algorithms often results in reduced acceptance of such systems. This calls for transparency and justification or rationale for the provided output from the users’ perspective. Explainable artificial intelligence (XAI) provides insights into the algorithms and elucidates outputs to the users, thus gaining more importance in various applications as a significant contributor to user acceptance and trust in artificial intelligence (AI) systems. This paper presents a Wizard of Oz user study with a between-subjects design comparing two versions of a vacation planning chatbot (low and high explainability) with 60 participants. The study explored the impact of explainability on users’ understanding, trust and acceptance. The results indicated that explanations (between-subject factor) significantly influence users’ understanding, trust and acceptance. According to our results, high explainability leads to increased trust and acceptance of the chatbot.