As conversational AI agents are increasingly integrated into educational settings, understanding their cognitive impact on learners is essential. This study quantifies the cognitive load experienced by students solving GRE-style verbal reasoning problems with and without generative AI chatbot assistance. Using a within-subjects Wizard-of-Oz design, 31 university students completed equivalent verbal tasks under four controlled response conditions—Standard, Lengthy, Unstructured, and Ambiguous. We combined eye tracking metrics (pupil diameter and fixation duration) with subjective workload ratings to capture both autonomic and perceptual dimensions of mental effort. Results demonstrate that AI assistance improves overall task accuracy (from 30.6% to 74.0%) but that response quality critically modulates cognitive load. Specifically, Lengthy, Unstructured, and Ambiguous chatbot outputs elicited subjective scores, whereas concise, Structured (Standard) responses minimized both physiological arousal and perceived effort. These findings offer concrete design guidelines for chatbots, highlighting the value of clear, structured, and succinct responses to maximize learner success and minimize unnecessary effort.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

How AI Chatbot Response Style Affects Cognitive Load and Performance in Educational Tasks

  • Ashwini Srinivasaprasad,
  • Kamelia Sepanloo,
  • Saba Naderian Jahromi,
  • Denny Yu,
  • Vincent G. Duffy

摘要

As conversational AI agents are increasingly integrated into educational settings, understanding their cognitive impact on learners is essential. This study quantifies the cognitive load experienced by students solving GRE-style verbal reasoning problems with and without generative AI chatbot assistance. Using a within-subjects Wizard-of-Oz design, 31 university students completed equivalent verbal tasks under four controlled response conditions—Standard, Lengthy, Unstructured, and Ambiguous. We combined eye tracking metrics (pupil diameter and fixation duration) with subjective workload ratings to capture both autonomic and perceptual dimensions of mental effort. Results demonstrate that AI assistance improves overall task accuracy (from 30.6% to 74.0%) but that response quality critically modulates cognitive load. Specifically, Lengthy, Unstructured, and Ambiguous chatbot outputs elicited subjective scores, whereas concise, Structured (Standard) responses minimized both physiological arousal and perceived effort. These findings offer concrete design guidelines for chatbots, highlighting the value of clear, structured, and succinct responses to maximize learner success and minimize unnecessary effort.