Higher education systems have long struggled to provide students with flexible, on-demand academic support and feedback, and this has contributed to underperformance and dropout. Large language models (LLMs) offer a new, scalable source of student support, but their use in academic learning environments remains mostly unmonitored, and there is mixed evidence of their impact on student outcomes. To understand how university students use LLMs for help with coursework, we developed an LLM-powered educational platform called HiTA, which serves as a course assistant with access to instructor-provided course materials. We deployed the system to thousands of students across several courses and universities. Our analysis of student interaction data indicates that the system effectively addresses gaps in support during evenings and nights and that engagement is higher in introductory courses. Aligning the system with established pedagogical frameworks, such as inquiry-based learning, remains challenging. Students frequently deviate from structured learning paths, and LLMs struggle to generate probing, higher-order questions to promote critical thinking and productive struggle. Our findings highlight significant opportunities for increased student support, the need for educators in the development process, and the importance of grounding LLM-based edtech in principles of good pedagogy.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Understanding Student Engagement with Large Language Model-Powered Course Assistants

  • Chang Liu,
  • Loc Hoang,
  • Andrew Stolman,
  • Rene F. Kizilcec,
  • Bo Wu

摘要

Higher education systems have long struggled to provide students with flexible, on-demand academic support and feedback, and this has contributed to underperformance and dropout. Large language models (LLMs) offer a new, scalable source of student support, but their use in academic learning environments remains mostly unmonitored, and there is mixed evidence of their impact on student outcomes. To understand how university students use LLMs for help with coursework, we developed an LLM-powered educational platform called HiTA, which serves as a course assistant with access to instructor-provided course materials. We deployed the system to thousands of students across several courses and universities. Our analysis of student interaction data indicates that the system effectively addresses gaps in support during evenings and nights and that engagement is higher in introductory courses. Aligning the system with established pedagogical frameworks, such as inquiry-based learning, remains challenging. Students frequently deviate from structured learning paths, and LLMs struggle to generate probing, higher-order questions to promote critical thinking and productive struggle. Our findings highlight significant opportunities for increased student support, the need for educators in the development process, and the importance of grounding LLM-based edtech in principles of good pedagogy.