Intent-Based vs. LLM-Based Chatbots in MOOCs: Do Diverse Students Experience Them Differently?
摘要
Despite the growing adoption of chatbots in massive open online courses (MOOCs), little research has examined how demographic factors shape students’ learning experiences with different chatbot types. Existing studies focus on the general effectiveness of intent-based and large language model (LLM)-based chatbots, but how student characteristics influence their perceptions and use remains unclear. This study investigates how age, gender, socioeconomic region, and native language influence students’ learning experiences with intent-based and LLM-based chatbots across five domains: social presence, teaching and cognitive presence, self-regulation, perceived ease of use, and behavioral intention to use. Survey data were collected from two MOOC courses, one utilizing an intent-based (n = 340) and another an LLM-based chatbot (n = 121). The results revealed that age significantly influenced students’ learning experiences with the LLM-based chatbot, as younger students reported higher social presence, perceived ease of use, and behavioral intention to use it, whereas no significant age-related differences were found in the intent-based chatbot group. Socioeconomic status affected both chatbot users’ social presence and teaching and cognitive presence, with lower-middle-income students reporting stronger levels in these learning domains across both chatbot groups. However, gender and native language had no significant effects in either chatbot group. These findings suggest that LLM-based chatbots may be more effective for younger students, while intent-based chatbots could support older students better. Additionally, the stronger presence perceptions among lower-middle-income students indicate the potential of chatbots to address educational disparities. Future research should explore hybrid chatbot models that integrate AI-driven flexibility with structured guidance to optimize learning experiences across diverse MOOC students.