Detect and Remedy Deceptive Overgeneralization in Adaptive Learning
摘要
Adaptive learning technologies observe learner performance, infer mastery, and dynamically tailor instruction. However, this approach can fall short when encountering a learning phenomenon we term “deceptive overgeneralization”, where learners perform correct actions based on incomplete understanding. This phenomenon “deceives” adaptive systems into prematurely stopping necessary practice, leaving overgeneralization unaddressed. To address this, we empirically investigated the mechanisms, risks, and remediation strategies for deceptive overgeneralization through experiments using Intelligent Tutoring Systems (ITSs) for Riichi Mahjong. Experiment 1 provided evidence supporting the effectiveness of ITSs in teaching Riichi Mahjong, showing a large effect size. However, despite their overall effectiveness, Experiment 2 revealed that ITSs may fall short in cases of deceptive overgeneralization, as learners, after demonstrating seemingly satisfactory performance during initial practice, subsequently misapplied learned actions “confidently” in scenarios that did not warrant them. Experiments 3–4 replicated this finding and further revealed that adaptive learning systems relying on observed correctness can prematurely cease practice, leaving deceptive overgeneralization unaddressed. Experiments 5–6 replicated the findings of Experiments 3–4, and also extended the methodology by incorporating tailored practice designed via our systematic detection/remediation procedure, which successfully addressed deceptive overgeneralization. This study contributes both theoretically and practically to enhancing the precision of adaptive learning, enabling more accurate mastery assessments and improved learning outcomes.