Adaptive learning technologies observe learner performance, infer mastery, and dynamically tailor instruction. However, this approach can fall short when encountering a learning phenomenon we term “deceptive overgeneralization”, where learners perform correct actions based on incomplete understanding. This phenomenon “deceives” adaptive systems into prematurely stopping necessary practice, leaving overgeneralization unaddressed. To address this, we empirically investigated the mechanisms, risks, and remediation strategies for deceptive overgeneralization through experiments using Intelligent Tutoring Systems (ITSs) for Riichi Mahjong. Experiment 1 provided evidence supporting the effectiveness of ITSs in teaching Riichi Mahjong, showing a large effect size. However, despite their overall effectiveness, Experiment 2 revealed that ITSs may fall short in cases of deceptive overgeneralization, as learners, after demonstrating seemingly satisfactory performance during initial practice, subsequently misapplied learned actions “confidently” in scenarios that did not warrant them. Experiments 3–4 replicated this finding and further revealed that adaptive learning systems relying on observed correctness can prematurely cease practice, leaving deceptive overgeneralization unaddressed. Experiments 5–6 replicated the findings of Experiments 3–4, and also extended the methodology by incorporating tailored practice designed via our systematic detection/remediation procedure, which successfully addressed deceptive overgeneralization. This study contributes both theoretically and practically to enhancing the precision of adaptive learning, enabling more accurate mastery assessments and improved learning outcomes.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Detect and Remedy Deceptive Overgeneralization in Adaptive Learning

  • Marshall An,
  • John Stamper

摘要

Adaptive learning technologies observe learner performance, infer mastery, and dynamically tailor instruction. However, this approach can fall short when encountering a learning phenomenon we term “deceptive overgeneralization”, where learners perform correct actions based on incomplete understanding. This phenomenon “deceives” adaptive systems into prematurely stopping necessary practice, leaving overgeneralization unaddressed. To address this, we empirically investigated the mechanisms, risks, and remediation strategies for deceptive overgeneralization through experiments using Intelligent Tutoring Systems (ITSs) for Riichi Mahjong. Experiment 1 provided evidence supporting the effectiveness of ITSs in teaching Riichi Mahjong, showing a large effect size. However, despite their overall effectiveness, Experiment 2 revealed that ITSs may fall short in cases of deceptive overgeneralization, as learners, after demonstrating seemingly satisfactory performance during initial practice, subsequently misapplied learned actions “confidently” in scenarios that did not warrant them. Experiments 3–4 replicated this finding and further revealed that adaptive learning systems relying on observed correctness can prematurely cease practice, leaving deceptive overgeneralization unaddressed. Experiments 5–6 replicated the findings of Experiments 3–4, and also extended the methodology by incorporating tailored practice designed via our systematic detection/remediation procedure, which successfully addressed deceptive overgeneralization. This study contributes both theoretically and practically to enhancing the precision of adaptive learning, enabling more accurate mastery assessments and improved learning outcomes.