Scholarlink

<p>This classroom-embedded case study investigates a multidimensional diagnostic learning analytics pipeline for programming education. Specifically, we propose the multidimensional cognitive diagnosis–feedback–orchestration (MCD-FO) framework to connect granular artifact evidence with moment-to-moment scaffolding and cohort-level decision-making. To evaluate this approach, we conducted an empirical study in an undergraduate programming course where 327 students submitted 1423 milestone artifacts. A rubric-constrained large language model (LLM) agent annotated these submissions across 29 indicators within four dimensions—syntactic understanding (SU), algorithmic reasoning (AR), debugging strategies (DS), and metacognitive regulation (MR). The framework yielded highly reliable cognitive-diagnostic profiles (overall kappa = 0.82). Results showed that students exhibited significant pre-to-post gains on the Computer Programming Cognitive Development Assessment (dz = 1.50) and steady weekly improvements in weighted cognitive-diagnostic profile score, alongside declining severe runtime errors. Epistemic network analysis further revealed progressively stronger co-occurrence patterns among cognitive dimensions, particularly in SU–AR and AR–DS linkages. Triangulated evidence indicated that dashboard usage supported instructors in making data-informed pedagogical adaptations, including targeted remediation and adaptive pacing. The practical implementation of the MCD-FO framework provides a cognitively grounded and replicable pathway for scalable, auditable personalized education.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Artifact-based diagnostic learning analytics via large language models: a case study in programming classroom

  • 期刊论文

  • 发表日期：

    2026年7月6日

查看全文
  • Kai Liang

摘要

This classroom-embedded case study investigates a multidimensional diagnostic learning analytics pipeline for programming education. Specifically, we propose the multidimensional cognitive diagnosis–feedback–orchestration (MCD-FO) framework to connect granular artifact evidence with moment-to-moment scaffolding and cohort-level decision-making. To evaluate this approach, we conducted an empirical study in an undergraduate programming course where 327 students submitted 1423 milestone artifacts. A rubric-constrained large language model (LLM) agent annotated these submissions across 29 indicators within four dimensions—syntactic understanding (SU), algorithmic reasoning (AR), debugging strategies (DS), and metacognitive regulation (MR). The framework yielded highly reliable cognitive-diagnostic profiles (overall kappa = 0.82). Results showed that students exhibited significant pre-to-post gains on the Computer Programming Cognitive Development Assessment (dz = 1.50) and steady weekly improvements in weighted cognitive-diagnostic profile score, alongside declining severe runtime errors. Epistemic network analysis further revealed progressively stronger co-occurrence patterns among cognitive dimensions, particularly in SU–AR and AR–DS linkages. Triangulated evidence indicated that dashboard usage supported instructors in making data-informed pedagogical adaptations, including targeted remediation and adaptive pacing. The practical implementation of the MCD-FO framework provides a cognitively grounded and replicable pathway for scalable, auditable personalized education.