Scholarlink

<p>Success in organic chemistry often hinges on a learner’s ability to explain reaction mechanisms, where electrophiles and nucleophiles play a central role. Asking learners to explain <i>what</i> happens in a reaction and <i>why</i> can provide deeper insights into the sophistication of their understanding (<i>e.g.</i>, <i>Absent, Descriptive, Foundational, Complex</i>). However, scoring undergraduate learners' explanations is time-intensive and often impractical in traditionally large enrollment organic chemistry courses. In this work, we developed two machine learning models to address this challenge, with one model designed to automatically code learners’ written explanations of electrophiles and a second model designed to code explanations of nucleophiles in reaction mechanisms. Models were trained using 13,910 learner explanations across 85 reaction mechanisms. Model performance was evaluated using twofold cross-validation, as well as with additional explanations not previously seen by the models via hold-out validation (<i>n</i> = 6,026) and external validation (<i>n</i> = 1,039, using 5 additional reaction mechanisms) sets. A comprehensive collection of metrics (<i>e.g.</i>, Cohen’s kappa, F<sub>1</sub>, etc.) were used to evaluate each validation set; metrics demonstrated moderate performance (<i>e.g.</i>, percent accuracies ranging from ~ 71—92%) for both models across each set. Results of this work support model generalizability for a wide range of reaction mechanisms and use for formative assessment. This work highlights the practical use of machine learning models as tools for formative assessment and offers a scalable approach for insight into learners’ reasoning and conceptual understanding in organic chemistry.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Development of Machine Learning Models to Predict the Level of Explanation Sophistication for Organic Chemistry Mechanisms

  • 期刊论文

  • 发表日期：

    2026年3月9日

查看全文
  • Caroline J. Crowder,
  • Brandon J. Yik,
  • Stephanie J. H. Frost,
  • Daniel Cruz-Ramírez de Arellano,
  • Kimberly Bliss-Roche,
  • Jeffrey R. Raker

摘要

Success in organic chemistry often hinges on a learner’s ability to explain reaction mechanisms, where electrophiles and nucleophiles play a central role. Asking learners to explain what happens in a reaction and why can provide deeper insights into the sophistication of their understanding (e.g., Absent, Descriptive, Foundational, Complex). However, scoring undergraduate learners' explanations is time-intensive and often impractical in traditionally large enrollment organic chemistry courses. In this work, we developed two machine learning models to address this challenge, with one model designed to automatically code learners’ written explanations of electrophiles and a second model designed to code explanations of nucleophiles in reaction mechanisms. Models were trained using 13,910 learner explanations across 85 reaction mechanisms. Model performance was evaluated using twofold cross-validation, as well as with additional explanations not previously seen by the models via hold-out validation (n = 6,026) and external validation (n = 1,039, using 5 additional reaction mechanisms) sets. A comprehensive collection of metrics (e.g., Cohen’s kappa, F1, etc.) were used to evaluate each validation set; metrics demonstrated moderate performance (e.g., percent accuracies ranging from ~ 71—92%) for both models across each set. Results of this work support model generalizability for a wide range of reaction mechanisms and use for formative assessment. This work highlights the practical use of machine learning models as tools for formative assessment and offers a scalable approach for insight into learners’ reasoning and conceptual understanding in organic chemistry.