Scholarlink

Objective <p>Accreditation bodies are driving competency-based education in healthcare, prompting curriculum reform. Simulation-based education (SBE) addresses challenges curriculum reform has uncovered, like lack of standardization in bedside teaching. This study explores the impact of an AI-powered Automated System Protocol (ASP) for grading students’ post-encounter notes in Clerkship OSCEs, comparing it to the legacy human grader system.</p> Methods <p>The ASP, utilizing GPT-4, mapped rubric items to prompts. Analyzing post-encounter notes from 684 medical students across four academic years, we compared ASP with legacy Standardized Patient Evaluator (SPE) grades. Time efficiency, cost savings, and ROI analyses assessed educational and financial implications.</p> Results <p>Significant cost savings and efficiency gains were observed utilizing GPT-4 in comparison to SPEs. The Cost of Investment for ASP totaled $69,112 over 1,150&#xa0;h. Comparing ASP to three SP graders yielded $13,112 in increased costs and initial time investment was required. However, beyond development time ASP execution-only, compared to legacy, showed an ROI of 589.44%, saving $47,877 with 87.5% time efficiency. ASP-execution versus three MD graders demonstrated an even stronger ROI of 797.09%.</p> Conclusion <p>Implementing ASP in medical education provides substantial time and cost savings, enhancing ROI compared to legacy grading models. These findings highlight significant cost savings and efficiency improvements achievable through ASP implementation, positioning automated assessment as an innovative force shaping the future of medical education. By liberating human resources from manual grading and enhancing the immediacy of feedback, this approach contributes to a more efficient, effective, and engaging learning experience.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Applying state-of-the-art artificial intelligence to grading in simulation-based education: assessment, feedback, and ROI

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年8月8日

查看全文
  • Krystle K. Campbell,
  • Michael J. Holcomb,
  • Sol Vedovato,
  • Lenora Young,
  • Gaudenz Danuser,
  • Thomas O. Dalton,
  • Andrew R. Jamieson,
  • Daniel J. Scott

摘要

Objective

Accreditation bodies are driving competency-based education in healthcare, prompting curriculum reform. Simulation-based education (SBE) addresses challenges curriculum reform has uncovered, like lack of standardization in bedside teaching. This study explores the impact of an AI-powered Automated System Protocol (ASP) for grading students’ post-encounter notes in Clerkship OSCEs, comparing it to the legacy human grader system.

Methods

The ASP, utilizing GPT-4, mapped rubric items to prompts. Analyzing post-encounter notes from 684 medical students across four academic years, we compared ASP with legacy Standardized Patient Evaluator (SPE) grades. Time efficiency, cost savings, and ROI analyses assessed educational and financial implications.

Results

Significant cost savings and efficiency gains were observed utilizing GPT-4 in comparison to SPEs. The Cost of Investment for ASP totaled $69,112 over 1,150 h. Comparing ASP to three SP graders yielded $13,112 in increased costs and initial time investment was required. However, beyond development time ASP execution-only, compared to legacy, showed an ROI of 589.44%, saving $47,877 with 87.5% time efficiency. ASP-execution versus three MD graders demonstrated an even stronger ROI of 797.09%.

Conclusion

Implementing ASP in medical education provides substantial time and cost savings, enhancing ROI compared to legacy grading models. These findings highlight significant cost savings and efficiency improvements achievable through ASP implementation, positioning automated assessment as an innovative force shaping the future of medical education. By liberating human resources from manual grading and enhancing the immediacy of feedback, this approach contributes to a more efficient, effective, and engaging learning experience.