Scholarlink

Background <p>The integration of artificial intelligence (AI) into clinical practice is reshaping the competency requirements for medical trainees. Yet, validated evaluation instruments aligned with outcome-based education (OBE) frameworks remain scarce.</p> Methods <p>We conducted a sequential mixed methods study to develop and preliminarily evaluate an OBE-based competency assessment matrix for clinical medical trainees in China. The framework was derived from national and international competency standards and refined through a three-round Delphi process with 16 medical education experts. Empirical evaluation involved 276 respondents including residents, postgraduate students, and clinical educators who completed the finalized 72-item instrument via a digital assessment platform. Reliability and exploratory structural characteristics were examined using Cronbach’s α, exploratory factor analysis (EFA), and inter-item correlation matrices. Subgroup differences were examined descriptively and visualized with radar plots.</p> Results <p>The Delphi panel reached consensus on 72 items across three domains—Importance, Feasibility, and Clarity—with progressive convergence (Kendall’s W ranging from 0.65 in Round 1 to 0.74 in Round 3). The resulting scale showed excellent internal consistency (Cronbach’s α = 0.928) and strong sampling adequacy (KMO = 0.884). Bartlett’s test of sphericity was highly significant (χ<sup>2</sup> = 421.35, df = 28, <i>p</i> &lt; 0.001), confirming the suitability of the data for structural exploration. EFA of aggregated domain scores yielded a three-component pattern that cumulatively explained 74.5% of the variance. The resulting loading profile suggested meaningful contributions of Importance, Feasibility, and Clarity, offering exploratory support for the proposed domain-level structure. Radar plots revealed systematic but role-dependent differences: faculty emphasized Importance, residents prioritized Feasibility, and postgraduates rated Clarity slightly higher.</p> Conclusion <p>This study provides a context-sensitive evaluation matrix with encouraging initial psychometric evidence, tailored to the evolving demands of AI-informed clinical education. The framework offers a promising platform for competency assessment and curriculum development in Chinese teaching hospitals and may serve as a reference model for other AI-integrating medical education systems, while highlighting the need for confirmatory factor analysis in independent samples to more definitively establish its dimensional structure.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Development and validation of a competency-based evaluation framework for clinical medical trainees in the era of artificial intelligence: a mixed-methods study in China

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月12日

查看全文
  • Feiyan Li,
  • Haojing Zhao,
  • Qiqiao Huang,
  • Chunhui Zhang,
  • Geyu Chen

摘要

Background

The integration of artificial intelligence (AI) into clinical practice is reshaping the competency requirements for medical trainees. Yet, validated evaluation instruments aligned with outcome-based education (OBE) frameworks remain scarce.

Methods

We conducted a sequential mixed methods study to develop and preliminarily evaluate an OBE-based competency assessment matrix for clinical medical trainees in China. The framework was derived from national and international competency standards and refined through a three-round Delphi process with 16 medical education experts. Empirical evaluation involved 276 respondents including residents, postgraduate students, and clinical educators who completed the finalized 72-item instrument via a digital assessment platform. Reliability and exploratory structural characteristics were examined using Cronbach’s α, exploratory factor analysis (EFA), and inter-item correlation matrices. Subgroup differences were examined descriptively and visualized with radar plots.

Results

The Delphi panel reached consensus on 72 items across three domains—Importance, Feasibility, and Clarity—with progressive convergence (Kendall’s W ranging from 0.65 in Round 1 to 0.74 in Round 3). The resulting scale showed excellent internal consistency (Cronbach’s α = 0.928) and strong sampling adequacy (KMO = 0.884). Bartlett’s test of sphericity was highly significant (χ2 = 421.35, df = 28, p < 0.001), confirming the suitability of the data for structural exploration. EFA of aggregated domain scores yielded a three-component pattern that cumulatively explained 74.5% of the variance. The resulting loading profile suggested meaningful contributions of Importance, Feasibility, and Clarity, offering exploratory support for the proposed domain-level structure. Radar plots revealed systematic but role-dependent differences: faculty emphasized Importance, residents prioritized Feasibility, and postgraduates rated Clarity slightly higher.

Conclusion

This study provides a context-sensitive evaluation matrix with encouraging initial psychometric evidence, tailored to the evolving demands of AI-informed clinical education. The framework offers a promising platform for competency assessment and curriculum development in Chinese teaching hospitals and may serve as a reference model for other AI-integrating medical education systems, while highlighting the need for confirmatory factor analysis in independent samples to more definitively establish its dimensional structure.