Scholarlink

Background <p>Multiple Choice Questions (MCQs) remain the most widely used written assessment format in health professions education due to their efficiency and reliability; however, their susceptibility to cueing raises concerns regarding their authenticity in assessing true knowledge. Very Short Answer Questions (VSAQs) have emerged as a promising alternative that reduces cueing effects and requires learners to generate responses with greater cognitive engagement, while retaining the logistical advantages of computer-marked assessments. This systematic review and meta-analysis evaluates the comparative performance, discrimination, reliability, and practice effects of VSAQs versus MCQs.</p> Methods <p>Following PRISMA guidelines (31), a systematic search identified studies comparing VSAQs and MCQs in undergraduate and postgraduate health professions education. Data extraction and risk of bias assessment were conducted independently by two reviewers, with disagreements resolved through consensus. Mean scores, standard deviations, discrimination indices, and reliability coefficients were extracted. Standardized Mean Differences (SMD) were pooled using a random-effects model. Psychometric indices were transformed using Fisher’s z-scores. Heterogeneity was quantified using the I² statistic, and sensitivity and subgroup analyses were performed to explore the impact of outliers.</p> Results <p>Six cohorts (<i>n</i> = 1,191) derived from three included studies contributed data for score comparisons. The initial pooled analysis showed no significant difference between VSAQs and MCQs (SMD = − 0.52; 95% CI − 1.34 to 0.30), with extreme heterogeneity (I² = 98%). Sensitivity analysis identified Dhok et al. (2023) as an outlier. Excluding this dataset yielded a significant effect favoring MCQs (SMD = − 0.86; 95% CI − 1.01 to − 0.70; <i>p</i> &lt; 0.00001) with markedly reduced heterogeneity (I² = 4%). Meta-analysis of psychometric properties demonstrated strong discrimination (pooled Fisher’s z = 1.45) and acceptable reliability (pooled Fisher’s z = 0.43) for VSAQs. Evidence on practice effects was mixed, with no consistent advantage for either format.</p> Conclusion <p>VSAQs exhibit strong psychometric integrity and are designed to reduce the cueing effect observed in MCQs. Although associated with lower student scores, VSAQs likely provide a more accurate reflection of independent knowledge and clinical reasoning ability. Their integration into medical assessment systems may enhance the authenticity and validity of written examinations.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Very short answer questions vs. multiple choice questions in health professions education: a systematic review and meta-analysis

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月1日

查看全文
  • Ali Sher,
  • Muhammad Moin Ud Din Arshad,
  • Mesum Abbas,
  • Abdul Hannan Asghar,
  • Anees Hussain Khan,
  • Junaid Ali,
  • Muhammad Ibrahim Yaqoob,
  • Rana Aftab Alam,
  • Muhammad Bilal Akram,
  • Muhammad Abdullah Masood,
  • Amit Kumar Thakur

摘要

Background

Multiple Choice Questions (MCQs) remain the most widely used written assessment format in health professions education due to their efficiency and reliability; however, their susceptibility to cueing raises concerns regarding their authenticity in assessing true knowledge. Very Short Answer Questions (VSAQs) have emerged as a promising alternative that reduces cueing effects and requires learners to generate responses with greater cognitive engagement, while retaining the logistical advantages of computer-marked assessments. This systematic review and meta-analysis evaluates the comparative performance, discrimination, reliability, and practice effects of VSAQs versus MCQs.

Methods

Following PRISMA guidelines (31), a systematic search identified studies comparing VSAQs and MCQs in undergraduate and postgraduate health professions education. Data extraction and risk of bias assessment were conducted independently by two reviewers, with disagreements resolved through consensus. Mean scores, standard deviations, discrimination indices, and reliability coefficients were extracted. Standardized Mean Differences (SMD) were pooled using a random-effects model. Psychometric indices were transformed using Fisher’s z-scores. Heterogeneity was quantified using the I² statistic, and sensitivity and subgroup analyses were performed to explore the impact of outliers.

Results

Six cohorts (n = 1,191) derived from three included studies contributed data for score comparisons. The initial pooled analysis showed no significant difference between VSAQs and MCQs (SMD = − 0.52; 95% CI − 1.34 to 0.30), with extreme heterogeneity (I² = 98%). Sensitivity analysis identified Dhok et al. (2023) as an outlier. Excluding this dataset yielded a significant effect favoring MCQs (SMD = − 0.86; 95% CI − 1.01 to − 0.70; p < 0.00001) with markedly reduced heterogeneity (I² = 4%). Meta-analysis of psychometric properties demonstrated strong discrimination (pooled Fisher’s z = 1.45) and acceptable reliability (pooled Fisher’s z = 0.43) for VSAQs. Evidence on practice effects was mixed, with no consistent advantage for either format.

Conclusion

VSAQs exhibit strong psychometric integrity and are designed to reduce the cueing effect observed in MCQs. Although associated with lower student scores, VSAQs likely provide a more accurate reflection of independent knowledge and clinical reasoning ability. Their integration into medical assessment systems may enhance the authenticity and validity of written examinations.