The role of holistic review in diversifying graduate medical education: a systematic review and meta-analysis

Diversity is crucial to the success of healthcare teams and inclusive patient care. The emphasis on traditional academic metrics has served as a barrier to the diversification of graduate medical education (GME) programs. Holistic review, defined by the Association of American Medical Colleges as an emphasis on experiences rather than academic metrics, has been proposed as a solution to improve diversity in medical education. However, skepticism about the implementation of holistic review and a lack of data to support that it improves diversity in GME exist. We, therefore, performed a systematic review and meta-analysis aiming to identify the components of holistic review described by programs employing it and quantify its impact on diversity. We hypothesized that programs implementing holistic review would have increased odds of interviewing and matriculating those that are underrepresented in medicine (URiM) and women when compared to those utilizing traditional applicant review.

PubMed and Embase were searched from inception to February 2023 for articles studying holistic review in United States GME programs. Articles were included if they described components of holistic review and/or compared the diversity of traditionally reviewed cohorts to those holistically reviewed. Two authors performed title/abstract screening; any disagreements were adjudicated by a third reviewer. Eligible studies were submitted to full-text screening. The association between holistic review and the proportion of URiM students interviewed/matriculated was assessed using random-effects meta-analysis. In addition, components of holistic review were categorized and described.

Of 201 screened abstracts, 21 articles were included. Four features of holistic review were consistently described by the included studies: (1) incorporating a mission-guided approach to applicant selection, (2) mitigating bias, (3) de-emphasizing academic metrics, (4) commitment to diversity, equity, and inclusion. Six studies compared proportions of URiM students interviewed using holistic versus traditional applicant review; holistic review was associated with increased odds of interviewing URiM students at these programs (pooled OR 2.30, 95% CI 1.19, 4.44; I2 = 85.66%). Five studies reported data on URiMs matriculated into their programs using holistic versus traditional applicant review; holistic review was also associated with increased pooled odds of URiM students matriculating into these programs (pooled OR 2.73, 95% CI 1.25, 5.95; I2 = 22.10%). There was insufficient data to complete meta-analysis on the effect of holistic review on proportions of women interviewed or matriculated into residency programs. The GRADE quality of evidence to support the implementation of holistic review to improve diversity in GME programs based upon the studies included in this review is moderate.

Holistic review is a feasible and effective way to increase diversity in GME programs. This systematic review outlines the different methods by which programs can screen, interview, and rank applicants with a mission-driven lens and increased emphasis on experiences and attributes.