Development and initial experience of a novel Competency-Based Assessment of Robotic Surgery Skills (CARS) scale for general surgery residents

As general surgery education and credentialing shifts towards a competency basis, there is a need for updated task-based scoring scales to correlate, especially in robotic surgery. The purpose of this study is to introduce a novel Competency-Based Assessment of Robotic Surgery Skills (CARS) Scale and report on its initial potential to evaluate general surgery resident development of robotic surgery skills.

Resident robotic surgery performance was evaluated at a single institution using the CARS scale over the course of one academic year. Twenty de-identified and randomized operative robotic procedure videos were scored using the CARS and Global Evaluative Assessment of Robotic Skills (GEARS) scales by two expert blinded reviewers. Inter- and intra-rater reliability was analyzed.

A total of seven Post-Graduate Year (PGY)-1, seven PGY-2, six PGY-3, four PGY-4, and five PGY-5 general surgery residents were evaluated in-person using the CARS scale. Robot docking exhibited the highest mean score across all PGY levels, while robotic stapler use exhibited the lowest score. In-person assessments utilizing the CARS scale reliably differentiated advanced (PGY-4–5) resident performance from both intermediate (PGY-3) and beginner (PGY-1-2) resident performance. Additionally, the CARS scale reliably differentiated intermediate (PGY-3) from beginner (PGY-1-2) resident performance. In blinded video review, both the CARS and GEARS scales reliably differentiated attending surgeon from beginner, intermediate, and advanced resident performance. Inter-rater reliability of the CARS scale was moderate when used to evaluate inguinal hernia repair video samples.

The CARS scale represents a first step towards establishing competency-based scoring of general surgery resident robotic surgery performance. Further studies are necessary to evaluate the potential of the CARS scale to replace non-competency-based robotic surgery scoring scales.