Using simulation to assess basic ultrasound skills in plastic surgical trainees: a prospective cohort validation study

Ultrasound has become an invaluable diagnostic tool in plastic surgery. It is quick and easy to use, and it increases the safeness of invasive procedures. Its applications include among others identification of perforators when designing flaps, conducting safe seroma aspirations, and detecting lymph nodes. Despite its extensive use, a standardized competency test for ultrasound examination among junior level plastic surgical trainees has not yet been established. The study aimed to develop and gather validity evidence for a stimulation-based, three-station test in plastic surgery and to establish a credible pass/fail score.

An expert panel reached consensus on three relevant ultrasound scenarios for junior plastic surgeons: (1) ultrasound-guided seroma aspiration, (2) identification of a perforator for DIEP (deep inferior epigastric perforator) flap reconstruction, and (3) lymph node detection. Thirty-eight participants were categorized as novices, intermediates, or experienced, based on their number of performed ultrasound procedures. Performance was assessed using a modified OSAUS (Objective Structured Assessment of Ultrasound Skills) score. Validity evidence was gathered through Messick’s framework, and statistical analyses included Cronbach’s alpha for reliability and t-tests for group comparisons. A pass/fail score was determined using the contrasting groups’ method.

Mean scores were 27.9 ± 9.5 points for novices (n = 15), 78.3 ± 15.1 points (n = 12) for intermediates, and 88.2 ± 11.4 points (n = 11) for experienced. The test’s reliability was high, with a Cronbach’s alpha of 0.94. The pass/fail score was set to 56 out of 120 points. The test had no false negatives and one false positive.

This study established a reliable simulation-based ultrasound test for junior level plastic surgical trainees. It provided validity evidence and showed a correlation between experience and test performance.