Data for Learning and Program Evaluation: Managing, Analyzing, and Reporting OSCE Data
摘要
Assessing performance is usually a sine qua non—an essential aspect—of the OSCE, whether the results are to be used for formative feedback, summative grading, entrustable or milestone judgments, program evaluation, or all of the above. In Chap. 2 (Organizing OSCEs (and Other SP Exercises) in Ten Steps), Steps 5, 7, and 9 provide broad overviews and practical tips for, respectively, designing assessments, training assessors, and collecting, managing, and analyzing OSCE data. In this chapter, we aim to provide a more in-depth guide for maximizing the educational yield of OSCE data: wrangling, analyzing, visualizing, and reporting on OSCE assessment data to support learning and skill development at all levels, including individual learners, faculty educators and coaches, and education/training programs. While the specifics of data and statistical and psychometric analyses are certainly beyond the scope that can be covered in a chapter, as are the wealth of new programs and applications used in data collection and investigation, we hope to provide readers with a relatively simple and understandable outline for how to approach OSCE data management, reporting, and analysis. Ultimately, we believe that with the explosion of data science advances (e.g., the era of big data, growth and adoption of new analytic programming languages such as R and Python and platforms such as Tableau and other dynamic dashboard applications, artificial intelligence/machine learning, and the increasing analytic sophistication of the education community), there is tremendous opportunity for enhancing our effective and efficient use of OSCE assessment data to achieve the mission of ensuring a highly skilled and competent health professional workforce.