Applying Topic Modeling to Understand Assessment Practices of U.S. College Instructors in Response to the COVID-19 Pandemic

We sought to understand the U.S. college instructors’ perceptions of the impact of COVID-19 on their instructional and assessment practices of college instructors. Between February and July 2021, 145 faculty teaching at over 80 different U.S. institutions (Meanage = 41.29 years, SDage = 10.95; %female = 69.7) completed an online survey. Given the novelty of the circumstances, and the lack of existing measures to adequately study the context at the time, the survey consisted mainly of constructed response questions. To analyze such a volume of written responses, we conducted topic modeling using latent Dirichlet allocation (LDA), a machine learning approach Blei et al. J Mach Learn Res 3:993–1022 2003 [8]. This method is appropriate for identifying key themes within a set of responses. Most instructors reported that during Spring 2020 at least one course they taught shifted from an in-person to an online format (88.7%). Topic models based on instructors’ responses to open-ended questions provided additional insights about how instructors prioritized content, adjusted grading policies, prepared students to complete exams, navigated the challenges of administering online assessments, and addressed concerns of academic integrity. Even in non-emergency teaching situations, recommendations for best practices for online and remotely delivered exams in higher education contexts are influenced by an understanding of assessment-related challenges during COVID-19. The findings from the present study not only provide an application of a machine learning approach such as topic modeling, but also contribute to a growing understanding of how COVID-19 affected assessment practices in higher education.