Assessing Computational Thinking Using Log Data: A Scoping Review
摘要
With the increasing presence of technology in education, foundational skills that can encourage learners’ understanding of and meaningful interactions with technology should not be overlooked. Computational thinking (CT) is one such skill with applications to multiple domains. While CT assessment has traditionally focused on final outcomes, a more comprehensive perspective can emerge by also examining the process through means such as log data. The aims of this scoping review are to map how existing research has utilized log data to assess CT and to present research gaps for future studies in this area. Multiple databases from different disciplines were searched for peer-reviewed journal articles and conference papers. Relevant studies analyzed log data of individual learners from open-ended tasks to gauge CT performance. The following five areas were examined from each study: 1) form of CT, 2) education level, 3) log data process features, 4) interpretations of the features, and 5) theoretical contextualization of the features. The analysis revealed text-based programming as the most researched form of CT, followed by block-based programming. Additionally, higher education was the most studied level of education. Regarding log data features, attempts and time on task were among the most extracted. In studies justifying the choice of features, a variety of theories and empirical studies were referenced. The suggested interpretation of the features also varied, leading to inconsistent conclusions about what could be inferred from the log data. This valuable insight into existing research patterns can steer future studies towards theory-based measures in underrepresented areas of research, advancing the discussion on how technology can enhance learning. Furthermore, using log data for assessment can enable scalable personalized feedback and scaffolding. Combining a strong theoretical understanding of these topics from educational sciences with the technology-oriented approach to log data could substantially promote new ways of developing effective interventions.