Software Tools for Analyzing Log Data
摘要
The analysis of log file data from computer-based large-scale assessments promises to provide new insights into test-taking processes by combining different perspectives and research disciplines. To gain such insights, the extraction of low-level features and the aggregation to process indicators is necessary. However, log file data generated by assessment software cannot be analyzed using standard educational research and psychometric modeling tools directly. Based on the typical preparation for results data, this chapter starts with a description of a workflow for log data preparation and the separation of feature extraction and derived process indicators. Five requirements for software tools for log file analyses are derived from the workflow: the need to handle the contextual dependency of log events (R1), openness regarding the developed workflow (R2), a design that enables reproducibility of log file analyses (R3), extensibility to allow extraction of new features (R4), and versatility to use data from different assessment platforms (R5). The requirements are then used to compare three different tools: the general R environment, a specific R package (LogFSM), and the PIAAC Log Data Analyzer. No gold standard yet exists concerning the level of detail at which log file data from large-scale assessments are made available to data users. Hence, the closing discussion summarizes how tools for the analyses of log data could further be developed and used by data providers and data users.