A Taxonomy for Enhancing Metacognitive Adaptivity and Personalization in Serious Games Using Multimodal Trace Data
摘要
This paper presents a theoretically-driven and empirically-based preliminary taxonomy for optimizing metacognitive adaptivity and personalization in serious games by leveraging multimodal trace data. By integrating diverse sources of multimodal trace data, such as eye tracking, log files, concurrent verbalizations, facial expressions of emotions, physiological sensors, and screen recordings, the taxonomy aims to capture nuanced insights into learners’ metacognitive processes during gameplay. Our taxonomy focuses on six specific metacognitive processes, including judgments of learning (JOLs), feelings of knowing (FOKs), content evaluations (CEs), monitoring progress towards goals (MPTG), and monitoring use of strategies (MUS), and self-questioning (SQ). These metacognitive processes are critical in learning, reasoning, and problem solving across several learning technologies, including serious games. We provide operational definitions and examples of how each process can be captured by each multimodal data channel during gameplay. More specifically, the taxonomy facilitates the development of serious games that dynamically adjust difficulty levels, provide personalized feedback, and offer tailored scaffolding to enhance metacognitive development using advanced machine learning techniques, including generative AI, for real-time multimodal analysis. Through this taxonomy, researchers and developers can design and evaluate adaptive serious games that optimize metacognitive awareness, monitoring, regulation, and reflection, contributing to advancing the science of learning with serious games. Lastly, future research needs to empirically test these recommendations, and we expect further refinements based on such testing with different serious games across various learners, tasks, domains, and educational contexts.