This paper presents a theoretically-driven and empirically-based preliminary taxonomy for optimizing metacognitive adaptivity and personalization in serious games by leveraging multimodal trace data. By integrating diverse sources of multimodal trace data, such as eye tracking, log files, concurrent verbalizations, facial expressions of emotions, physiological sensors, and screen recordings, the taxonomy aims to capture nuanced insights into learners’ metacognitive processes during gameplay. Our taxonomy focuses on six specific metacognitive processes, including judgments of learning (JOLs), feelings of knowing (FOKs), content evaluations (CEs), monitoring progress towards goals (MPTG), and monitoring use of strategies (MUS), and self-questioning (SQ). These metacognitive processes are critical in learning, reasoning, and problem solving across several learning technologies, including serious games. We provide operational definitions and examples of how each process can be captured by each multimodal data channel during gameplay. More specifically, the taxonomy facilitates the development of serious games that dynamically adjust difficulty levels, provide personalized feedback, and offer tailored scaffolding to enhance metacognitive development using advanced machine learning techniques, including generative AI, for real-time multimodal analysis. Through this taxonomy, researchers and developers can design and evaluate adaptive serious games that optimize metacognitive awareness, monitoring, regulation, and reflection, contributing to advancing the science of learning with serious games. Lastly, future research needs to empirically test these recommendations, and we expect further refinements based on such testing with different serious games across various learners, tasks, domains, and educational contexts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Taxonomy for Enhancing Metacognitive Adaptivity and Personalization in Serious Games Using Multimodal Trace Data

  • Roger Azevedo,
  • Daryn Dever,
  • Megan Wiedbusch,
  • Annamarie Brosnihan,
  • Tara Delgado,
  • Cameron Marano,
  • Milouni Patel,
  • Kevin Smith

摘要

This paper presents a theoretically-driven and empirically-based preliminary taxonomy for optimizing metacognitive adaptivity and personalization in serious games by leveraging multimodal trace data. By integrating diverse sources of multimodal trace data, such as eye tracking, log files, concurrent verbalizations, facial expressions of emotions, physiological sensors, and screen recordings, the taxonomy aims to capture nuanced insights into learners’ metacognitive processes during gameplay. Our taxonomy focuses on six specific metacognitive processes, including judgments of learning (JOLs), feelings of knowing (FOKs), content evaluations (CEs), monitoring progress towards goals (MPTG), and monitoring use of strategies (MUS), and self-questioning (SQ). These metacognitive processes are critical in learning, reasoning, and problem solving across several learning technologies, including serious games. We provide operational definitions and examples of how each process can be captured by each multimodal data channel during gameplay. More specifically, the taxonomy facilitates the development of serious games that dynamically adjust difficulty levels, provide personalized feedback, and offer tailored scaffolding to enhance metacognitive development using advanced machine learning techniques, including generative AI, for real-time multimodal analysis. Through this taxonomy, researchers and developers can design and evaluate adaptive serious games that optimize metacognitive awareness, monitoring, regulation, and reflection, contributing to advancing the science of learning with serious games. Lastly, future research needs to empirically test these recommendations, and we expect further refinements based on such testing with different serious games across various learners, tasks, domains, and educational contexts.