Scholarlink

<p>Extended Reality (XR) and Digital Twins (DTs) present promising opportunities for immersive, state-aware procedural learning, making them well-suited for step-by-step instructional tasks. Their ability to simulate real-world contexts aligns closely with experiential learning principles, which emphasize active engagement, reflection, and learning by doing in authentic environments. Building on this potential, our prior work introduced a DT-driven XR framework for learning procedural Rubik’s Cube notations, comparing three instructional modalities— a baseline visual static information condition with physical cube manipulation (M1), in-situ visual overlays with physical cube manipulation (M2), and in-situ visual overlays with virtual-then-physical cube manipulation (M3). However, the potential of multimodal guidance remained underexplored. This study extends the previous work by introducing and evaluating a fourth instructional modality, namely voice-augmented visual guidance (M4), which synchronizes auditory narration with in-situ visual overlays during physical cube manipulation. We assessed all four modalities across two post-training phases using task accuracy, completion time, and intra-action pause duration, alongside global measures of usability, acceptance, and cognitive effort. In the evaluated setting, M4 did not yield statistically significant differences relative to the other modalities (M1–M3), but achieved comparable performance and user-experience outcomes without additional cognitive or usability costs. These results delineate boundary conditions for multimodal instruction in this context and suggest that voice augmentation is best viewed as a complementary design element rather than an inherently performance-enhancing mechanism.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Voice-augmented extended reality digital twin instruction: the case of the Rubik’s Cube

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月19日

查看全文
  • Shirin Hajahmadi,
  • Pasquale Cascarano,
  • Lorenzo Stacchio,
  • Alessandro Giacché,
  • Gustavo Marfia

摘要

Extended Reality (XR) and Digital Twins (DTs) present promising opportunities for immersive, state-aware procedural learning, making them well-suited for step-by-step instructional tasks. Their ability to simulate real-world contexts aligns closely with experiential learning principles, which emphasize active engagement, reflection, and learning by doing in authentic environments. Building on this potential, our prior work introduced a DT-driven XR framework for learning procedural Rubik’s Cube notations, comparing three instructional modalities— a baseline visual static information condition with physical cube manipulation (M1), in-situ visual overlays with physical cube manipulation (M2), and in-situ visual overlays with virtual-then-physical cube manipulation (M3). However, the potential of multimodal guidance remained underexplored. This study extends the previous work by introducing and evaluating a fourth instructional modality, namely voice-augmented visual guidance (M4), which synchronizes auditory narration with in-situ visual overlays during physical cube manipulation. We assessed all four modalities across two post-training phases using task accuracy, completion time, and intra-action pause duration, alongside global measures of usability, acceptance, and cognitive effort. In the evaluated setting, M4 did not yield statistically significant differences relative to the other modalities (M1–M3), but achieved comparable performance and user-experience outcomes without additional cognitive or usability costs. These results delineate boundary conditions for multimodal instruction in this context and suggest that voice augmentation is best viewed as a complementary design element rather than an inherently performance-enhancing mechanism.