Scholarlink

<p>In-situ assistance plays a vital role in enhancing operational performance and process reliability in manufacturing. Conventional operator training systems often rely on manual supervision from experienced workers, which can lead to inefficiency, inconsistent guidance, and high labor costs. To address these limitations, this work introduces AssemblyMate, an intelligent XR-based assembly assistant designed to support human operators during complex assembly procedures. The system integrates two core components: a multimodal reasoning agent and a spatial-temporal reasoning agent, enabling dynamic guidance, real-time feedback and systematic evaluation for operators. Specifically, the multimodal reasoning agent provides context-aware instructions, demonstration videos, and interaction with operators in augmented reality through the learning process. Meanwhile, the spatial-temporal reasoning agent continuously analyzes action sequences and task states, enabling dynamic performance evaluation and coordinating with a collaborative robot to manage product transfer and assembly flow. The proposed framework is validated on two representative industrial scenarios—calculator assembly and vehicle assembly. It demonstrates that the integrated reasoning architecture significantly improves task efficiency, accuracy, and human-robot collaboration. These results highlight the potential of intelligent assistant to transform in-situ operations into adaptive, data-informed processes that advance workforce augmentation in smart manufacturing.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AssemblyMate: an interactive context-aware AI-XR co-worker with multimodal spatial-temporal reasoning for manufacturing assembly

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年8月4日

查看全文
  • Xueting Wang,
  • Xiwen Dengxiong,
  • Jie-Ke Pan,
  • Ke Jing,
  • Zipeng Guo,
  • Rui Liu,
  • Qinqin Xiao,
  • Yunbo Zhang

摘要

In-situ assistance plays a vital role in enhancing operational performance and process reliability in manufacturing. Conventional operator training systems often rely on manual supervision from experienced workers, which can lead to inefficiency, inconsistent guidance, and high labor costs. To address these limitations, this work introduces AssemblyMate, an intelligent XR-based assembly assistant designed to support human operators during complex assembly procedures. The system integrates two core components: a multimodal reasoning agent and a spatial-temporal reasoning agent, enabling dynamic guidance, real-time feedback and systematic evaluation for operators. Specifically, the multimodal reasoning agent provides context-aware instructions, demonstration videos, and interaction with operators in augmented reality through the learning process. Meanwhile, the spatial-temporal reasoning agent continuously analyzes action sequences and task states, enabling dynamic performance evaluation and coordinating with a collaborative robot to manage product transfer and assembly flow. The proposed framework is validated on two representative industrial scenarios—calculator assembly and vehicle assembly. It demonstrates that the integrated reasoning architecture significantly improves task efficiency, accuracy, and human-robot collaboration. These results highlight the potential of intelligent assistant to transform in-situ operations into adaptive, data-informed processes that advance workforce augmentation in smart manufacturing.