Empowering robots to accurately comprehend task commands and plan their actions aids in the transition of assembly robots from automation to intelligence. However, the prerequisite for understanding instructions is to master domain knowledge and be able to generalize. In this paper, we adopt a top-down approach to ontology design, initially highlighting nine fundamental concepts. Subsequently, we introduced internal relationships and proceeded to extract knowledge from robot assembly operations, thereby forming a specific domain knowledge graph, namely, area-part composite graph (APCG). It stores domain knowledge of 3C (an abbreviation for “computer, communication, and consumer electronics”) assembly structurally. Then we construct a knowledge-based assembly sequence reasoning (KBASR) that mainly integrates with the Graph Attention Network (GAT) and Transformer model. KBASR incorporates knowledge into the Transformer module by using knowledge embedding as initial feature vectors for task and action descriptions, as well as words in the target sequence. To embed input words into the Transformer module, the GAT module is utilized for embedding words found in APCG. In four sets of ablation experiments, KBASR achieves 99.38% accuracy and perplexity of 1.002 on average by introducing domain knowledge through GAT on top of the Transformer, resulting in an improvement in accuracy by 5.97% and a decrease in perplexity by 1.86%. Additionally, to validate the feasibility of the sequences, we constructed assembly scenarios in both the Unity environment and the real world, conducting experimental verification with various parts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Area-Part Composite Graph Guided Robot Manipulation Reasoning in 3C Assembly

  • Haiming Huang,
  • Zerui Wu,
  • Lianghong Wu,
  • Weiwei Chen,
  • Zhenkun Wen

摘要

Empowering robots to accurately comprehend task commands and plan their actions aids in the transition of assembly robots from automation to intelligence. However, the prerequisite for understanding instructions is to master domain knowledge and be able to generalize. In this paper, we adopt a top-down approach to ontology design, initially highlighting nine fundamental concepts. Subsequently, we introduced internal relationships and proceeded to extract knowledge from robot assembly operations, thereby forming a specific domain knowledge graph, namely, area-part composite graph (APCG). It stores domain knowledge of 3C (an abbreviation for “computer, communication, and consumer electronics”) assembly structurally. Then we construct a knowledge-based assembly sequence reasoning (KBASR) that mainly integrates with the Graph Attention Network (GAT) and Transformer model. KBASR incorporates knowledge into the Transformer module by using knowledge embedding as initial feature vectors for task and action descriptions, as well as words in the target sequence. To embed input words into the Transformer module, the GAT module is utilized for embedding words found in APCG. In four sets of ablation experiments, KBASR achieves 99.38% accuracy and perplexity of 1.002 on average by introducing domain knowledge through GAT on top of the Transformer, resulting in an improvement in accuracy by 5.97% and a decrease in perplexity by 1.86%. Additionally, to validate the feasibility of the sequences, we constructed assembly scenarios in both the Unity environment and the real world, conducting experimental verification with various parts.