Learning Hierarchical Robot Skills Represented by Behavior Trees from Natural Language

Learning from natural language is a programming-free and user friendly teaching method that allows users without programming knowledge or demonstration capabilities to instruct robots, which has great value in industry and daily life. The manipulation skills of robots are often hierarchical skills composed of low-level primitive skills, so they can be conveniently represented by behavior trees (BTs). Based on this idea, we propose NL2BT, a framework for generating behavior trees from natural language and controlling robots to complete hierarchical tasks in real time. The framework consists of two language processing stages, an initial behavior tree library composed of primitive skill subtrees, and a BT-Generation algorithm. To validate the effectiveness of NL2BT, we use it to build a Chinese natural language system for instructing robots in performing 3C assembly tasks, which is a significant application of Industry 4.0. We also discuss the positive impact of real-time teaching, visual student models, and the synonymous skill module in the framework. In addition to the demonstrated application, NL2BT can be easily migrated to other languages and hierarchical task learning scenarios.