Robotic Arm Control System for Vision-Language-Action
摘要
When it comes to the design and understanding of intelligent behavior in embodied and situated agents, embodied intelligence is the highest level of artificial intelligence (AI). It highlights the inherent relationship between the agent and its surroundings, which is impacted by the limitations of the agent’s body, perceptual and motor systems, and brain. This paper focuses on how to deploy embodied intelligence to the robotic arm of the grasping task and build a complete architecture from the overall implementation to the construction of specific links. The innovation of this paper lies in constructing a Vision-Language-Action (VLA) architecture. Firstly, the system receives instructions and uses the multi-modal vision model (depth camera with lidar or millimeter wave radar) to capture scene information and build a world model. Secondly, the already trained large language model (LLM) functions as the brain, reasoning and analyzing the order in which the instructions are executed in the real world to produce an instruction stream with a predetermined format. Finally, in order to run the robot, a bespoke compiler decodes the prepared instruction stream and translates it to the concrete control system’s interface.