Free-Form Instruction Guided Robotic Navigation Path Planning with Large Vision-Language Model
摘要
The integration of multiple pre-trained models in robotic navigation has the advantage of combining diverse strengths, leading to robust and generalized performance. However, the effectiveness of these models is often limited by path planning strategies, necessitating improvements in navigation capabilities. To overcome this, we introduce the Free-form Instruction Guided Robotic Navigation Path Planning with Large Vision-Language Model (FIG-RN). This model leverages free-form instructions to extract landmarks and directional cues, utilizing a pre-trained visual-language model to associate these landmarks with map nodes, thereby laying the groundwork for subsequent path planning. It evaluates landmark-node matches, node accessibility, and orientation to optimize path planning. Compared to traditional models, FIG-RN offers significant benefits: (i) it requires no map annotations due to its use of high-quality pre-trained models, (ii) it maximizes information use from instructions for better path efficacy, and (iii) it refines visual-language model matching values for improved local navigation. Experimentally, FIG-RN outperforms LM-Nav in success rate, efficiency, and accuracy, with improvements of 0.2, 0.2143, and 0.208, respectively.