Multi-robot Path Planning Based on IPPO Reinforcement Learning and Imitation Learning
摘要
With the growing application of intelligent agents in real-world scenarios, multi-agent path planning in dynamic and time-sensitive environments has become increasingly important. However, traditional centralized planning methods struggle to adapt to real-time disturbances, while existing distributed reinforcement learning-based approaches suffer from sparse environmental rewards and long training times. To address these challenges, this study proposes a multi-agent reinforcement learning framework based on Independent Proximal Policy Optimization to solve the Multi-Agent Path Finding problem. In this framework, each agent independently learns its own policy using local observations and normalized goal directions. To improve training efficiency, we further integrate imitation learning by introducing expert demonstrations generated from CBS. A dense reward structure is also designed, combining step-level feedback and global completion rewards. We conduct extensive experiments in a grid-based simulation environment, comparing IPPO with PPO and Actor-Critic. Results show that IPPO consistently outperforms PPO and AC in terms of task success rate, convergence speed, reward stability, and training robustness. Furthermore, the integration of IL with IPPO significantly improves early-stage performance and final policy effectiveness.