Costmap A* Guided Reinforcement Learning Path Planning Method for Complex Environments Navigation

This paper presents a costmap A* guided soft actor-critic (CMA-SAC) path planning method to optimize the navigation performance of robots in long-distance and complex environments. Initially, a costmap is constructed to calculate the cost for approaching obstacles. With the costmap, the improved A* algorithm effectively avoids the paths being too close to obstacles. Subsequently, a local path planner based on deep reinforcement learning is constructed to directly generate control commands for the robot. Lastly, a tightly coupled strategy of global and local path planning is employed, where the results of global path planning are incorporated as part of the input to the deep neural network and integrated into the reward function of reinforcement learning (RL). Simulation experiments indicate that the CMA-SAC method outperforms deep deterministic policy gradient and SAC algorithms in terms of learning speed and stability during training. And in the test tasks, the CMA-SAC method performs better than other RL-based methods in navigation efficiency and has better dynamic obstacle avoidance performance than dynamic window approach. The proposed method has a success rate of 95.8% in the maze environment and the highest success rate in the long-distance and dynamic environment, demonstrating the method’s ability in complex navigation tasks.