Mobile robot path planning method based on sumtree-TD3 algorithm with transfer learning
摘要
In response to the urgent demand for high real-time performance and reliability in mobile robot path planning in complex dynamic environments, an improved Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is proposed, addressing the issues of slow convergence and low sample efficiency during training in the TD3 algorithm. This improved algorithm is designed for efficient computation and rapid deployment. Firstly, the noise exploration is designed based on the idea of Softmax, which increases the exploration intensity of the mobile robot near the obstacles. Secondly, the priority sampling experience pool sampling strategy of SumTree data structure is introduced into the algorithm to improve the utilization rate of samples. Then the reward function is designed based on the idea of artificial potential field and the path smoothness index, so that the robot can get more single-step rewards to avoid the problem of reward sparsity and at the same time improve the efficiency of path planning. The pre-trained network parameters in the source task environment are migrated to the target task environment through transfer learning to improve the learning speed of the robot and make the algorithm converge quickly. Finally, three different experimental scenarios are built in Gazebo environment for experiments and the feasibility of the proposed algorithm is verified in the real environment. The results of simulation experiments show that compared with the TD3 algorithm, the success rate algorithm of the proposed algorithm is improved by 14.2% for training in dynamic environments and 17.9% for training in static environments. The results of real environment experiments show that the path planning time of the proposed algorithm is reduced by 37.3% and the path length is shortened by 26.4% compared with the TD3 algorithm.