3D path planning of unmanned ground vehicles based on improved DDQN

For safe and efficient path planning of unmanned ground vehicles (UGVs) in complex three-dimensional (3D) environment, this paper proposes an improved deep reinforcement learning algorithm (dual experience dynamic target DDQN, DEDT DDQN) to solve sparse reward and overestimation that are difficulties for traditional DDQN algorithms. The algorithm improves the performance of the DDQN algorithm in complex environments by incorporating high-quality experiences with dynamic weighting and dynamically integrating the prior knowledge of DDQN and averaged DDQN during network parameter training. For 3D maps, this paper adopts a digital elevation model (DEM) that considers environmental features and time costs for path planning. The DEDT DDQN algorithm was tested in a simulation experiment on a 3D map that models a real-world environment. The results showed that the DEDT DDQN algorithm reduced the number of turning points and the average slope by 40 and 16.7%, respectively. Additionally, it improved optimization performance and convergence speed by 5.34 and 60%, respectively. The proposed DEDT DDQN algorithm can be applied to two different types of maps, demonstrating its effectiveness and robustness.