Scholarlink

<p>Unmanned aerial vehicle (UAV) path planning in complex 3D environments poses significant challenges for reinforcement learning agents, particularly in efficiently navigating around obstacles while ensuring goal reachability. Traditional deep <i>Q</i>-network (DQN) approach often struggles with slow convergence, suboptimal exploration, overestimation bias, and poor generalization in complex 3D UAV navigation tasks. These limitations are primarily due to sparse-reward signals, redundant action selections, and the inability to effectively resolve action-value ties. Inherent overestimation issue of DQN leads to unstable policies and suboptimal path choices especially when navigating high-dimensional obstacle-cluttered UAV environments. To address these issues, we propose an improved double DQN (DDQN) framework for UAV path planning that incorporates two key enhancements: dynamic reward shaping and heuristic-guided tie-breaking in case of nearly equal actions. The reward shaping strategy provides a denser learning signal by assigning rewards based on the agent’s Euclidean distance to the target which encourages steady progress towards the goal. Additionally, the heuristic-guided tie-breaking mechanism prioritizes goal-directed actions when multiple actions have nearly equal <i>Q</i>-values. Moreover, the adoption of DDQN over standard DQN solves the issue of overestimation bias. Experimental results in cluttered 3D grid environments show that our method outperforms baseline DQN and DDQN in terms of convergence speed, success rate, and path efficiency. These findings demonstrate the effectiveness of our approach in enabling more reliable and efficient autonomous navigation for UAVs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A modified double DQN for UAV path planning: dynamic reward shaping and heuristic-guided tie-breaking

  • 期刊论文

  • 发表日期：

    2026年5月12日

查看全文
  • Ghulam Farid,
  • Lanyong Zhang,
  • Ishaq Ahmed,
  • Muhammad Usman,
  • Asma Iqbal,
  • Talha Younas

摘要

Unmanned aerial vehicle (UAV) path planning in complex 3D environments poses significant challenges for reinforcement learning agents, particularly in efficiently navigating around obstacles while ensuring goal reachability. Traditional deep Q-network (DQN) approach often struggles with slow convergence, suboptimal exploration, overestimation bias, and poor generalization in complex 3D UAV navigation tasks. These limitations are primarily due to sparse-reward signals, redundant action selections, and the inability to effectively resolve action-value ties. Inherent overestimation issue of DQN leads to unstable policies and suboptimal path choices especially when navigating high-dimensional obstacle-cluttered UAV environments. To address these issues, we propose an improved double DQN (DDQN) framework for UAV path planning that incorporates two key enhancements: dynamic reward shaping and heuristic-guided tie-breaking in case of nearly equal actions. The reward shaping strategy provides a denser learning signal by assigning rewards based on the agent’s Euclidean distance to the target which encourages steady progress towards the goal. Additionally, the heuristic-guided tie-breaking mechanism prioritizes goal-directed actions when multiple actions have nearly equal Q-values. Moreover, the adoption of DDQN over standard DQN solves the issue of overestimation bias. Experimental results in cluttered 3D grid environments show that our method outperforms baseline DQN and DDQN in terms of convergence speed, success rate, and path efficiency. These findings demonstrate the effectiveness of our approach in enabling more reliable and efficient autonomous navigation for UAVs.