Alleviating Local Optima and Enhancing Path Planning: A Deep Reinforcement Learning Approach for Autonomous Exploration

In recent years, there have been significant breakthroughs in the development of navigation technologies. Reinforcement learning’s ability to enable autonomous learning for unmanned systems in complex environments, empowering intelligent agents with autonomous decision-making and planning capabilities, has undoubtedly made reinforcement learning navigation an innovation approach that garners significant attention. In this paper, we propose an autonomous navigation method based on Deep Reinforcement Learning (DRL), which enables an agent to autonomously explore the unknown environment and navigate to a target point. When exploring unknown environment, DRL models are prone to get trapped in local optima. To address this, we improve the DRL model by incorporating a cyclic optimum detection module, which allows the model to alleviate the local optima and resume the exploration process quickly. Furthermore, we designed a Shortest Path Reward Function, which encourages the agent to explore the environment with the objective of finding the shortest path. This reward function ensures that the agent’s actions are guided by the principle of minimizing the accumulated traveling distance. Experimental results demonstrate that our navigation method achieves a success rate approximately 15 \(\%\) higher than the Goal-Driven Autonomous Exploration (GDAE) method, and the distance of the agents’ exploration during navigation is also shorter.