Research on UUV Escape Method Based on Deep Reinforcement Learning

Unmanned underwater vehicle (UUV) has attracted more and more attention in the research of underwater attack and defense confrontation. In order to make UUV get rid of the pursuit of adversary, ensure it safety and complete the confrontation task, this paper applies deep reinforcement learning to the study of UUV escape problem, and proposes a UUV escape method based on Twin Delayed Deep Deterministic policy gradient algorithm (TD3). Firstly, according to the application requirements, an environment model composed of one escaping UUV, multiple pursuing UUVs and obstacles is established. Then, it is stipulated that the escaping UUV can choose the strategy of using the interference of obstacles or its own maneuver to escape by comparing the distance between it and the obstacle and the distance between it and the pursuing UUV. A strategy reward function is designed for the TD3 algorithm, and it is used to make decisions on the actions of the escaping UUV. Finally, the proposed method is trained and evaluated in the built environment, which proves that the method is convergent and controllable, and the UUV can achieve a high escape success rate. The simulation results show that the UUV escape method based on TD3 proposed in this paper can make the UUV use its own actions and the environment to successfully escape in underwater confrontation.