Robust Navigation for Unmanned Surface Vehicle Utilizing Improved Distributional Soft Actor-Critic

Navigating unmanned surface vehicles (USVs) efficiently and robustly in the presence of obstacles and ocean current interference in marine environments is highly challenging. To achieve robust navigation without environment maps and prior information, we follow the three effective improvements of the distributed algorithm distributional soft actor-critic with three refinements (DSACT) over distributional soft actor-critic (DSAC): expected value substitution, double value distribution learning, and variance-based critic gradient adjustment. In order to further optimize the learning rate of DSACT, we optimize DSACT through loss-adjusted prioritized experience replay (LAP) and propose a local path planner called LAP-DSACT for USV navigation. In order to offset the disturbance of ocean currents and plan a smooth and safe trajectory, the motion compensation is considered in the USV motion model. The experimental results clearly demonstrate that LAP-DSACT algorithm outperforms the comparison algorithms in terms of task time, energy efficiency, and the quality of path.