The intelligent navigation performance of unmanned surface vehicles (USVs) is recognized as a critical guarantee for the safety and efficiency of marine engineering. However, current USV navigation control still faces the following challenges: (1) The traditional artificial potential field (APF) method can hardly adapt to environments where randomly moving targets coexist with multiple static and dynamic obstacles. (2) USVs are significantly affected by external disturbances in the marine environment, resulting in insufficient robustness of conventional controllers. To address these issues, an intelligent navigation control method integrating an improved APF with RL is proposed in this paper. In the guidance layer, collision risk assessment, relative position and velocity information are introduced to resolve obstacle avoidance and tracking in complex scenarios. Besides, an actor-critic neural network (AC-NNs) is constructed to compensate for model uncertainties and external disturbances. Finally, a numerical simulation is conducted to verify the robustness of the proposed strategy.