With the rapid development of ocean utilization and the rise of unmanned surface vehicle (USV) cluster applications, the demand for dealing with complex sea condition is increasing. Multi-USV cooperative control strategy has become an important technology to improve operational efficiency. This paper aims to propose a multi-agent path planning algorithm combing Model Predictive Control (MPC) and Deep Q Network (DQN) to achieve cooperative control of multiple USVs in the marine environment. Firstly, based on the simple_adversary environment of MPE, this study verifies the effectiveness of the combination of MPC and DQN algorithms and realizes the trajectory planning of multiple USVs. Then, through comparative experiments, this paper analyzes the performance of the proposed strategy in the simulated confrontation environments and proves its superiority under dynamic and uncertain conditions. Finally, considering the particularity and challenges of the marine environment, this paper further discusses the feasibility of the application of the strategy in the actual marine environment. The simulation results show that the strategy combined with MPC and DQN effectively realizes the multi-agent path planning and show its application potential in the USV cluster control scenario. Further research is needed to explore the scalability and robustness in the real environment. This exploration will help to develop a more superior cluster control system to better cope with the uncertain marine environment.