This paper investigates the robust optimal consensus control problem for a multi-agent system of unmanned surface vehicles (USVs) with linearized dynamics and norm-bounded model uncertainties. A policy iteration (PI) approach, rooted in reinforcement learning, is proposed to solve the robust optimal control problem. This method circumvents the need to directly solve the complex coupled algebraic Riccati equation (ARE) by iteratively solving a sequence of linear Lyapunov equations. The proposed control protocol is fully distributed, relying only on local information exchange between neighboring agents. Sufficient conditions for achieving robust optimal consensus are rigorously established through Lyapunov stability analysis. The convergence of the PI algorithm to the optimal solution is guaranteed. Finally, numerical simulations with a group of three USVs are presented to demonstrate the effectiveness and performance of the proposed approach.