This paper investigates the cooperative learning control under full state constraints for multi-agent systems. By introducing a nonlinear mapping (NM), the constrained tracking errors are transformed into equivalent unconstrained ones. Then, using the Lyapunov method, the boundedness of all the signals is proved and the full state constraints are never violated. Further, we prove that all estimated weights of neural networks (NNs) can converge to their optimal values with small errors, which means the NNs can accurately approximate unknown continuous functions. Thus, the cooperative learning is achieved. Simulation results verify the effectiveness of the proposed control scheme.