This paper studies the optimal formation tracking control problem for the position loop of multiple quadrotor unmanned aerial vehicles (multi-QUAVs) based on neural network and integral sliding mode method. Firstly, a neural network is applied to approximate uncertainties and external disturbances, and an integral sliding mode controller is designed to compensate for the impact of them on the system. Then, the robust optimal tracking control problem of original system is converted into the optimal control problem of a nominal system. An adaptive dynamic programming framework based on a single critic network is proposed to obtain the optimal cost function, and the optimal controller law is calculated based on the optimal cost function. The effectiveness of the proposed method is ultimately verified through numerical simulation.