Optimal control of unknown nonlinear systems via online parameter estimation with interval excitation

This paper proposes an identifier-critic (IC) framework for optimal control of nonlinear systems with completely unknown dynamics. Unlike existing indirect adaptive/approximate dynamic programming algorithms, the developed IC framework consists of two adaptive neural networks (NNs): identifier and critic networks. The identifier networks are established to estimate the unknown dynamics, while the critic networks are applied to formulate the optimal control for an affine nonlinear system. To facilitate easy implementation and online computation, a linear regression method is introduced to establish the parameter estimation problem for the unknown weights in the NNs. Moreover, an interval excitation assumption, which is weaker than the persistence of excitation condition, is developed to ensure the convergence of the unknown weights in the IC framework within a finite time. The convergence of the weight estimation and the stability of the closed-loop system are theoretically demonstrated via Lyapunov theory. Finally, numerical simulation results are presented to demonstrate the effectiveness of the proposed control method.