This study presents a comprehensive investigation into the optimal control strategy formulation for orbital pursuit–evasion games (OPEG) with relative state constraints. A state-constrained critic-only online approximator (SCOA) approach is proposed to achieve pursuit using nonlinear relative dynamics and considering \(J_2\) perturbations. The proposed methodology enables the derivation of an optimal control strategy that guarantees a saddle-point equilibrium solution for the OPEG. The relative state constraints are handled by introducing a barrier function, and asymmetric bounds are considered as well. Furthermore, the optimal strategy is derived by utilizing a critic-only neural network to approximate the Hamilton–Jacobi–Isaacs (HJI) equation solution, circumventing direct computation, thereby streamlining network design and minimizing computational expense. Besides, the stability of the closed-loop system and the uniform ultimate boundedness of the weight estimation errors of the critic network are demonstrated using the Lyapunov method. Finally, simulation results show that the proposed method can maintain the relative states within predefined bounds and successfully obtain the saddle-point solution.
