Acceleration Dual Q-Learning Control for Nonlinear Discrete-Time Systems via Swarm Intelligence

This paper proposes an accelerated dual Q-learning control framework for nonlinear discrete-time systems subject to external disturbances. An acceleration factor \(\alpha >1\) is introduced in the Q-function update law, which substantially improves convergence speed over traditional Q-learning. The method solves a minimax optimal control problem formulated as a discounted performance index with \(L2\) -gain robustness, iteratively approximating the Hamilton-Jacobi-Bellman equation. We rigorously prove that the Q-value sequence is bounded, non-decreasing, and converges to the optimal solution when the discount factor \(\gamma \) satisfies \(\gamma \le \frac{1}{\alpha }\) . The implementation utilizes an actor-critic neural network structure: a critic network approximates the Q-function while action networks handle the optimal control and worst-case disturbance policies. Particle swarm optimization with adaptively scheduled parameters efficiently searches the compressed policy space. Simulation results on a helicopter attitude control model show markedly faster convergence and enhanced disturbance rejection capability compared to standard value iteration methods, confirming the theoretical predictions and practical utility of the proposed approach.