Time-varying formation optimization tracking of multi-agent systems with semi-Markov switching topology

This paper investigates the problem of achieving time-varying formation (TVF) optimal tracking control for second-order nonlinear multi-agent systems (MASs) with switching topology. The main objective is to enable the follower agent to track the leader while achieving a given TVF. Considering that MASs often operate in complex environments and encounter diverse changes, this paper introduces a semi-Markov switching strategy. The purpose is to enable the system to achieve the desired time-varying formation even under anomalous conditions. Furthermore, when it comes to Hamilton–Jacobi–Bellman (HJB) optimization, directly dealing with unknown equations becomes a difficult task. However, this challenge can be effectively addressed through the implementation of an actor-critic structural network. In existing approaches, the neural network parameters are updated in a more intricate manner by employing a gradient descent algorithm on the square of the approximate HJB equation, also known as the Bellman residuals. In the optimization scheme proposed in this paper, the neural network parameters are updated using a concise method derived from the negative gradient of a simple positive function. This approach offers a more streamlined alternative compared to existing update methods. By employing this method, an optimal control scheme is provided to tackle the TVF control problem with switching topology. Finally, the validity of the theoretical approach is substantiated through the utilization of Lyapunov stability theory and numerical simulation, thereby demonstrating its effectiveness in the field of MASs optimization.