Reinforcement Learning Based Optimal Fault Tolerant Formation Control of Multi-UAVs: Theory and Experiment
摘要
This paper focuses on fault-tolerant formation control of multi-UAV systems subject to actuator faults and external disturbances. An integral sliding mode control (SMC) strategy is developed, treating both actuator faults and external disturbances as lumped uncertainties. To counteract these uncertainties, an adaptive compensation mechanism based on neural networks is integrated into the SMC framework, ensuring robust tracking performance even when individual UAVs experience faults. To reduce the control complexity caused by state coupling in multi-agent systems, the formation tracking problem is reformulated as an optimal control problem. A global performance index is constructed using nominal error dynamics to enable continuous performance enhancement. Furthermore, a reinforcement learning-based controller with an actor-critic architecture is proposed to solve the Hamilton–Jacobi–Bellman equation. In contrast to conventional methods, the proposed approach does not require persistent excitation, while still achieving accurate parameter estimation and efficient training. Lyapunov-based analysis ensures the stability of the closed-loop system. Real-time experiments validate the effectiveness and robustness of the proposed method in achieving reliable and optimal formation control under uncertain conditions.