Comparative Analysis of RL-Based Algorithms for Complex Systems Control

The increasing complexity and nonlinearity of modern control systems pose serious challenges to achieving effective control. While differential equations have traditionally been the foundation of control theory and optimization with constraints, they may not be sufficient to accurately model chaotic and unbalanced systems. In response to these challenges, newer approaches gaining prominence include the use of policy iteration and Reinforcement Learning (RL). These techniques revolve around the concepts of sequences of actions and rewards, providing a dynamic and adaptable framework for controllers. Learning with reinforcement offers a promising avenue for addressing control theory, enabling systems to robustly adapt to dynamic environments. This paradigm shift away from traditional approaches such as linear–quadratic regulator (LQR) controllers to a more robust reward-based control mechanism is particularly noteworthy. Through a comprehensive analysis of RL-based algorithms, including Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC) and Trust Region Policy Optimization (TRPO), we examine their suitability and performance in crane control scenarios. This study aims to shed light on the advantages and limitations of each algorithm in optimizing crane operations, offering valuable insights for future applications of RL in complex control systems.