Learning Soft Robotic Arm Control: A Data-Driven Approach with Forward Dynamics Transformer and Reinforcement Learning

Due to their nonlinear and intricate dynamics, developing analytic models and learning the control of soft robotic arms presents a significant challenge. Additionally, the challenge associated with developing analytical models for soft robotic arms is compounded by the often unpredictable variability of relevant mechanical properties inherent in these systems. Recent efforts in this domain have focused on exploring the potential of employing neural network-based, data-driven methods as a promising solution for controlling these manipulators. This paper introduces a comprehensive learning framework that seeks to acquire the control policy for a soft robotic arm through the application of reinforcement learning techniques. This framework proposes an innovative method for direct acquisition of the forward dynamics of a soft robotic arm, utilizing data collected directly from the soft arm itself. The forward dynamic model (dubbed DynaFormer) is meticulously crafted using a transformer-based architectural approach. To further advance the capabilities of this system, a reinforcement learning agent is subsequently trained using the twin-delayed deep deterministic policy gradient (TD3) algorithm. The purpose of this training is to enable the soft robotic arm to execute a specific task, namely, the precise reaching of a designated point.