The Deep Reinforcement Learning Based Motion Control of Helical Micro swimmers In Flow Rate Environment

Soft Magnetic Miniature Robots (SMMRs) hold promising prospects for biomedical applications due to their flexible size and mobility in confined environments. However, achieving precise control performance and high repeatability to navigate the robot to a target location in unstructured environments, especially under varying flow conditions, remains a challenge. In this study, inspired by the control requirements for drug delivery and release in dynamic biofluids, we propose a flow rates rejection control strategy based on a Deep Reinforcement Learning (DRL) framework to manipulate SMMRs for goal-reaching and hovering in fluidic tubes. To achieve this, we first fabricate an SMMR that can be operated by an external magnetic field to fulfill its intended functions. Subsequently, a simulator is constructed based on neural networks to establish the relationship between the applied magnetic field and the robot’s locomotion states. With minimal prior knowledge about the environment and dynamics, a Gated Recurrent Unit (GRU)-based DRL algorithm is formulated, considering the designed history state-action and estimated flow rates. Additionally, randomization techniques are applied during training to distill a general control policy for the physical SMMR. Results from numerical simulations and experiments demonstrate the robustness and efficacy of the proposed control framework. Finally, in-depth analysis and discussions indicate the potential of DRL for soft magnetic robots in biomedical applications.