Impact of Control Frequency on Deep RL-Based Torque Controller for Bipedal Locomotion

In the realm of traditional humanoid torque control, a prevalent practice involves meticulously tracking preplanned joint motions using high-frequency controllers to solve locomotion problems. However, recent studies have suggested the possibility of reducing excessive control frequencies of torque controllers through the utilization of deep reinforcement learning policies. This paper presents the impact of control frequency on torque-based deep reinforcement learning controllers, ranging from 250 Hz to 40 Hz. The study also outlines the method used to train the torque-based deep RL policy in various control frequencies while isolating the effect of frequency changes to ensure fair performance comparisons. The lower the control frequency, the more robust the results were against robot system delays, even on unexpected terrain or at higher target walking speeds within the training range.