Effects of Increased Entropy on Robustness of Reinforcement Learning for Robot Box-Pushing

Deterministic reinforcement learning (RL) methods, which produce a single solution for a given system state, have shown impressive results in simulated environments with well-defined parameters. However, their performance often declines when applied to real-world systems, where parameters fluctuate. In contrast, probabilistic RL approaches, capable of generating a spectrum of potential solutions, demonstrate enhanced robustness to variations in parameters. Our study evaluates the performance of probabilistic RL in a robotic task of manipulating a box on a table. We investigate the robustness of policies trained with varying degrees of entropy by training them with one set of box mass and friction parameters and then testing their performance in a simulated environment with altered parameters. Our findings reveal that policies with lower entropy perform best in stable conditions, while those with higher entropy become more effective as parameter variability increases. Given that applying learned policies to real robots requires further transfer learning (TL), where the efficiency heavily depends on the quality of the initial policy, selecting the optimal entropy level for initial training could significantly enhance the TL process.