Improved Priority-Based Hindsight Experience Replay in Reinforcement Learning

Reinforcement Learning (RL) algorithms, crucial for robotic tasks in sparse reward settings, often require extensive interactions with the environment to learn effectively. Hindsight Experience Replay (HER) addresses this challenge by introducing virtual goals, thereby enhancing sample efficiency by learning from transitions in which the original goal was not achieved. However, randomly sampled additional goals in HER may hinder learning due to their irrelevance. To mitigate this, we propose the improved Priority-based Hindsight experience replay method to prioritize additional goals, improving learning speed and robustness across various robotic tasks. Our improved Priority-based Hindsight Experience Replay method samples with higher probability on the trajectories with temporal difference error and reducible loss, prioritizing those with greater learning potential. Through empirical validation, strategies taken by improved Priority-based Hindsight experience replay method have been demonstrated to enhance training performance and sample efficiency, offering promising avenues for advancing RL in robotic control tasks.