Efficient and stable deep reinforcement learning: selective priority timing entropy

Deep reinforcement learning (DRL) has made significant strides in addressing tasks with high-dimensional continuous action spaces. However, the field still faces the challenges of low sample utilization and insufficient exploration-exploitation balance, limiting the generalizability of algorithms across different environments. To effectively improve sample utilization, optimize the exploration-exploitation balance, and achieve higher rewards in tasks, this paper designs the selective priority timing entropy (SPTE) algorithm. Subsequently, selective prioritized experience replay (SPER) is proposed, which employs frequent replay of multiframe memories to enhance sample utilization and improve the stability of policy updates. Additionally, the temporal advantage with decay (TAD) method introduces a decay factor to help adjust the weights of the variance and bias, thereby reducing estimation errors. The reward mechanism is augmented with multientropy (ME) for entropy-regularized training, achieving a balance between information exploration and exploitation. Finally, experimental testing on the challenging Arcade platform demonstrated that the SPTE algorithm surpasses the average testing level of human players by 104.936%. Furthermore, compared to other algorithms, SPTE achieves an average score increase of over 32.75%, and it consistently outperforms the compared methods in more than 60% of tasks, indicating its strong adaptability and robustness.

The implementation process of SPTE