Memory Constrained Adaptive Soft Actor Critic algorithm with stochastic policy for efficient robotic path planning
摘要
The issue with robotic path planning in dynamic environments also causes the problem to be a challenging issue, with nonlinear motion dynamics, uncertainty of the environment, and model inaccuracy being limiting. Under such circumstances, traditional control and model based approaches are frequently problematic, with regards to flexibility and real time decision making. Reinforcement Learning (RL) offers an alternative which is data-driven, which allows an agent to acquire navigation strategies during interaction with the environment. Soft Actor-Critic (SAC) is one of the RL approaches that perform well in continuous control problems due to its stability and exploration (based on entropy). Our proposed work discusses the developed modified SAC framework, which is also known as Memory-Constrained Adaptive Soft Actor-Critic (MCA-SAC) to improve learning efficiency and stability of path planning on robots. The algorithm incorporates adaptive priority of experience, dynamic reward shaping, and temporal action clustering into a memory constrained replay network. These adjustments enhance the use of samples, the speed of convergence, and the smoothness of policies when the obstacles are changed. The performance of the simulation shows that MCA-SAC converges quicker, nearly 10% more path optimality than baseline SAC Algorithms. Collision rate is reduced up to 2.3% ensuring safety concerns in dynamic environment. It has high average rewards and more smooth curves than baseline SAC and its new variations, which confirms its usefulness in efficient and adaptive robotic navigation.