Lane Change Decision Control of Autonomous Vehicle Based on A3C Algorithm

Coordinating the lateral and longitudinal control of vehicles during lane changing, while considering the vehicle’s operational state and the surrounding environment, poses a highly challenging task. In recent years, deep reinforcement learning (DRL) technology has experienced rapid development and has found widespread applications in the traditional automatic control industry. With the improvement of driving safety requirements, DRL technology provides a new research direction and an effective way for the development of vehicle autonomous lane change. This paper investigates automated decision control for lane changing in autonomous vehicles using the Asynchronous Advantage Actor-Critic (A3C) algorithm, while also proposing reasonable multi-objective performance evaluation metrics. Nevertheless, the traditional A3C algorithm frequently encounters convergence oscillation or degradation issues, hindering agents from attaining the highest reward. To address the mentioned issues, an improved parameter updating method based on a weighted average of advantage value is proposed. The simulation results on the highway simulation platform demonstrate that the enhanced A3C algorithm offers increased stability in comparison to the traditional A3C algorithm. Moreover, in comparison to the Deep Q-Network (DQN) algorithm and the Deep Deterministic Policy Gradient (DDPG) algorithm, the enhanced A3C algorithm showcases faster convergence speed and a higher success rate, hence confirming the superiority of the proposed improvement.