Bridging DRL and LLM: Explainable and Efficient Autonomous Vehicle Control for Off-Ramp in Highway

Highway ramps are one of the most challenging scenarios in the field of autonomous driving technology. In this study, we proposed an integrated autonomous vehicle control framework with deep reinforcement learning (DRL) and large language model (LLM) to realize the autonomous vehicle off-ramp task in the highway scenario. The framework is mainly composed of three parts: (1) the Transformer module processes multi-dimensional state information in the driving environment under multi-temporal sequences; (2) the agent (Hybrid Soft Actor-Critic, Hybrid SAC) receives the information delivered by the Transformer module to make a decision; (3) the LLM evaluates the agent’s decision based on the multi-modal information in the simulation environment and transmits the evaluation results to the agent to improve its learning efficiency. Finally, we use SUMO and Carla to carry out joint simulation experiments of auto-driving vehicle converging off-ramp, which verifies the proposed method can significantly improve the efficiency and safety of the autonomous vehicles.