Predictive Trajectory Planning Using Hierarchical Reinforcement Learning for Lane Change
摘要
Trajectory planning for lane change in highly dynamic and uncertain conditions is a crucial challenge for automated driving. Traditional planning algorithms based on manually-designed constraints can hardly cover all possible situations. Deep Reinforcement Learning (DRL) is a promising solution because of its adaption to diverse scenarios. However, to utilize temporal information in planning, most DRL methods largely increases the complexity of state space and network structure. This results in lower output frequency and generalizability. To address this problem, we propose a Hierarchical RL framework using Deep Deterministic Policy Gradient (DDPG). The high-level network selects the best target position and velocity for the ego vehicle. A novel expression of predicted trajectories in state space and rewards is designed to keep the network light-weighted. The low-level network calculates ego vehicle controls to follow the target while maintaining safe distance based on only current states. A multi-stage training methodology is proposed to improve the network performance. Experiments on overtaking scenarios show that our framework outperforms both classic DRL approaches and another HRL method in travel efficiency without loss of safety.