Sequential Masking Imitation Learning for Handling Causal Confusion in Autonomous Driving

Training agents for autonomous driving using imitation learning seems like a promising way since its only requirement is the demonstration from expert drivers. However, causal confusion is a problem existing in imitation learning, which is that with more features offered, an agent may perform even worse. Here, we aim to augment agents’ imitation ability in driving scenarios under sequential setting, by a novel method we proposed: Sequential Masking Imitation Learning(SEMI). First, we train a Vector Quantised-Variational AutoEncoder(VQ-VAE) to encode a sequence of images into a latent representation with discrete codes. After that we deploy several masks on the encoded images, the masks here will randomly hide some semantic objects in the encoded images. Finally, we design the behavior clone network as a predictor of expert action, using an encoded and masked image sequence as input, encouraging the network to make expert-like predictions when some partition of information about the environment is missing. The masking procedure in SEMI helps the imitator identify the contribution of each encoded feature to the expert’s prediction. We demonstrate that this method could alleviate causal confusion in driving simulation by deploying it to the CARLA simulator, and compared it with other methods. Experimental results show that SEMI can effectively reduce confusion in autonomous driving. The agent trained with SEMI method reduces the collision rate by 45 \( \% \) compared to methods without masking procedure, and obtain the highest average survival timesteps among competing methods.