Perception to Control: End-to-End Autonomous Driving Systems

End-to-end autonomous driving systems have garnered a lot of attention in recent years, and researchers have been exploring different ways to make them work. In this paper, we provide an overview of the field with a focus on the two main types of systems: those that use only RGB images and those that use a combination of multiple modalities. We review the literature in each area, highlighting the strengths and limitations of each approach. We also discuss the challenges of integrating these systems into a complete end-to-end autonomous driving pipeline, including issues related to perception, decision-making, and control. Lastly, we identify areas where more research is needed to make autonomous driving systems work better and be safer. Overall, this paper provides a comprehensive look at the current state-of-the-art in end-to-end autonomous driving, with a focus on the technical challenges and opportunities for future research.