Sim2Real Autonomous Driving Using Convolutional Neural Network for Urban Environments

We introduce an autonomous driving framework that employs convolutional neural networks. This framework utilizes forward-facing stereo camera images, vehicle speed, traffic light status, and higher-level navigation commands to predict future waypoints for the vehicle’s trajectory. The model was trained on a dataset collected from the CARLA Simulator and underwent testing in both simulation and real-world settings without any additional fine-tuning on real-world datasets. In simulation testing, the model successfully navigated previously unseen maps and weather conditions, covering a distance of 3000 m without encountering collisions or traffic light violations. Real-world testing on a differential drive vehicle demonstrated the model’s ability to navigate without lane invasions.