Autonomous vehicles rely on various sensors for environment perception. The placement of the sensors turns out to be critical, as their position determines which environmental information can be perceived. In this paper, we investigate the influence of the position of three front-oriented RGB cameras and a LiDAR sensor on the end-to-end autonomous driving performance. Furthermore, we explore the effects of a mismatch in positioning during training and test (i.e., vehicle operation). In total, four sensor configurations are investigated. We employ the CARLA simulator and the recently published TransFuser architecture for end-to-end autonomous driving. To ensure comparability between runs despite CARLA’s non-deterministic traffic manager, we collect the sensor data for all configurations in a single simulation run. We discover that sensor positions close to and above the rear mirror excel both the roof center and the very high (impractical) baseline position w.r.t. the overall driving score. A sensor position mismatch between training and testing leads to a drop in all performance metrics. However, multi-condition models trained on a mix of sensor positions significantly regain performance regarding the infraction score, thereby improving the model’s robustness against domain shifts caused by sensor mismatches during training and test.