End-to-End Urban Autonomous Navigation with Decision Hindsight

Urban autonomous navigation has broad application prospects. Reinforcement Learning (RL) based navigation models can be continuously optimized through self-exploration, eliminating the need for human heuristics. However, training effective navigation models faces challenges due to the dynamic nature of urban traffic conditions and the exploration-exploitation dilemma in RL. Moreover, the limited vehicle perception and traffic uncertainty introduce potential safety hazards, hampering the real-world application of RL-based navigation models. In this paper, we proposed a novel end-to-end urban navigation framework with decision hindsight. By formulating the problem of Partially Observable Markov Decision Process (POMDP), we employ a causal Transformer-based autoregressive modeling approach to process the historical navigation information as supplementary observations. We then combine these historical observations with current perceptions to construct a history-feedforward state representation that enhances global awareness, improving data availability and decision predictability. Furthermore, by integrating the historical-feedforward state encoding upstream, we develop an end-to-end learning framework based on RL to obtain a navigation model with decision hindsight, enabling more reliable navigation. To validate the effectiveness of our proposed method, we conduct experiments on challenging urban navigation tasks using the CARLA simulator. The results demonstrate that our method achieves higher learning efficiency and improved driving performance, taking priority over prior methods on urban navigation benchmarks.