Traditional map-based navigation requires time-consuming mapping, while map-free methods often involve inefficient exploration. Both are ill-suited for time-critical scenarios such as emergency rescue. The readily available structural semantic map (eg, an evacuation map) is inherently well-suited for such scenarios, as it provides crucial geometric and semantic cues to support efficient navigation. However, applying the map to robot navigation tasks remains challenging due to the discrepancy between the map and the real environment, caused by the lack of metric information and inherent geometric distortions. To address these challenges, we propose ENAV, a unified framework that integrates room topology extraction, topology-based localization through alignment, and vision–language model (VLM)-guided planning to enable efficient navigation using evacuation maps. Specifically, given a target room, ENAV first extracts room topology from both the evacuation map and the real-time constructed metric map, and performs localization via topology alignment. It then employs a vision–language model (VLM) to generate intermediate sub-goals, and finally plans low-level actions to reach each sub-goal incrementally. Extensive experiments on our curated dataset demonstrate that our algorithm outperforms other baselines by a large margin in terms of SR and SPL metrics, highlighting the effectiveness and efficiency of the proposed framework.