Digital geographical maps of China are widely used across various fields, but many of these maps contain common errors, such as inaccurate borders and missing islands, which can severely impact national security and sovereignty. Therefore, we propose Geo-DETR, a novel approach for map accuracy assessment. Initially, to address the challenge of extracting intricate and subtle boundary information, we present the Pristine Gradient Extraction Module (PGEM), which enhances boundary detection through gradient-based features. Subsequently, the Gradual Attention Fusion Module (GAFM) and Dual Layer Attention (DLA) mechanism adopt a multi-scale, multi-path strategy to optimize the fusion of boundary and semantic features, reducing information loss. Additionally, we design a Cross-Scale Fusion Encoder (CSFE) that enhances the model’s ability to capture both high-level semantic representations and fine-grained details. Experimental results show that Geo-DETR significantly outperforms existing methods in map detection tasks, efficiently and accurately identifying map errors, even in resource-constrained environments.