This paper introduces HisDoc-DETR, a novel set prediction-based approach for historical document layout analysis. The method specifically addresses the unique challenges of analyzing historical Chinese documents, particularly their sparse foreground characteristics and complex layout structures. HisDoc-DETR incorporates three key modules: (1) a semantic relationship learning module based on Transformer architecture that effectively captures long-range dependencies in document layouts, (2) a dual-stream feature fusion module that intelligently combines high-level semantic features with low-level details through channel and spatial coordinate attention mechanisms, and (3) a GIoU-aware prediction head that enhances layout localization accuracy by linking classification confidence with localization quality. Extensive experiments on the SCUT-CAB dataset demonstrate that HisDoc-DETR significantly outperforms existing methods, achieving improvements of 4.1% and 3.2% in AP metrics for logical and physical layout analysis respectively compared to the DINO baseline. The method shows particular strength in handling complex historical document layouts, achieving state-of-the-art performance across most layout categories.