<p>In response to the challenges of detecting irregular defect patterns on rolled steel surfaces, which include significant irrelevant noise, low contrast, and defects that exhibit only slight features, existing defect detection algorithms struggle to identify various types of defects in a timely, comprehensive, and accurate manner. To tackle these issues, this paper introduces the ADE-YOLO (attention dilated early YOLO) algorithm, which integrates an attention mechanism with multi-scale dilated convolutional feature enhancement. Central to this approach is the novel adaptive attention enhanced convolution (AAEC) feature extraction module, which replaces the conventional C2f module. By employing fine-grained channel segmentation, an adaptive attention mechanism, and a residual mechanism, the AAEC module substantially improves the detection of small-scale defects. Furthermore, the integration of the dilated convolution feature fusion module (DC-FFM) into the YOLOv8 backbone enables the fusion of feature maps from various dilated convolutions across multiple pathways, augmenting the model’s capability in multi-target and multi-scale defect detection. Additionally, the ADE-YOLO algorithm incorporates the early fusion feature pyramid network (EF-FPN), leveraging early intervention and cross-scale connectivity strategies to capture more nuanced, shallow defect features. Experimental analysis was conducted using the NEU-DET and GC10-DET datasets. The ADE-YOLO model achieved mAP of 81.5% on the NEU-DET dataset and 68.2% on the GC10-DET dataset. Additionally, the detection speed reached 120 FPS and 277 FPS on the respective datasets. These results underscore the proposed algorithm’s capacity to balance high detection accuracy with real-time performance, satisfying the comprehensive demands of rolled steel surface defect detection tasks.</p>