Defect detection aims to find and identify defects in various situations. It is easily impacted by lighting, shooting equipment, and other environmental elements in a complicated production setting, which leads to low detection efficiency and inadequate precision. This research proposes a dual-stripe window transform pyramid-based hot-rolled steel surface flaw detection network (DSWPN). To improve the ability to recognize defect features, a multi-level feature extraction (MFE) unit is specifically designed to combine the feature extraction network of CSPDarkNet53 and DCSwin-Transformer blocks. Additionally, an optimized fused dual feature pyramid network (BiFPN) is proposed, which optimizes the multi-scale feature representation through weighted feature fusion and efficient cross-scale connections. The attention technique aims to increase the model’s sensitivity to fine-grained information. Finally, a multi-head context integration module (MCI) is introduced to increase the precision of defect identification. Results from experiments on the NEU-DET and GC10-DET open-access databases prove that the suggested algorithm performs better than current approaches across a number of performance metrics, proving its practical application potential in steel defect detection.