In the process of industrial production, some defects will appear on the surface of metal products, which will affect the performance and life of the products, and even affect the safety and stability of equipment operation. A metal surface defect detection algorithm based on improved YOLOv7 is proposed to improve the detection accuracy of defects of different shapes and sizes, which is suitable for images collected under different working conditions. Firstly, in view of the large difference in the size of metal surface defects, the NWD is added on the basis of the CIOU loss function to improve the network's adaptability to surface defects of different sizes. Secondly, SPDConv module is added before the 20 × 20 prediction head to solve the problem of difficulty in detecting small defects and improves detection accuracy. Finally, the polarized self-attention module is integrated into the feature fusion network to strengthen the focus and utilization of key points in diverse features and improve the network's feature extraction and generalization capabilities. A dataset of various metal surface defect images is established with the images of metal parts collected on a real motor production line, combined with public datasets, to train the network for metal surface defect detection under different materials and environments. The experimental results show that the mAP (Mean Average Precision) of the proposed method is 88.3%, and the number of floating-point operations is 103.3 GFLOPs, which has better detection accuracy and efficiency than the original YOLOv7 algorithm and other mainstream algorithms.