<p>Automated quality control systems play an important role in the industrial field and widely use deep-learning-based defect inspection methods. Defect severity serves as an interface between automated industrial quality control systems and users to meet the requirements of dynamic quality inspection standards. However, most of the methods cannot evaluate the severity of the defect, which may lead to missing inspection or excessive inspection. To address the challenge, we proposed a two-stage defect detection and severity prediction network (DDSPNet). Firstly, we presented a gate-based global feature fusion block and a scale-aware local feature fusion block to solve the problem of widely varying sizes of defect regions. The method pays more attention to defective regions during global feature fusion and utilizes coefficients regressed from the feature maps of each defect to fuse multi-scale local features. Secondly, we use soft label to model the prior of the ordinal relationship and subjectivity of the defect severity in the dataset, and employ improved focal loss (soft focal loss) to train the proposed network. To validate the effectiveness of our method, we build the scratches on the surface of auto parts (SSAP) dataset. Experiments demonstrate that DDSPNet outperforms classification and object detection methods, achieving state-of-the-art performance on the SSAP dataset.</p>