This article proposes an automatic detection algorithm for multi-scale feature detection and comprehensive feature analysis by YOLOv8 networks based on squeeze excitation (SE) attention mechanism. Firstly, the image of the car carriage notch is collected and subjected to geometric rotation, color change and other image enhancement processing to simulate the changes in ambient light in actual scenes. Then, the YOLOv8 network based on SE attention mechanism is used to segment and locate the notch area in the image. The edge detection algorithm in OpenCV is used to obtain the contour line of the notch, and relative features are extracted to eliminate scene differences. Subsequently, Spearman correlation analysis combined with feature selection method was used to select 8 obvious feature values and input them into various machine learning models such as support vector machines, B-P neural networks, and random forests to achieve automatic detection of notch quality. The research results indicate that the support vector machine has the highest model goodness of 98.7%. The proposed algorithm is not sensitive to ambient light and has low requirements for image quality. It does not require a dedicated monitor setups to eliminate the influence of scene differences. It can achieve rapid non-destructive testing of laser cutting quality in railway carriages by simple detection equipment under complex environment.