The performance of bridge structure has always been concerned, and in the whole bridge, the bridge span component plays a crucial role. With the support of machine learning techniques, this paper builds a huge data set for training and prediction. This paper proposes to use machine learning algorithm to design a Bayesian classifier, and carry out steps such as feature extraction and parameter recognition in MATLAB environment to improve the accurate identification accuracy of box defects in the study area. At the same time, this paper uses machine learning algorithms to reconstruct bridge structural damage images, and uses simulation software to test the performance of the tester to verify its feasibility and accuracy. The results show that the classification error rate of cracks is 12%, that of corrosion is 15%, that of deformation is 18% and that of spalling is 14%. Cracks and spalling are two types of damage with relatively low classification error rates, while corrosion and deformation are higher.