Traditional methods for detecting defects in railway vehicle wheelsets mostly rely on a single data source (vibration and visual images), which has the challenges of insufficient detection accuracy, low efficiency, and inability to cope with complex defects. This paper applies a new method for detecting defects in railway vehicle wheelsets that integrates multimodal data and intelligent algorithms. It combines multiple sensor data such as vision, vibration, and acoustics, and jointly analyzes multimodal data through convolutional neural networks and support vector machines. The labeled defect data set is used for supervised learning to improve the detection accuracy. This method not only improves the accuracy and robustness of defect detection but also has high real-time performance and can better adapt to the defect detection needs under different working conditions. The experimental results show that the detection method proposed in this paper has good results in detection accuracy, real-time performance and handling of small defects in complex scenes. The accuracy of multiple defect types is higher than 0.9, and the detection time does not exceed 180 ms. It provides a feasible solution for the digital health management of railway vehicle wheelsets.