Rail Surface Defect Detection and Severity Analysis Using CNNs on Camera and Axle Box Acceleration Data

Rail surface defect detection is a relevant problem in the field of data-driven railway maintenance. Artificial intelligence and neural networks (NN) for axle box acceleration (ABA) or camera data show great potential for defect detection and classification. However, a sufficient amount of labeled training data is required, all the more if the defect severity is to be estimated. A unique dataset of time-synchronized ABA and camera data is employed that contains labeled defect instances. For the image analysis, RetinaNet as a single-stage object detector (with the backbone of ResNet-50 and a feature pyramid network) is used to achieve high classification performance for the two most common rail surface defects (squat and corrugation). Additionally, a machine learning-based method on ABA data to estimate defect severity levels (low, medium, heavy) is proposed. False positives are detected in the original labels by both classifiers during evaluation. The inspection of the false positives in image data reveals that defects have been overlooked in the initial labeling. The insights of this work help to reduce the dependency on labeled data by using only a few labeled samples and by exploiting complementary data sources instead of increasing the number of labeled instances.