An Automatic Annotation Method for Rail Defects Based on Clustering Algorithm and Large Model Fine-Tuning

Rail defects pose critical threats to transportation safety, yet manual annotation remains inefficient and costly, particularly for imbalanced datasets. With the advancement of computer vision, automated defect annotation becomes indispensable. This paper addresses rail defect annotation by proposing a two-stage framework: adaptive K-means clustering and LoRA-fine-tuned Grounding DINO. In the first stage, K-means clustering processes a large number of rail images to screen high-purity defect candidates, achieving 97.3% defect purity and reducing manual labeling workload by 95%. In the second stage, LoRA—a lightweight fine-tuning technique—is applied to train Grounding DINO on the screened defects, aligning visual features with defect semantics via class-specific text prompts. Experimental results on rail defect dataset show the framework achieves 82% mean IoU and 94.2% class accuracy compared to manual annotations, while ensuring effective training with low computational costs. This work provides a practical solution for automated rail defect annotation, balancing efficiency and accuracy to support intelligent railway maintenance systems.