This paper presents a novel deep-learning pipeline to segment large railway datasets with minimal manual annotation, notoriously time consuming. The pipeline adapts DINOv2 [11] for labeling point clouds, with tailored self-distillation pre-training and fine-tuning. The adopted transformer architecture successfully generalizes to multiple railway datasets, with a lightweight pipeline that outperforms manual labeling speed by a factor of 6, despite requiring a final segmentation check and correction. This groundbreaking achievement bridges the gap between the need for annotated point clouds in railway industry and the lack of publicly available annotated datasets.