SeMA-UNet: A Semi-Supervised Learning with Multimodal Approach of UNet for Effective Segmentation of Key Components in Railway Images

Railway system maintenance, crucial for ensuring safety and efficiency, faces challenges in effectively managing its vital components, including tracks, railroad ties, and fasteners. While various methodologies target these components, the limited availability of diverse railway datasets presents a significant hurdle. Addressing this, we introduce SeMA-UNet, a pioneering deep learning model designed to optimize performance in data-constrained scenarios. Seamlessly integrating semi-supervised learning with multimodal strategies, SeMA-UNet excels in preprocessing railway images, conducting comprehensive feature extraction to generate rich multimodal data. This process is further augmented by advanced techniques, notably the Monte Carlo simulation. Empirical results underscore SeMA-UNet’s robustness, with metrics such as an IoU of 0.9464, an AUC of 0.9796, and an mAP of 0.9468. Beyond its primary function of accurately identifying maintenance-critical regions, the model’s capabilities extend to advanced anomaly detection, heralding a new era in enhancing the reliability and safety of railway systems.