<p>Real-time and accurate semantic segmentation is essential for autonomous rail transit systems, yet it remains challenging due to the need to preserve the continuity of thin rail structures while maintaining reliable recognition of surrounding safety-critical objects. Existing real-time segmentation models, which are largely developed for generic road scenes, often produce fragmented rail predictions and unstable boundaries in complex railway environments. To address this issue, we propose RailSegNet, a real-time semantic segmentation framework tailored for rail scenes. Built upon the efficient Deep Dual-Resolution Network (DDRNet), the proposed framework adopts a dual-output decoding strategy during training, where context and spatial branches are jointly supervised to enhance both semantic consistency and structural detail preservation. In addition, we introduce two lightweight training-time regularization terms, namely cross-branch consistency regularization and morphology-aware rail regularization, to improve prediction coherence and reinforce rail continuity without increasing inference complexity. During inference, only the main context branch is retained, thereby preserving real-time efficiency. Extensive experiments on the challenging RailSem19 benchmark demonstrate that RailSegNet achieves a competitive accuracy–efficiency trade-off among the compared real-time baselines, while providing more structurally coherent rail predictions. The multi-seed evaluation shows that RailSegNet provides moderate but stable improvements over the DDRNet baseline, while rail-oriented structural metrics provide additional evidence of improved rail-region continuity and boundary quality.</p>