<p>Low visibility and image degradation caused by adverse weather seriously threaten the operational safety of high-speed railways. This poses core challenges for existing visual detection methods in complex degradation scenarios, including missed detection of small objects, blurred feature extraction, and insufficient task collaboration. To address these issues, this paper proposes an end-to-end recognition framework that integrates real-time semantic segmentation and object detection. The framework first designs a coordinate attention-guided lightweight segmentation network (LR-ASPP), which achieves precise localization of high-risk areas in low-contrast environments by enhancing the perception of long-range spatial structures of the railway tracks. Subsequently, a Mamba-YOLO detection model is constructed. Its integrated CPA-Enhancer module adaptively restores degraded image details, while the introduction of the State Space Model (SSM) significantly enhances the model’s ability to capture global contextual information and small object features. Experiments on a self-constructed multi-weather dataset show that the proposed method achieves a mean Average Precision (mAP) of 86.6% in foreign object intrusion detection tasks. The detection precision for small and medium-sized objects is improved by 7.9% and 8.2%, respectively, compared to the baseline model, and it achieves a real-time inference speed of 103 FPS on edge devices. This research not only provides a new perspective for collaborative perception under adverse weather conditions but also lays a solid engineering foundation for realizing all-weather, highly reliable railway safety monitoring systems.</p>