Multi Path Real-time Semantic Segmentation Network in Road Scenarios

Semantic segmentation is a critical task in computer vision. Existing methods often struggle to balance accuracy and computational efficiency when processing high-resolution images, limiting their application scenarios. To address these limitations, we introduce RepMPSeg, a novel re-parameterization-based multi-path real-time semantic segmentation network. RepMPSeg improves upon traditional dual-branch architectures. It features two independent yet interconnected branches: a high-resolution branch for detailed feature capture and a low-resolution branch for global semantic information extraction. By employing re-parameterization techniques, the basic convolutional blocks are optimized to enhance feature capture and local context information. During training, parallel convolution structures are utilized, which are then streamlined into a single kernel during inference to maintain performance while reducing computational complexity. The high-resolution branch leverages sub-pixel sampling and 1 \(\times \) $\times$ 1 convolutions to improve the receptive field and minimize computational load, while the low-resolution branch uses 4 \(\times \) $\times$ downsampling to enhance semantic information extraction. Features from both branches are fused through the re-parameterization-based Hybrid Downsampling Module (RepHDM), which aligns and combines features effectively. Our experiments on the Cityscapes and CamVid datasets demonstrate that RepMPSeg achieves a balance between speed and accuracy, outperforming state-of-the-art methods. Specifically, RepMPSeg achieves an mIoU of 77.4 and an FPS of 91.4 on the Cityscapes dataset, and an mIoU of 78.6 and an FPS of 117.33 on the CamVid dataset, making it a highly efficient solution for real-time semantic segmentation in road scenarios.