YOLO-DCR: Vehicle Instance Segmentation Algorithm in Complex Scenes Based on Improved YOLOv8-Seg

This paper introduces YOLO-DCR, a vehicle instance segmentation method designed to tackle the challenges of missed multi-scale target detection, dense occlusion, and substantial background interference inherent in urban vehicle instance segmentation tasks. The proposed method augments YOLOv8n-seg, a state-of-the-art framework. A dynamic up-sampling module (DySample) is introduced, employing adaptive weight interpolation to optimize feature fusion and enhance performance. Additionally, a channel-space attention mechanism module (CBAM) is integrated at the Neck layer's terminus to augment feature representation within critical regions and mitigate background interference. The segmentation head is further optimized through integration of receptive-field attention convolution (RFAConv). This module enhances detail capture for occluded targets via dynamic weight assignment. Experimental results indicate that YOLO-DCR surpasses the baseline model, elevating mAPBox50 by 1.2% and 1.0%, and mAPMask50-95 by 1.8% and 1.1%, respectively, on the COCO vehicle subset. These improvements validate the model’s efficacy in optimizing multi-scale target detection, occlusion robustness, and background interference resistance.