<p>The quality of road defect images captured on rainy days is often degraded, making defect recognition a significant challenge. Detection models trained under normal weather conditions typically perform poorly in rainy conditions. This paper introduces a road defect detection model, YOLO-MSDD, specifically tailored for rainy-day scenarios to tackle the identified issue. Employing the advanced rain-removal model NeRD-Rain, synthetic rain streaks are eliminated from images. Additionally, a real rainy-day road defect dataset has been developed. The proposed YOLO-MSDD model utilizes the MobileViT network as its backbone, enabling it to learn both local features and global representations comprehensively. Strip Pooling has been integrated to concentrate on elongated defects and eliminate irrelevant information. Furthermore, the deformable large kernel attention module is improved, resulting in a new C2f_DLKA attention module, which enhances the fusion of multi-scale road defect features. Dynamic Convolution replaces standard convolution to adjust convolution kernels based on different defect types dynamically. Experimental results show that the YOLO-MSDD algorithm achieves significantly higher accuracy on the De-raining Road Damage Detection dataset, the real rainy-day road defects dataset, and the Crack-Forest dataset, with mAP50 improvements of 4.8%, 11.1%, and 3.1%, respectively, compared to the YOLOv8n model. Additionally, mAP50-90 improved by 4.1%, 7.3%, and 2%, respectively. The YOLO-MSDD model also outperforms other state-of-the-art object detection algorithms, demonstrating its high applicability and reliability for road defect identification in rainy conditions.</p>