Automated pavement defect detection is crucial for maintaining traffic safety. However, existing methods are prone to both missed detections and false positives when addressing defects that are distant and vary significantly in size. Additionally, current pavement defect detection models are large and parameter-heavy, limiting their suitability for resource-constrained edge deployment. The paper proposes a road defect detection model, named YOLO-MOB-C3, which bases on heavily parameterized structure and residual module. Firstly, a heavily parameterized backbone network based on MobileOne is designed for efficient extraction of small target pavement defect features. This strategy reduces the model size and memory usage without compromising detection accuracy. Secondly, a lightweight C3 module is introduced to construct a neck network. This design enhances the extraction of pavement defect features with large variations in shape and scale, and further reducing the model size and parameter count. Finally, the WIoU loss function improves the model’s generalization and accelerates convergence, which ensures that the model meets real-time requirements. Experiments were performed on the RDD2020 dataset to assess the performance of the proposed model. The results reveal that, compared to YOLOv7, the proposed model achieves a 2.6% increase in mAP@0.5, a 1.4% improvement in detection accuracy, and a 10.42% reduction in model size. This demonstrates the model’s ability to significantly reduce size while maintaining high accuracy.