The present paper addresses multi-scale target imbalance, background noise interference, and edge deployment needs in pavement defect detection within complex road scenarios. The following proposal is hereby submitted: Initially, the Cascaded Channel Group Attention (CCGA) module is employed to reconfigure the Cross-Stage Partial Spatial Attention (C2PSA) with cascading grouped attention, thereby enabling multi-branch feature decoupling. This process is undertaken to enhance the accuracy of small targets and overlapping defect detection. Secondly, the Channel-Transposed Kernel Collaborative Attention (CKCA) module employs multi-head grouped self-attention with a learnable temperature to enhance crack classification and reduce the amount of computation. Thirdly, a lightweight Context-GuidedBlock-based downsampling module retains fine crack features via multi-branch context fusion and adaptive pooling. Furthermore, the P2 detection head, equipped with a two-layer cavity and dynamic convolution, enhances the detection of small targets through high-resolution feature fusion and dynamic weighting. The RDD2022 dataset was utilized in a series of experiments, which yielded enhancements of 2.4%, 2.6%, and 5.8% in mAP@50, precision, and recall, respectively. These enhancements were effective in reducing missed detections and false alarms, thereby supporting real-time road inspection and intelligent highway development.