An end-to-end repair-based joint training framework for weakly supervised pavement crack segmentation

Weakly supervised pavement crack segmentation aims to assign each pixel of pavement surface images a category label (crack or non-crack) using limited annotation information. Most existing methods adopt the multi-stage approach based on the class activation map (CAM), which outputs from a pre-trained crack classification model, to generate the pixel-level pseudo labels, and then train a segmentation model using these pseudo labels. However, these methods have two notable drawbacks: (i) CAM focuses primarily on the most discriminative regions of the image, potentially resulting in an incomplete estimation of crack regions, and (ii) the process of obtaining pixel-level pseudo labels from CAM is complex and may introduce noise. To avoid these issues, we propose an end-to-end repair-based joint training framework for weakly supervised pavement crack segmentation. Specifically, to better guide the model to pay more attention to the repair process of crack regions, we introduce the segmentation loss, which assigns different weights for crack regions and background regions, through the Residual Project Network. Additionally, to alleviate the impact of the domain gap between synthetic and real datasets, we introduce the consistency loss through the joint training paradigm of unpaired data that incorporates both crack and non-crack images into the training process simultaneously. Furthermore, to enable the model to effectively extract local features and reduce the model's computational complexity, we introduce the Lightweight Multi-head Self-attention module. The experiment results on the CRACK500 Dataset, the CFD Dataset, and the AEL Dataset demonstrate that our model achieves better performance than some existing weakly supervised pavement segmentation methods.