A deep vision approach for pavement pothole quantification enhanced by marker-based spatial scaling

Potholes pose a serious threat to road safety and economic stability. Over $26.5 billion per year is the cost of vehicle damage and lost productivity from potholes worldwide. Potholes are also responsible for thousands of accidents and place infrastructure budgets under various pressures in both developed and developing countries. Previous traditional detection methods performed poorly with variations in lighting, shadows, and surface textures. Most deep learning methods lack precise scale calibration for accurate area estimation. The scalability of sensor-based systems remains limited because of their high cost. This study addresses these key issues with a cost-effective, sufficiently accurate, and reliable hybrid vision-based solution. It combines deep segmentation based on Mask R-CNN with precise scaling using ArUco markers. A total of 716 images were obtained using low-cost Android cameras along urban and rural roads in Ongole, Andhra Pradesh, India. The images were collected within the range of 0.5 to 2 m, demonstrating that data collection with low-cost cameras is feasible and reliable. Annotations were performed using LabelMe in the COCO(Common Object In Context) JSON format with an inter-annotator IoU > 0.8. The fine-tuned Mask R-CNN based on ResNet-50 accurately detected potholes and ArUco markers. Area and perimeter estimate by pixel counting on calibrated masks further enhance the reliability of images captured by drones. This was achieved by leveraging this robust methodology, which was evaluated on a robust test set, to achieve an average IoU of 0.889, AP@0.50:0.95 of 0.889, and area estimation error below 8%. These results significantly outperform those of YOLOv5 (AP ≈ 0.78) and earlier versions of Mask R-CNN (AP 0.613). This innovation provides affordable, highly accurate, and reliable maintenance cost reductions, reinforces smart city frameworks, and has multiple game-changing applications.