<p>Accurate identification of road defects like potholes and cracks is essential for timely driver warnings, maintenance prioritization, and accident prevention. However, there is a gap in developing practical real-time road damage detection algorithms. This survey reviews the most efficient deep learning models and available Road Damage Detection (RDD) datasets, comparing them based on accuracy, complexity, and inference rate for real-time application suitability. This survey investigates the available datasets for road damage detection and the evaluation metrics used to assess object detection models. Additionally, it explores recent methods and deep learning models for small object detection, including You Only Look Once (YOLO), Region-based Convolutional Neural Network (R-CNN), and Single Shot Detector (SSD). The survey also provides an overview of Convolutional Neural Networks (CNN) fundamentals and attention mechanisms. A comparative analysis of the models and datasets is conducted to highlight their strengths and limitations. The key findings in road damage detection, particularly those related to the Crowdsensing-based Road Damage Detection Challenge (CRDDC, 2022), are summarized.</p>