This paper provides a comprehensive review of deep learning methods for detecting and assessing road anomalies, which include issues like potholes, open manholes, alligator cracks, debris and exposed cable trays etc. These anomalies arise from factors such as heavy traffic, rainfall, overloaded vehicles, and the use of poor-quality construction materials. We examine key aspects of anomaly detection, including dataset preprocessing, image annotation, and feature extraction techniques. Additionally, we compare the performance of real-time object detection models from YOLOv4 to YOLOv8 as these models may have low inference time, using various evaluation metrics. Real time detection requires to detect objects quickly which can be accomplished with these YOLO versions. Our review synthesizes findings from over 30 studies published in leading journals and conferences, offering valuable insights into the application of deep learning for road anomaly detection.