Roads play a vital role in the social, cultural, and economic landscape of a community, serving as essential transportation infrastructure. Properly maintained roads improve both comfort and economic activity. However, not all roads are well-constructed and may face challenges such as potholes. Conventional methods for pothole identification typically analyze the entire dataset, leading to inefficiencies and delays. This paper introduces a two-stage methodology to enhance efficiency. In the initial stage, a classification model determines if an image includes a pothole. Only those images identified as having potholes proceed to the second stage, where the actual detection takes place. This targeted approach significantly lessens the computational load compared to techniques that apply detection across the complete dataset. For the classification phase, we utilize MobileNet due to its efficient architecture, making it suitable for deployment on edge devices. In the subsequent stage, we implement the SSD model with a VGG16 backbone on the server side for accurate pothole detection. Our approach is assessed using real-world datasets, and the findings highlight its effectiveness. Moreover, we offer a simulation of our system to illustrate its practical application and performance in real-time situations.