Road signs provide the necessary information to the drivers and help protect them from the potential hazards on the roads. Hence, a pipeline involving detection, tracking, and recognition of road signs is an integral component of the Advanced Driver Assistance System (ADAS). A tracker helps to follow the detected object in real time. This paper presents a novel correlation-based approach for the localization of road signs while encountering unassociated tracks during tracking. The detection is necessary for the first time, but if in the subsequent frames, an unassociated track is found, then correlation is proposed as the remedial step to estimate the location of the road sign. The correlation along with the neighborhood restriction is attempted to associate the road signs and also help reduce the identity switches. Compared to deep learning-based association techniques, correlation-based tracking is computationally less expensive and can be exploited for real-time tasks. Lastly, the recognition module is included in the proposed pipeline to reduce the false positives and the experimental results validate the effectiveness of this pipeline.