While various non-destructive techniques are being increasingly adopted for comprehensive condition assessment, Ground Penetrating Radar (GPR) is recommended for bridge evaluation due to its distinct advantages of identifying major subsurface defects rapidly. The interpretation of data obtained from GPR profiles is a major issue for bridge inspectors due to the difficulty in correlating with the actual condition. The commonly utilized amplitude-based approach yields results that are not always reliable, as it ignores most of the information contained within a GPR profile. A novel approach based on image-based analysis involves an experienced analyst reviewing the GPR profiles and marking attenuated areas across them while considering the structural and surface anomalies, and other several parameters. However, this approach is rather subjective, can be time-consuming, and is dependent on the level of knowledge of the analyst. To address these shortcomings, the proposed generalized framework leverages the benefits of engineering judgment derived from image-based analysis by incorporating user-assisted input. The process involves the following steps: (a) user-input for the element being inspected to determine optimal clusters; (b) automated detection of hyperbolic regions using machine learning or deep learning methods; (c) a user-assigned anomalies module that assists the user in identifying anomalies not related to corrosion; (d) entropy evaluation of hyperbolic regions; and finally, (e) clustering to generate condition maps. It is recommended that adopting such a holistic system leads to effective GPR data analysis, as the results would be closer to the ground truth and can be easily adapted by transportation authorities.