Medication recommendation plays a crucial role in healthcare by supporting clinical decision-making processes. Recent studies demonstrate that combining thoughtfully devised patient representations with historical prescriptions, which serve as informative references, can substantially improve recommendation performance. However, due to the clinical semantic gap between the high-level abstraction of patient conditions embedded in such representations and the loosely structured medication sets, it is challenging to integrate them who differ in granularity. In this study, we highlight that multi-level summarization of historical prescriptions based on clinical semantics can bridge the gap. To this end, we propose a novel Graph Attention-Based MUlti-head Voting Strategy for Unsupervised Ranking (GAMUV), which performs clinically semantic-guided walk and integrates the results with coarse-grained temporal ordering of prescription dates to infer drug efficacy as informative references for the recommendation task. Experiments on real-world clinical records demonstrate that GAMUV outperforms all baseline methods and exhibits robustness across varying medication sequences lengths.