Claim verification has become an increasingly important field with the rise of online misinformation. Content-based claim verification systems do not provide explicit evidence and are vulnerable to adversarial attacks. Current evidence-based approaches typically employ embedding-based Retrieval-Augmented Generation (RAG), but these conventional RAG systems struggle with multi-hop reasoning across documents. Recent work has proposed RAG systems that address this by indexing the document store into a Knowledge Graph (KG) for improved inter-document connectivity. In this study, we evaluate an explainable LLM-guided evidence retrieval framework using KGs for claim verification on the AVeriTeC dataset and propose a novel “ask first, index later” sparse retrieval approach to improve efficiency and cost-effectiveness. With an AVeriTeC score of 0.57, 0.46 above the baseline, our framework ranks among the top-performing systems. Additionally, it achieved Q only and Q+A scores of 0.47 and 0.34, respectively, which are highly comparable to state-of-the-art results, demonstrating its strong evidence retrieval capabilities and effectiveness in verifying real-world claims.