Retrieval-Augmented Generation (RAG) is a pioneering approach that greatly improves the factual accuracy and contextual relevance of model outputs by seamlessly incorporating background knowledge into the generation process. The efficacy of RAG hinges on the selection of pertinent evidence that not only exhibits semantic alignment with the query but also demonstrably contributes to the downstream task. However, a significant limitation of existing retrieval methods lies in their tendency to conflate relevance with utility to generate accurate and coherent answers. This conflation often leads to the retrieval of information that, while topically related, fails to provide substantial benefit for answer generation. In this work, we address this critical challenge within the context of visual document question answering (VDQA), a task demanding precise integration of visual and textual information from complex document layouts. Through a rigorous analysis of the interplay between relevance and utility, we demonstrate that while relevant information exhibits strong semantic alignment with queries, it frequently falls short of providing the necessary cues for generating accurate and contextually coherent answers in VDQA. Specifically, we observe that highly relevant document segments may contain redundant or distracting information, hindering the model’s ability to extract the essential visual and textual cues required for accurate question answering. Based on these findings, we propose a novel utility-driven retrieval method that explicitly prioritizes evidence based on its predicted contribution to answer quality. Our approach incorporates a learned utility scoring mechanism that assesses the potential of retrieved document segments to facilitate accurate generation.