Improving Uncertainty Quantification and Knowledge-Intensive Routing via Query Understanding in Large Language Models

Reliable Uncertainty Quantification (UQ) is essential for evaluating the trustworthiness of large language models (LLMs). However, existing UQ methods often rely on entropy signals that are themselves erroneous: the entropy associated with a response can be low even when the answer is wrong, or high when the answer is correct. Such entropy errors arise in part because the model generates responses without sufficiently validating its own understanding of the query, leading to misaligned uncertainty estimates rather than merely generic overconfidence. We hypothesize that explicitly modeling an LLM’s internal understanding of the query can reduce these entropy errors and yield more accurate uncertainty. To this end, we propose a Query Understanding (QU)–based reasoning framework that guides the model through four steps: paraphrasing the input, checking semantic consistency among paraphrases, verbalizing an understanding score, and finally generating an answer conditioned on this self-assessment. By integrating QU into the generation process, we induce more informative entropy patterns and improve multiple UQ metrics, including Predictive Entropy, Length-Normalized Predictive Entropy, Semantic Entropy, and Shifting Attention to Relevance. Leveraging these refined uncertainty estimates, we further introduce an uncertainty-driven routing mechanism for knowledge-intensive generation (KIG), where KIG is instantiated by methods such as RAG and RECITATION. Experiments across four LLM families and three QA benchmarks show that our approach consistently improves AUROC for UQ, and enhances QA performance through selective KIG.