Conditional Contrastive Confidence-Based Uncertainty Quantification for LLMs

Uncertainty quantification (UQ) is critical, since LLMs frequently produce confident-sounding but incorrect responses even in high-stakes applications. Existing UQ methods for LLMs, such as Semantic Entropy and SelfCheckGPT, rely on multiple sampled generations, resulting in high computational cost and limited scalability. Recent work suggests that LLM confidence is closely tied to prompt sensitivity: confident predictions remain stable under minor instruction changes, whereas uncertain ones exhibit greater variability. Building on this insight, we introduce \({\textbf {C}}^3\) , a fast contrastive confidence-based UQ method that estimates the uncertainty of an LLM’s response to a user prompt, given a system instruction, by measuring how its token-level output distributions change when that system instruction is perturbed by a learned instruction-level modification. This learned system-instruction perturbation is optimized to maximize the token-level distributional contrast between confident and hallucination-prone regions within the same generated response, and it generalizes across user inputs, models, and tasks. With only one additional forward pass, \(\text {C}^3\) achieves comparable or superior hallucination-detection performance to state-of-the-art baselines while reducing inference cost by over 97%. Extensive analyses further demonstrate the generalizability and interpretability of the learned contrastive instructions, establishing \(\text {C}^3\) as an efficient and effective tool for uncertainty estimation in modern LLMs.