Adaptive Beam Search with Shannon Entropy for Data-Centric Reasoning in LLMs

Reasoning capabilities of large language models (LLMs) have been significantly enhanced by structured prompting methods that leverage search over reasoning structures (e.g., Tree-of-Thoughts), making them increasingly valuable for data-centric reasoning and data-driven decision-making. However, existing methods with fixed exploration strategies often lead to suboptimal solutions or high computational costs due to exhaustive search. We propose an adaptive beam search method that uses entropy to dynamically adjust exploration at each reasoning step, balancing accuracy and efficiency. Uncertainty at each step is quantified through Shannon entropy of the confidence distribution, which serves as a dynamic threshold calibrating beam coverage. This enables broader exploration under high uncertainty and narrower exploration when uncertainty is low. Experimental results on data-centric tasks, arithmetic, commonsense, and symbolic reasoning tasks with Llama and GPT models demonstrate substantial improvements over state-of-the-art structured prompting methods with reduced computational cost, highlighting the efficacy of entropy-based beam adjustment in enhancing the reasoning capabilities of LLMs. The code for our method is publicly available at our GitHub repository .