We present a framework that leverages Markov Chain Monte Carlo (MCMC) to enhance abductive reasoning in large language models (LLMs). Abductive reasoning, the task of inferring the most plausible explanation for a given observation, remains a difficult task for LLMs, especially when information is incomplete or ambiguous. Existing methods typically rely on static retrieval strategies that struggle to adapt to diverse reasoning contexts. In contrast, our approach employs an unsupervised MCMC algorithm to efficiently explore large premise spaces, balancing exploration and exploitation to identify the most relevant supporting evidence. These premises are dynamically reordered to appear at the beginning of the prompt, guiding LLMs toward generating more accurate and coherent hypotheses. Experimental results demonstrate substantial gains in both premise recall and hypothesis consistency, highlighting the effectiveness of probabilistic modeling in complex reasoning tasks. When evaluated on the Entailment Bank dataset, our method significantly improves premise retrieval, enabling LLMs to generate hypotheses that better align with the ground truth.