RefineChain: A Dual-Phase Refinement Framework for Complex Question Decomposition in Large Language Models

Large language models (LLMs) have shown strong performance on complex natural language reasoning tasks when equipped with step-by-step decomposition strategies such as Chain-of-Thought (CoT) or Tree-of-Thought (ToT). However, these decomposition-based methods often suffer from issues like incomplete reasoning steps and logical inconsistencies, which limit their reliability and generalizability. To address these problems, we propose RefineChain, a lightweight and post-hoc refinement framework that enhances reasoning quality by explicitly targeting these two common pitfalls. RefineChain operates in two stages: Self-completion for filling missing steps, and Self-reflection for revising illogical reasoning. Moreover, it is model-agnostic and compatible with various reasoning paradigms. Experimental results across three diverse benchmarks, i.e., HotpotQA, 24 Game, and Crosswords, demonstrate that RefineChain consistently improves performance without modifying the original reasoning structure. In particular, it boosts CoT’s exact match accuracy on HotpotQA from 67.11 to 70.89%. These findings highlight RefineChain’s potential as a general-purpose, plug-and-play solution for text-based optimizing reasoning in LLMs.