Large Language models have demonstrated remarkable fluency in solving complex problems that require multi-step reasoning. However, in practice, LLMs are essentially statistical pattern matchers that rely on auto-regressive next-token prediction, along with partial predefined knowledge about the solution space. When it comes to sequential reasoning, LLMs generate a reasoning step after another in a linear sequence, much like a chain of thoughts. Still, when a step leads to a dead end, it cannot go back, and instead, it would start another single reasoning trajectory, which leads to locally optimal choices without exploring the global solution space. In this work, we present a systematic approach that combines the Tree of Thoughts (ToT) with the A* search algorithm to evaluate neural states, thereby facilitating a simultaneous exploration of multiple reasoning trajectories. The main contribution is to use structured search to sift through the LLM’s reasoning paths, then employ the LLM as a local monitoring agent to evaluate each path’s consistency over the overall reasoning trajectories. We demonstrate the effectiveness of our approach on the Game of 24 while incorporating the AceMath-1.5B-Instruct model as the backbone of our approach. Our framework achieves a 100% success rate on 30 of the most complex cases with the lowest solve rate, which outperforms established frontier approaches such as ToT(b=5) 74%, CoT-SC 9.0%. This shows that heuristic search coupled with Tree-of-Thoughts can help improve the reasoning capabilities of LLMs by evaluating the effectiveness of the reasoning path early, which helps recursively correct their thought generation process to avoid inconsistent answers.