Large language models (LLMs) have achieved impressive success across natural language processing tasks but face persistent challenges in inference efficiency due to their autoregressive nature. While speculative decoding and beam sampling offer improvements, traditional methods verify draft sequences sequentially and uniformly, causing unnecessary overhead. This work proposes the Hierarchical Verification Tree (HVT), a framework that restructures speculative beam decoding by prioritizing high-likelihood drafts and pruning suboptimal candidates early. A formal verification-pruning algorithm ensures correctness and efficiency. HVT integrates with standard LLM pipelines without retraining. Experiments demonstrate that HVT outperforms existing methods, achieving significant reductions in inference time and energy consumption while maintaining output quality. These results highlight the promise of hierarchical verification for accelerating LLM inference.