The Traveling Salesman Problem (TSP) is a classic combinatorial optimization challenge with broad applications in logistics and transportation. While traditional heuristics remain dominant due to their efficiency and reliability, recent developments in Deep Reinforcement Learning (DRL) have introduced promising new directions for data-driven approaches to solving the TSP. Although DRL-based methods show promise, they often do not yet match classical heuristics in terms of computational efficiency and solution quality. One promising direction to bridging this gap is the integration of classical heuristics with learning-based methods. The Learn-to-Improve (L2I) framework follows this hybrid paradigm, combining heuristics with reinforcement learning to iteratively refine solutions. In this paper, we propose a novel multi-action sampling strategy that further enhances the L2I framework for solving TSP. The core idea is to improve solution quality by averaging rewards over multiple actions, which reduces bias and encourages more effective exploration compared to single-action strategies. During inference, multi-action sampling is applied in the later stages to explore diverse solution paths in parallel, helping to prevent premature convergence. Experimental results demonstrate that the proposed method outperforms existing L2I approaches while maintaining competitive computational efficiency.