As a core component of autonomous driving systems, trajectory tracking control technology has been widely studied in the field of intelligent vehicles. However, in scenarios such as mining transportation and disaster relief, the heavy-duty characteristics of vehicles and the unstructured nature of the roads present significant challenges to the implementation of autonomous driving technology. This paper addresses the issue of model mismatch caused by road curvature and surface adhesion variations on unstructured roads by proposing a tracking control strategy based on Tube Model Predictive Control (Tube-MPC) with a robust invariant set. The core of the method involves analyzing the impact of road curvature and surface adhesion on tire lateral forces, solving for the nominal system control input through the optimization of the invariant set, and combining it with the feedback gain from the closed-loop control system to obtain the actual control input. This approach aims to achieve higher tracking accuracy on unstructured roads. The effectiveness of the proposed method is validated through simulation experiments.