This work introduces an algorithm called Digital Twin-Aided Contextual Bandit Learning-based Two-Sided Matching (DTCBL-TSM), to efficiently offload computation tasks in dynamic fog computing networks (FCNs). Our proposed framework leverages digital twins (DTs) to enable precise monitoring and prediction of resource states of fog nodes (FNs). By integrating contextual bandit learning, our approach effectively handles the changing network conditions by the optimal learning strategy over time. Indeed, the two-sided matching mechanism ensures a balanced and fair allocation of tasks to fog nodes, considering both the task requirements and node capacities. Extensive simulations demonstrate that DTCBL-TSM outperforms existing methods in terms of task completion time, resource utilization, and adaptability to network dynamics.