Vehicular Fog Computing (VFC) provides low-latency services for mobile edge computing (MEC) signal coverage blind spots by offloading latency-sensitive tasks to nearby mobile fog vehicle nodes. However, service interruptions caused by the dynamic mobility of vehicles and limited onboard computing resources result in extremely high task dropout rates for traditional offloading methods. To address this challenge, this paper proposes a Dynamically Constrained Q-learning (DC-QL) algorithm. This algorithm utilizes a dynamic action selection mechanism to optimize the task offloading process, maximizing the average delay utility of tasks over a period of time under resource availability and service availability constraints. Compared to the Semi-Markov Decision Process (SMDP), which relies on pre-trained strategies in static environments, DC-QL introduces a constraint-aware greedy exploration mechanism, effectively avoiding invalid decisions caused by dynamic changes in the network topology. Validation in real-world dynamic traffic flow scenarios demonstrates that the proposed algorithm not only improves the latency utility of task execution but also effectively reduces the task abandonment rate in high-load scenarios. Compared to the baseline algorithm, our proposed algorithm achieves latency efficiency improvements of 11.43%, 10.04%, and 19.93% in computationally intensive scenarios, while reducing task dropout rates by 19.02%, 53.72%, and 59.51%, respectively, thereby demonstrating its robustness in dynamic resource-constrained scenarios.