The importance of vehicle ad hoc networks (VANETs) lies in their ability to improve traffic monitoring, enhance road safety, and provide in-car infotainment. However, these networks face significant challenges, such as frequent disconnections between vehicles due to high mobility, limited bandwidth, roadside obstructions, and a scarcity of roadside units. Effective routing becomes a critical aspect in addressing these issues. In this paper, a novel approach utilizing a reinforcement learning (RL) strategy based on the Q-learning algorithm is presented. The objective is to use RL to enable vehicles to establish and maintain a stable connection even in the presence of multiple roadside units, thereby mitigating the problem of frequent disconnections. This innovative solution aims to enhance the overall performance of VANETs, & hence contributing to more seamless and efficient V2V2I communication among vehicles on the road.