The increasing deployment of Unmanned Aerial Vehicles (UAVs) in smart cities has accelerated the demand for adaptive and autonomous communication infrastructures. However, traditional routing protocols such as Ad hoc On-demand Distance Vector (AODV) and Greedy Perimeter Stateless Routing (GPSR) struggle to maintain reliable performance in highly dynamic UAVs. In this work, we propose a novel reinforcement learning (RL)-based routing protocol to address the challenges of UAVs operating in urban environments. By modeling the routing problem as a Markov Decision Process (MDP), each UAV agent learns to select optimal relay nodes based on local observations and transmission feedback. The proposed solution integrates Q-learning with adaptive reward mechanisms that consider link stability, energy efficiency, and end-to-end latency. The results demonstrated that our proposed protocol significantly outperforms traditional approaches, enhances adaptability, reduces latency, and improves throughput. The findings highlight the potential of RL-based routing strategies for next-generation aerial communication systems in smart cities.