Urban traffic congestion remains a critical challenge, often worsened by inefficient and frequent signal changes at intersections. Traditional reinforcement learning (RL) methods optimize for cumulative reward but neglect the cost of excessive actions, leading to unstable signal control. This is attributed to the distributed nature of the computational problem associated. The vehicular arrival pattern varies from junction to junction complicating the smooth transit at each junction without encountering a red signal. These patterns are observed by Reinforcement Learning agents, and they coordinate with each other. Overall, we propose a Greedy Action-Minimized Reinforcement Learning (GAM-RL) framework that integrates a sparsity-inducing penalty inspired by Basis Pursuit into the Q-learning objective. Each traffic intersection is modelled as an independent RL agent that observes lane-level congestion and learns when to change or retain signal phases. The Bellman update is modified to include an ℓ₁-regularized term, discouraging unnecessary phase switches and promoting control stability. Experiments conducted over 300 episodes demonstrate that GAM-RL achieves higher average cumulative rewards than standard RL while reducing the number of signal changes per episode by a significant margin. The results confirm that GAM-RL balances flow efficiency and actuation cost, making it suitable for real-world traffic systems where stability and fairness are as important as throughput.