The rapid growth of Internet of Things (IoT) devices in Smart Cities necessitates efficient energy management to extend device lifetimes while maintaining data freshness. Traditional rule-based approaches fail to adapt to dynamic conditions, often resulting in suboptimal performance. This paper’s primary goal is to conduct a comparative study of four Deep Reinforcement Learning (DRL) algorithms—DQN, DDQN, PPO, and A2C—within a simulated IoT environment featuring realistic battery, network, and data constraints. To ensure a fair assessment, we design a high-fidelity simulator that accurately models device energy consumption, communication overhead, and the variability of data generation and network conditions. Evaluated under identical conditions, the algorithms are assessed on their ability to learn adaptive transmission, sleep, and processing policies. The results demonstrate that DRL agents significantly improve energy efficiency and data delivery compared to static policies, while also highlighting key differences in learning stability, adaptability, and overall performance among the algorithms.