This paper presents a novel methodology to optimize energy in commercial buildings. It explores two different techniques in reinforcement learning, Q-Learning and DDPG. A custom environment has been made using OpenAI GYM for testing purposes. Through this environment, an agent was trained to balance thermal comfort and energy consumption in simulated HVAC systems. The graph generated to showcase Q-Learning’s results showcased many fluctuations, thereby not giving a reliable output. DDPG (Deep Deterministic Policy Gradient), on the other hand, shows an increase in the reward value over time; the result graph always shows an upward trend. Two major metrics: reward and energy consumption are used to measure performance. DDPG provided better results as compared to Q-Learning, showing that reinforcement learning has the potential to improve the efficiency of HVAC systems, therefore offering energy optimization. The insights gained through this research can be used to develop intelligent systems with enhanced comfort and sustainability in future. By leveraging advanced RL techniques, stakeholders in the built environment sector can optimize HVAC system operations, reduce energy consumption, and minimize environmental impact, thereby advancing the paradigm of smart and eco-friendly building management practices.