Digital devices like computers, laptops, mobiles and tablets, surveillance cameras and social media usage are becoming increasingly widespread, and have the potential to generate carbon emissions globally at a large scale. Sustainable computing plays a vital role in addressing these environmental challenges faced by the IT sector, particularly in reducing digital carbon footprints. The research work aims to reduce the digital carbon footprint in information technology systems, using the most famous Temporal Difference (TD) learning algorithms, Q-Learning and SARSA (State-Action-Reward-State-Action). Both TD algorithms aim to minimize carbon emissions by dynamically adjusting energy efficiency and renewable energy rates in CarbonReductionIT environment to update their value estimations based on the current state and future rewards. Q-Learning algorithm learns the optimal action-value function by updating its Q-values thus allowing for more efficient learning, whereas SARSA algorithm learns the optimal policy directly by interacting with the IT environment. Attempts have been made to identify the most suited temporal difference learning algorithm to learn the optimal policy to reduce carbon emissions. SARSA algorithm reduces the digital carbon footprint of IT systems, thereby enhancing its sustainability in IT operations. The experimental results clearly specify that SARSA algorithm performs better than Q-Learning algorithm, with respect to be more stable and converge faster, when learning rate is set as 0.9, being high. Discretization enables the SARSA agent to efficiently learn in complex IT environment to reduce emission reduction dynamically in developing AI-driven solution for global environmental changes.