Scholarlink

<p>Deep reinforcement learning (DRL) has achieved remarkable performance across a wide range of tasks, yet its growing computational demands raise significant environmental and economic concerns. While prior work has largely emphasized learning performance and sample efficiency, the energy consumption, carbon emissions, and monetary cost of DRL training remain poorly understood. In this work, we present a systematic energy benchmarking study of seven widely used DRL algorithms, i.e., DQN, TRPO, A2C, ARS, PPO, RecurrentPPO, and QR-DQN, evaluated on ten Atari 2600 benchmarks under identical hardware and software configurations. Each algorithm is trained for one million steps, with real-time power measurements used to estimate total energy usage, CO<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(_2\)</EquationSource> </InlineEquation>-equivalent emissions, and electricity cost. Our results over ten runs reveal substantial disparities in energy efficiency across algorithms: derivative-free methods such as ARS achieve comparable performance to DQN while consuming approximately 77% less energy. In contrast, distributional and recurrent approaches incur substantially higher environmental and monetary overheads: for example, RecurrentPPO produces over 1344% more CO<InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(_2\)</EquationSource> </InlineEquation> emissions and incurs more than 1367% higher national monetary cost than the most energy-efficient alternative (ARS), while QR-DQN exhibits moderately higher emissions relative to DQN. These findings demonstrate that algorithmic choice alone can induce multi-fold differences in energy efficiency without sacrificing performance, highlighting the importance of incorporating energy and sustainability metrics into the evaluation and design of future DRL systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Greener deep reinforcement learning: analysis of energy and carbon efficiency across Atari benchmarks

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月27日

查看全文
  • Jason Gardner,
  • Ayan Dutta,
  • Swapnoneel Roy,
  • O. Patrick Kreidl,
  • Ladislau Bölöni

摘要

Deep reinforcement learning (DRL) has achieved remarkable performance across a wide range of tasks, yet its growing computational demands raise significant environmental and economic concerns. While prior work has largely emphasized learning performance and sample efficiency, the energy consumption, carbon emissions, and monetary cost of DRL training remain poorly understood. In this work, we present a systematic energy benchmarking study of seven widely used DRL algorithms, i.e., DQN, TRPO, A2C, ARS, PPO, RecurrentPPO, and QR-DQN, evaluated on ten Atari 2600 benchmarks under identical hardware and software configurations. Each algorithm is trained for one million steps, with real-time power measurements used to estimate total energy usage, CO \(_2\) -equivalent emissions, and electricity cost. Our results over ten runs reveal substantial disparities in energy efficiency across algorithms: derivative-free methods such as ARS achieve comparable performance to DQN while consuming approximately 77% less energy. In contrast, distributional and recurrent approaches incur substantially higher environmental and monetary overheads: for example, RecurrentPPO produces over 1344% more CO \(_2\) emissions and incurs more than 1367% higher national monetary cost than the most energy-efficient alternative (ARS), while QR-DQN exhibits moderately higher emissions relative to DQN. These findings demonstrate that algorithmic choice alone can induce multi-fold differences in energy efficiency without sacrificing performance, highlighting the importance of incorporating energy and sustainability metrics into the evaluation and design of future DRL systems.