Scholarlink

<p>Reinforcement learning (RL) has emerged as a promising approach for building energy management (BEM). However, most existing research focuses on model-free reinforcement learning (MFRL) approaches, which can encounter the learning challenge for heating, ventilation and air conditioning (HVAC) control due to extensive trial-and-error explorations and lengthy training times. To address this challenge, we propose a model-based reinforcement learning (MBRL) framework that incorporates a virtual environment to augment the agent’s exploration. By leveraging the branched rollout strategy to generate short rollout predictions branched from the experience trajectory, the MBRL method mitigates compounding errors introduced by the time-series prediction model, enabling robust and efficient policy updates. Evaluated in an EnergyPlus testbed with real-world data verification, the proposed method demonstrates significant advantages: (1) RL-based controllers outperform the rule-based control (RBC) baseline after one training episode, (2) MBRL reduces training time by over 50% compared to MFRL while maintaining comparable control performance, and (3) an equal mix of real and synthetic data for MBRL training achieves an optimal trade-off between efficiency and control outcomes. This study contributes an efficient model-based training method for RL development in HVAC control, offering insights into advanced control strategies for BEM applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A model-based reinforcement learning framework for building heating management with branched rollout strategy and time-series prediction model

  • 期刊论文

  • 发表日期：

    2025年5月23日

查看全文
  • Kaichen Qu,
  • Hong Zhang,
  • Xin Zhou,
  • Martina Ferrando,
  • Francesco Causone

摘要

Reinforcement learning (RL) has emerged as a promising approach for building energy management (BEM). However, most existing research focuses on model-free reinforcement learning (MFRL) approaches, which can encounter the learning challenge for heating, ventilation and air conditioning (HVAC) control due to extensive trial-and-error explorations and lengthy training times. To address this challenge, we propose a model-based reinforcement learning (MBRL) framework that incorporates a virtual environment to augment the agent’s exploration. By leveraging the branched rollout strategy to generate short rollout predictions branched from the experience trajectory, the MBRL method mitigates compounding errors introduced by the time-series prediction model, enabling robust and efficient policy updates. Evaluated in an EnergyPlus testbed with real-world data verification, the proposed method demonstrates significant advantages: (1) RL-based controllers outperform the rule-based control (RBC) baseline after one training episode, (2) MBRL reduces training time by over 50% compared to MFRL while maintaining comparable control performance, and (3) an equal mix of real and synthetic data for MBRL training achieves an optimal trade-off between efficiency and control outcomes. This study contributes an efficient model-based training method for RL development in HVAC control, offering insights into advanced control strategies for BEM applications.