Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning Algorithms
To face the global energy crisis, the requirement of energy transition and sustainable development has emphasized the importance of controlling building energy management systems. Reinforcement learning (RL) has shown notable energy-saving potential in the optimal control of heating, ventilation, an...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-04-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/18/9/2225 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850030765320962048 |
|---|---|
| author | Xiangrui Wang Qilin Zhang Zhihua Chen Jingjing Yang Yixing Chen |
| author_facet | Xiangrui Wang Qilin Zhang Zhihua Chen Jingjing Yang Yixing Chen |
| author_sort | Xiangrui Wang |
| collection | DOAJ |
| description | To face the global energy crisis, the requirement of energy transition and sustainable development has emphasized the importance of controlling building energy management systems. Reinforcement learning (RL) has shown notable energy-saving potential in the optimal control of heating, ventilation, and air-conditioning (HVAC) systems. However, the coupling of the algorithms and environments limits the cross-scenario application. This paper develops chiller plant models in OpenAI Gym environments to evaluate different RL algorithms for optimizing condenser water loop control. A shopping mall in Changsha, China, was selected as the case study building. First, an energy simulation model in EnergyPlus was generated using AutoBPS. Then, the OpenAI Gym chiller plant system model was developed and validated by comparing it with the EnergyPlus simulation results. Moreover, two RL algorithms, Deep-Q-Network (DQN) and Double Deep-Q-Network (DDQN), were deployed to control the condenser water flow rate and approach temperature of cooling towers in the RL environment. Finally, the optimization performance of DQN across three climate zones was evaluated using the AutoBPS-Gym toolkit. The findings indicated that during the cooling season in a shopping mall in Changsha, the DQN control method resulted in energy savings of 14.16% for the cooling water system, whereas the DDQN method achieved savings of 14.01%. Using the average control values from DQN, the EnergyPlus simulation recorded an energy-saving rate of 10.42% compared to the baseline. Furthermore, implementing the DQN algorithm across three different climatic zones led to an average energy savings of 4.0%, highlighting the toolkit’s ability to effectively utilize RL for optimal control in various environmental contexts. |
| format | Article |
| id | doaj-art-edefa42202504f11961b8eb393b8d751 |
| institution | DOAJ |
| issn | 1996-1073 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj-art-edefa42202504f11961b8eb393b8d7512025-08-20T02:59:08ZengMDPI AGEnergies1996-10732025-04-01189222510.3390/en18092225Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning AlgorithmsXiangrui Wang0Qilin Zhang1Zhihua Chen2Jingjing Yang3Yixing Chen4College of Civil Engineering, Hunan University, Changsha 410082, ChinaCollege of Civil Engineering, Hunan University, Changsha 410082, ChinaDepartment of Building Science, School of Architecture, Tsinghua University, Beijing 100084, ChinaCollege of Civil Engineering, Hunan University, Changsha 410082, ChinaCollege of Civil Engineering, Hunan University, Changsha 410082, ChinaTo face the global energy crisis, the requirement of energy transition and sustainable development has emphasized the importance of controlling building energy management systems. Reinforcement learning (RL) has shown notable energy-saving potential in the optimal control of heating, ventilation, and air-conditioning (HVAC) systems. However, the coupling of the algorithms and environments limits the cross-scenario application. This paper develops chiller plant models in OpenAI Gym environments to evaluate different RL algorithms for optimizing condenser water loop control. A shopping mall in Changsha, China, was selected as the case study building. First, an energy simulation model in EnergyPlus was generated using AutoBPS. Then, the OpenAI Gym chiller plant system model was developed and validated by comparing it with the EnergyPlus simulation results. Moreover, two RL algorithms, Deep-Q-Network (DQN) and Double Deep-Q-Network (DDQN), were deployed to control the condenser water flow rate and approach temperature of cooling towers in the RL environment. Finally, the optimization performance of DQN across three climate zones was evaluated using the AutoBPS-Gym toolkit. The findings indicated that during the cooling season in a shopping mall in Changsha, the DQN control method resulted in energy savings of 14.16% for the cooling water system, whereas the DDQN method achieved savings of 14.01%. Using the average control values from DQN, the EnergyPlus simulation recorded an energy-saving rate of 10.42% compared to the baseline. Furthermore, implementing the DQN algorithm across three different climatic zones led to an average energy savings of 4.0%, highlighting the toolkit’s ability to effectively utilize RL for optimal control in various environmental contexts.https://www.mdpi.com/1996-1073/18/9/2225reinforcement learningchiller plantOpenAI GymAutoBPSoptimal control |
| spellingShingle | Xiangrui Wang Qilin Zhang Zhihua Chen Jingjing Yang Yixing Chen Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning Algorithms Energies reinforcement learning chiller plant OpenAI Gym AutoBPS optimal control |
| title | Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning Algorithms |
| title_full | Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning Algorithms |
| title_fullStr | Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning Algorithms |
| title_full_unstemmed | Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning Algorithms |
| title_short | Development of Chiller Plant Models in OpenAI Gym Environment for Evaluating Reinforcement Learning Algorithms |
| title_sort | development of chiller plant models in openai gym environment for evaluating reinforcement learning algorithms |
| topic | reinforcement learning chiller plant OpenAI Gym AutoBPS optimal control |
| url | https://www.mdpi.com/1996-1073/18/9/2225 |
| work_keys_str_mv | AT xiangruiwang developmentofchillerplantmodelsinopenaigymenvironmentforevaluatingreinforcementlearningalgorithms AT qilinzhang developmentofchillerplantmodelsinopenaigymenvironmentforevaluatingreinforcementlearningalgorithms AT zhihuachen developmentofchillerplantmodelsinopenaigymenvironmentforevaluatingreinforcementlearningalgorithms AT jingjingyang developmentofchillerplantmodelsinopenaigymenvironmentforevaluatingreinforcementlearningalgorithms AT yixingchen developmentofchillerplantmodelsinopenaigymenvironmentforevaluatingreinforcementlearningalgorithms |