Deep Reinforcement Learning for Intraday Multireservoir Hydropower Management
This study investigates the application of Reinforcement Learning (RL) to optimize intraday operations of hydropower reservoirs. Unlike previous approaches that focus on long-term planning with coarse temporal resolutions and discretized state-action spaces, we propose an RL framework tailored to th...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/13/1/151 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549119858933760 |
|---|---|
| author | Rodrigo Castro-Freibott Álvaro García-Sánchez Francisco Espiga-Fernández Guillermo González-Santander de la Cruz |
| author_facet | Rodrigo Castro-Freibott Álvaro García-Sánchez Francisco Espiga-Fernández Guillermo González-Santander de la Cruz |
| author_sort | Rodrigo Castro-Freibott |
| collection | DOAJ |
| description | This study investigates the application of Reinforcement Learning (RL) to optimize intraday operations of hydropower reservoirs. Unlike previous approaches that focus on long-term planning with coarse temporal resolutions and discretized state-action spaces, we propose an RL framework tailored to the Hydropower Reservoirs Intraday Economic Optimization problem. This framework manages continuous state-action spaces while accounting for fine-grained temporal dynamics, including dam-to-turbine delays, gate movement constraints, and power group operations. Our methodology evaluates three distinct action space formulations (continuous, discrete, and adjustments) implemented using modern RL algorithms (A2C, PPO, and SAC). We compare them against both a greedy baseline and Mixed-Integer Linear Programming (MILP) solutions. Experiments on real-world data from a two-reservoir system and a simulated six-reservoir system demonstrate that while MILP achieves superior performance in the smaller system, its performance degrades significantly when scaled to six reservoirs. In contrast, RL agents, particularly those using discrete action spaces and trained with PPO, maintain consistent performance across both configurations, achieving considerable improvements with less than one second of execution time. These results suggest that RL offers a scalable alternative to traditional optimization methods for hydropower operations, particularly in scenarios requiring real-time decision making or involving larger systems. |
| format | Article |
| id | doaj-art-10eac581c999439cb199c4f0f8eec7b0 |
| institution | Kabale University |
| issn | 2227-7390 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj-art-10eac581c999439cb199c4f0f8eec7b02025-01-10T13:18:25ZengMDPI AGMathematics2227-73902025-01-0113115110.3390/math13010151Deep Reinforcement Learning for Intraday Multireservoir Hydropower ManagementRodrigo Castro-Freibott0Álvaro García-Sánchez1Francisco Espiga-Fernández2Guillermo González-Santander de la Cruz3baobab soluciones, José Abascal 55, 28003 Madrid, SpainIndustrial Engineering, Business Administration and Statistics Department, Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid, José Gutierrez Abascal 2, 28006 Madrid, SpainIndustrial Engineering, Business Administration and Statistics Department, Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid, José Gutierrez Abascal 2, 28006 Madrid, Spainbaobab soluciones, José Abascal 55, 28003 Madrid, SpainThis study investigates the application of Reinforcement Learning (RL) to optimize intraday operations of hydropower reservoirs. Unlike previous approaches that focus on long-term planning with coarse temporal resolutions and discretized state-action spaces, we propose an RL framework tailored to the Hydropower Reservoirs Intraday Economic Optimization problem. This framework manages continuous state-action spaces while accounting for fine-grained temporal dynamics, including dam-to-turbine delays, gate movement constraints, and power group operations. Our methodology evaluates three distinct action space formulations (continuous, discrete, and adjustments) implemented using modern RL algorithms (A2C, PPO, and SAC). We compare them against both a greedy baseline and Mixed-Integer Linear Programming (MILP) solutions. Experiments on real-world data from a two-reservoir system and a simulated six-reservoir system demonstrate that while MILP achieves superior performance in the smaller system, its performance degrades significantly when scaled to six reservoirs. In contrast, RL agents, particularly those using discrete action spaces and trained with PPO, maintain consistent performance across both configurations, achieving considerable improvements with less than one second of execution time. These results suggest that RL offers a scalable alternative to traditional optimization methods for hydropower operations, particularly in scenarios requiring real-time decision making or involving larger systems.https://www.mdpi.com/2227-7390/13/1/151daily optimizationhydropower generationmultireservoirreinforcement learningmixed integer linear programming |
| spellingShingle | Rodrigo Castro-Freibott Álvaro García-Sánchez Francisco Espiga-Fernández Guillermo González-Santander de la Cruz Deep Reinforcement Learning for Intraday Multireservoir Hydropower Management Mathematics daily optimization hydropower generation multireservoir reinforcement learning mixed integer linear programming |
| title | Deep Reinforcement Learning for Intraday Multireservoir Hydropower Management |
| title_full | Deep Reinforcement Learning for Intraday Multireservoir Hydropower Management |
| title_fullStr | Deep Reinforcement Learning for Intraday Multireservoir Hydropower Management |
| title_full_unstemmed | Deep Reinforcement Learning for Intraday Multireservoir Hydropower Management |
| title_short | Deep Reinforcement Learning for Intraday Multireservoir Hydropower Management |
| title_sort | deep reinforcement learning for intraday multireservoir hydropower management |
| topic | daily optimization hydropower generation multireservoir reinforcement learning mixed integer linear programming |
| url | https://www.mdpi.com/2227-7390/13/1/151 |
| work_keys_str_mv | AT rodrigocastrofreibott deepreinforcementlearningforintradaymultireservoirhydropowermanagement AT alvarogarciasanchez deepreinforcementlearningforintradaymultireservoirhydropowermanagement AT franciscoespigafernandez deepreinforcementlearningforintradaymultireservoirhydropowermanagement AT guillermogonzalezsantanderdelacruz deepreinforcementlearningforintradaymultireservoirhydropowermanagement |