Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement Learning
The increasing integration of renewable energy sources (RESs) introduces significant uncertainties in both generation and demand, presenting critical challenges to the convergence, feasibility, and real-time performance of optimal power flow (OPF). To address these challenges, a multi-agent deep rei...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-04-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/18/7/1809 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849730314999431168 |
|---|---|
| author | Liangcai Zhou Long Huo Linlin Liu Hao Xu Rui Chen Xin Chen |
| author_facet | Liangcai Zhou Long Huo Linlin Liu Hao Xu Rui Chen Xin Chen |
| author_sort | Liangcai Zhou |
| collection | DOAJ |
| description | The increasing integration of renewable energy sources (RESs) introduces significant uncertainties in both generation and demand, presenting critical challenges to the convergence, feasibility, and real-time performance of optimal power flow (OPF). To address these challenges, a multi-agent deep reinforcement learning (DRL) model is proposed to solve the OPF while ensuring constraints are satisfied rapidly. A heterogeneous multi-agent proximal policy optimization (H-MAPPO) DRL algorithm is introduced for multi-area power systems. Each agent is responsible for regulating the output of generation units in a specific area, and together, the agents work to achieve the global OPF objective, which reduces the complexity of the DRL model’s training process. Additionally, a graph neural network (GNN) is integrated into the DRL framework to capture spatiotemporal features such as RES fluctuations and power grid topological structures, enhancing input representation and improving the learning efficiency of the DRL model. The proposed DRL model is validated using the RTS-GMLC test system, and its performance is compared to MATPOWER with the interior-point iterative solver. The RTS-GMLC test system is a power system with high spatial–temporal resolution and near-real load profiles and generation curves. Test results demonstrate that the proposed DRL model achieves a 100% convergence and feasibility rate, with an optimal generation cost similar to that provided by MATPOWER. Furthermore, the proposed DRL model significantly accelerates computation, achieving up to 85 times faster processing than MATPOWER. |
| format | Article |
| id | doaj-art-3fce200b980046fc8bf136aad97e4588 |
| institution | DOAJ |
| issn | 1996-1073 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj-art-3fce200b980046fc8bf136aad97e45882025-08-20T03:08:55ZengMDPI AGEnergies1996-10732025-04-01187180910.3390/en18071809Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement LearningLiangcai Zhou0Long Huo1Linlin Liu2Hao Xu3Rui Chen4Xin Chen5East China Division, State Grid Corporation of China, No. 882, Pudong South Road, Pudong New Area, Shanghai 200002, ChinaCenter of Nanomaterials for Renewable Energy, State Key Laboratory of Electrical Insulation and Power Equipment, School of Electrical Engineering, Xi’an Jiaotong University, Xi’an 710049, ChinaEast China Division, State Grid Corporation of China, No. 882, Pudong South Road, Pudong New Area, Shanghai 200002, ChinaEast China Division, State Grid Corporation of China, No. 882, Pudong South Road, Pudong New Area, Shanghai 200002, ChinaCenter of Nanomaterials for Renewable Energy, State Key Laboratory of Electrical Insulation and Power Equipment, School of Electrical Engineering, Xi’an Jiaotong University, Xi’an 710049, ChinaCenter of Nanomaterials for Renewable Energy, State Key Laboratory of Electrical Insulation and Power Equipment, School of Electrical Engineering, Xi’an Jiaotong University, Xi’an 710049, ChinaThe increasing integration of renewable energy sources (RESs) introduces significant uncertainties in both generation and demand, presenting critical challenges to the convergence, feasibility, and real-time performance of optimal power flow (OPF). To address these challenges, a multi-agent deep reinforcement learning (DRL) model is proposed to solve the OPF while ensuring constraints are satisfied rapidly. A heterogeneous multi-agent proximal policy optimization (H-MAPPO) DRL algorithm is introduced for multi-area power systems. Each agent is responsible for regulating the output of generation units in a specific area, and together, the agents work to achieve the global OPF objective, which reduces the complexity of the DRL model’s training process. Additionally, a graph neural network (GNN) is integrated into the DRL framework to capture spatiotemporal features such as RES fluctuations and power grid topological structures, enhancing input representation and improving the learning efficiency of the DRL model. The proposed DRL model is validated using the RTS-GMLC test system, and its performance is compared to MATPOWER with the interior-point iterative solver. The RTS-GMLC test system is a power system with high spatial–temporal resolution and near-real load profiles and generation curves. Test results demonstrate that the proposed DRL model achieves a 100% convergence and feasibility rate, with an optimal generation cost similar to that provided by MATPOWER. Furthermore, the proposed DRL model significantly accelerates computation, achieving up to 85 times faster processing than MATPOWER.https://www.mdpi.com/1996-1073/18/7/1809multi-agent reinforcement learningoptimal power flowgraph neural networkrenewable energy source |
| spellingShingle | Liangcai Zhou Long Huo Linlin Liu Hao Xu Rui Chen Xin Chen Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement Learning Energies multi-agent reinforcement learning optimal power flow graph neural network renewable energy source |
| title | Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement Learning |
| title_full | Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement Learning |
| title_fullStr | Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement Learning |
| title_full_unstemmed | Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement Learning |
| title_short | Optimal Power Flow for High Spatial and Temporal Resolution Power Systems with High Renewable Energy Penetration Using Multi-Agent Deep Reinforcement Learning |
| title_sort | optimal power flow for high spatial and temporal resolution power systems with high renewable energy penetration using multi agent deep reinforcement learning |
| topic | multi-agent reinforcement learning optimal power flow graph neural network renewable energy source |
| url | https://www.mdpi.com/1996-1073/18/7/1809 |
| work_keys_str_mv | AT liangcaizhou optimalpowerflowforhighspatialandtemporalresolutionpowersystemswithhighrenewableenergypenetrationusingmultiagentdeepreinforcementlearning AT longhuo optimalpowerflowforhighspatialandtemporalresolutionpowersystemswithhighrenewableenergypenetrationusingmultiagentdeepreinforcementlearning AT linlinliu optimalpowerflowforhighspatialandtemporalresolutionpowersystemswithhighrenewableenergypenetrationusingmultiagentdeepreinforcementlearning AT haoxu optimalpowerflowforhighspatialandtemporalresolutionpowersystemswithhighrenewableenergypenetrationusingmultiagentdeepreinforcementlearning AT ruichen optimalpowerflowforhighspatialandtemporalresolutionpowersystemswithhighrenewableenergypenetrationusingmultiagentdeepreinforcementlearning AT xinchen optimalpowerflowforhighspatialandtemporalresolutionpowersystemswithhighrenewableenergypenetrationusingmultiagentdeepreinforcementlearning |