Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants
Inverter-based resources (IBRs) are becoming more prominent due to the increasing penetration of renewable energy sources that reduce power system inertia, compromising power system stability and grid support services. At present, optimal coordination among generation technologies remains a signific...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-05-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/18/10/2666 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850258085558353920 |
|---|---|
| author | Muhammad Ikram Daryoush Habibi Asma Aziz |
| author_facet | Muhammad Ikram Daryoush Habibi Asma Aziz |
| author_sort | Muhammad Ikram |
| collection | DOAJ |
| description | Inverter-based resources (IBRs) are becoming more prominent due to the increasing penetration of renewable energy sources that reduce power system inertia, compromising power system stability and grid support services. At present, optimal coordination among generation technologies remains a significant challenge for frequency control services. This paper presents a novel networked multi-agent deep reinforcement learning (N—MADRL) scheme for optimal dispatch and frequency control services. First, we develop a model-free environment consisting of a photovoltaic (PV) plant, a wind plant (WP), and an energy storage system (ESS) plant. The proposed framework uses a combination of multi-agent actor-critic (MAAC) and soft actor-critic (SAC) schemes for optimal dispatch of active power, mitigating frequency deviations, aiding reserve capacity management, and improving energy balancing. Second, frequency stability and optimal dispatch are formulated in the N—MADRL framework using the physical constraints under a dynamic simulation environment. Third, a decentralised coordinated control scheme is implemented in the HPP environment using communication-resilient scenarios to address system vulnerabilities. Finally, the practicality of the N—MADRL approach is demonstrated in a Grid2Op dynamic simulation environment for optimal dispatch, energy reserve management, and frequency control. Results demonstrated on the IEEE 14 bus network show that compared to PPO and DDPG, N—MADRL achieves 42.10% and 61.40% higher efficiency for optimal dispatch, along with improvements of 68.30% and 74.48% in mitigating frequency deviations, respectively. The proposed approach outperforms existing methods under partially, fully, and randomly connected scenarios by effectively handling uncertainties, system intermittency, and communication resiliency. |
| format | Article |
| id | doaj-art-150c1f024a5a4dc3bd89f8dc6de2d1b3 |
| institution | OA Journals |
| issn | 1996-1073 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj-art-150c1f024a5a4dc3bd89f8dc6de2d1b32025-08-20T01:56:16ZengMDPI AGEnergies1996-10732025-05-011810266610.3390/en18102666Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power PlantsMuhammad Ikram0Daryoush Habibi1Asma Aziz2School of Engineering, Edith Cowan University, Joondalup, Perth, WA 6027, AustraliaSchool of Engineering, Edith Cowan University, Joondalup, Perth, WA 6027, AustraliaSchool of Engineering, Edith Cowan University, Joondalup, Perth, WA 6027, AustraliaInverter-based resources (IBRs) are becoming more prominent due to the increasing penetration of renewable energy sources that reduce power system inertia, compromising power system stability and grid support services. At present, optimal coordination among generation technologies remains a significant challenge for frequency control services. This paper presents a novel networked multi-agent deep reinforcement learning (N—MADRL) scheme for optimal dispatch and frequency control services. First, we develop a model-free environment consisting of a photovoltaic (PV) plant, a wind plant (WP), and an energy storage system (ESS) plant. The proposed framework uses a combination of multi-agent actor-critic (MAAC) and soft actor-critic (SAC) schemes for optimal dispatch of active power, mitigating frequency deviations, aiding reserve capacity management, and improving energy balancing. Second, frequency stability and optimal dispatch are formulated in the N—MADRL framework using the physical constraints under a dynamic simulation environment. Third, a decentralised coordinated control scheme is implemented in the HPP environment using communication-resilient scenarios to address system vulnerabilities. Finally, the practicality of the N—MADRL approach is demonstrated in a Grid2Op dynamic simulation environment for optimal dispatch, energy reserve management, and frequency control. Results demonstrated on the IEEE 14 bus network show that compared to PPO and DDPG, N—MADRL achieves 42.10% and 61.40% higher efficiency for optimal dispatch, along with improvements of 68.30% and 74.48% in mitigating frequency deviations, respectively. The proposed approach outperforms existing methods under partially, fully, and randomly connected scenarios by effectively handling uncertainties, system intermittency, and communication resiliency.https://www.mdpi.com/1996-1073/18/10/2666multi-agent deep reinforcement learningsoft actor–critichybrid power plantsoptimal dispatchancillary servicesfrequency control |
| spellingShingle | Muhammad Ikram Daryoush Habibi Asma Aziz Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants Energies multi-agent deep reinforcement learning soft actor–critic hybrid power plants optimal dispatch ancillary services frequency control |
| title | Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants |
| title_full | Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants |
| title_fullStr | Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants |
| title_full_unstemmed | Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants |
| title_short | Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants |
| title_sort | networked multi agent deep reinforcement learning framework for the provision of ancillary services in hybrid power plants |
| topic | multi-agent deep reinforcement learning soft actor–critic hybrid power plants optimal dispatch ancillary services frequency control |
| url | https://www.mdpi.com/1996-1073/18/10/2666 |
| work_keys_str_mv | AT muhammadikram networkedmultiagentdeepreinforcementlearningframeworkfortheprovisionofancillaryservicesinhybridpowerplants AT daryoushhabibi networkedmultiagentdeepreinforcementlearningframeworkfortheprovisionofancillaryservicesinhybridpowerplants AT asmaaziz networkedmultiagentdeepreinforcementlearningframeworkfortheprovisionofancillaryservicesinhybridpowerplants |