Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System
The increasing penetration of renewable energy sources (RESs) has led to the proliferation of small-scale distributed energy resources (DERs) in modern power systems. Effective coordination of these DERs in active distribution systems benefits both utilities and consumers. This paper introduces a no...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10979328/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850192076157747200 |
|---|---|
| author | Anurak Deanseekeaw Watcharakorn Pinthurat Boonruang Marungsri |
| author_facet | Anurak Deanseekeaw Watcharakorn Pinthurat Boonruang Marungsri |
| author_sort | Anurak Deanseekeaw |
| collection | DOAJ |
| description | The increasing penetration of renewable energy sources (RESs) has led to the proliferation of small-scale distributed energy resources (DERs) in modern power systems. Effective coordination of these DERs in active distribution systems benefits both utilities and consumers. This paper introduces a novel distributed active voltage and operation cost control (DAVOCC) framework designed to minimize node voltage deviations and operation costs. The proposed framework employs a multi-objective optimization approach, integrating three advanced algorithms: multi-agent proximal policy optimization (MAPPO), multi-agent asynchronous actor-critic (MAA2C), and multi-agent twin delayed deep deterministic policy gradient (MATD3). Battery energy storage systems (BESSs) and diesel generators (DGs) are used as heterogeneous agents, with their actions being constrained within predefined limits to ensure safe operation. The proposed framework is trained and tested on real-world data in a modified IEEE 33-node distribution system by featuring centralized training and decentralized execution (CTDE) framework. The obtained results demonstrate that three algorithms effectively maintain node voltage deviations within acceptable limits, with the MATD3-based algorithm achieving superior performance. Specifically, it delivers node voltage deviations close to nominal values, with an average deviation of 0.0042 p.u. and a standard deviation of 0.0065 p.u. Furthermore, the MATD3 algorithm reduces operational costs to 56,837.85 THB/day while generating the highest net profit of 725,943.71 THB/day from energy trading. These findings underscore the potential of the developed DAVOCC framework in optimizing the power management of BESSs and DGs, reducing the dependence on external grid energy and ensuring effective voltage regulation in active distribution systems. |
| format | Article |
| id | doaj-art-9531fbc836e34ec0b990870ddcc0c106 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-9531fbc836e34ec0b990870ddcc0c1062025-08-20T02:14:42ZengIEEEIEEE Access2169-35362025-01-0113756487566510.1109/ACCESS.2025.356512310979328Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution SystemAnurak Deanseekeaw0https://orcid.org/0009-0006-8940-5136Watcharakorn Pinthurat1https://orcid.org/0000-0002-0976-5873Boonruang Marungsri2https://orcid.org/0000-0002-7086-2197School of Electrical Engineering, Suranaree University of Technology, Nakhon Ratchasima, ThailandDepartment of Electrical Engineering, Rajamangala University of Technology Tawan-ok, Chanthaburi, ThailandSchool of Electrical Engineering, Suranaree University of Technology, Nakhon Ratchasima, ThailandThe increasing penetration of renewable energy sources (RESs) has led to the proliferation of small-scale distributed energy resources (DERs) in modern power systems. Effective coordination of these DERs in active distribution systems benefits both utilities and consumers. This paper introduces a novel distributed active voltage and operation cost control (DAVOCC) framework designed to minimize node voltage deviations and operation costs. The proposed framework employs a multi-objective optimization approach, integrating three advanced algorithms: multi-agent proximal policy optimization (MAPPO), multi-agent asynchronous actor-critic (MAA2C), and multi-agent twin delayed deep deterministic policy gradient (MATD3). Battery energy storage systems (BESSs) and diesel generators (DGs) are used as heterogeneous agents, with their actions being constrained within predefined limits to ensure safe operation. The proposed framework is trained and tested on real-world data in a modified IEEE 33-node distribution system by featuring centralized training and decentralized execution (CTDE) framework. The obtained results demonstrate that three algorithms effectively maintain node voltage deviations within acceptable limits, with the MATD3-based algorithm achieving superior performance. Specifically, it delivers node voltage deviations close to nominal values, with an average deviation of 0.0042 p.u. and a standard deviation of 0.0065 p.u. Furthermore, the MATD3 algorithm reduces operational costs to 56,837.85 THB/day while generating the highest net profit of 725,943.71 THB/day from energy trading. These findings underscore the potential of the developed DAVOCC framework in optimizing the power management of BESSs and DGs, reducing the dependence on external grid energy and ensuring effective voltage regulation in active distribution systems.https://ieeexplore.ieee.org/document/10979328/Multi-agent deep reinforcement learningmicrogridvoltage controloperation cost reductionbattery energy storage systemdiesel generator |
| spellingShingle | Anurak Deanseekeaw Watcharakorn Pinthurat Boonruang Marungsri Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System IEEE Access Multi-agent deep reinforcement learning microgrid voltage control operation cost reduction battery energy storage system diesel generator |
| title | Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System |
| title_full | Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System |
| title_fullStr | Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System |
| title_full_unstemmed | Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System |
| title_short | Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System |
| title_sort | multi objective based multi heterogeneous agent deep reinforcement learning for minimization of voltage deviation and operation cost in active distribution system |
| topic | Multi-agent deep reinforcement learning microgrid voltage control operation cost reduction battery energy storage system diesel generator |
| url | https://ieeexplore.ieee.org/document/10979328/ |
| work_keys_str_mv | AT anurakdeanseekeaw multiobjectivebasedmultiheterogeneousagentdeepreinforcementlearningforminimizationofvoltagedeviationandoperationcostinactivedistributionsystem AT watcharakornpinthurat multiobjectivebasedmultiheterogeneousagentdeepreinforcementlearningforminimizationofvoltagedeviationandoperationcostinactivedistributionsystem AT boonruangmarungsri multiobjectivebasedmultiheterogeneousagentdeepreinforcementlearningforminimizationofvoltagedeviationandoperationcostinactivedistributionsystem |