Multi-Objective-Based Multi-Heterogeneous- Agent Deep Reinforcement Learning for Minimization of Voltage Deviation and Operation Cost in Active Distribution System - Anurak Deanseekeaw; Watcharakorn Pinthurat; Boonruang Marungsri (IEEE Access)
The increasing penetration of renewable energy sources (RESs) has led to the proliferation of small-scale distributed energy resources (DERs) in modern power systems. Effective coordination of these DERs in active distribution systems benefits both utilities and consumers. This paper introduces a novel distributed active voltage and operation cost control (DAVOCC) framework designed to minimize node voltage deviations and operation costs. The proposed framework employs a multi-objective optimization approach, integrating three advanced algorithms: multi-agent proximal policy optimization (MAPPO), multi-agent asynchronous actor-critic (MAA2C), and multi-agent twin delayed deep deterministic policy gradient (MATD3). Battery energy storage systems (BESSs) and diesel generators (DGs) are used as heterogeneous agents, with their actions being constrained within predefined limits to ensure safe operation. The proposed framework is trained and tested on real-world data in a modified IEEE 33-node distribution system by featuring centralized training and decentralized execution (CTDE) framework. The obtained results demonstrate that three algorithms effectively maintain node voltage deviations within acceptable limits, with the MATD3-based algorithm achieving superior performance. Specifically, it delivers node voltage deviations close to nominal values, with an average deviation of 0.0042 p.u. and a standard deviation of 0.0065 p.u. Furthermore, the MATD3 algorithm reduces operational costs to 56,837.85 THB/day while generating the highest net profit of 725,943.71 THB/day from energy trading. These findings underscore the potential of the developed DAVOCC framework in optimizing the power management of BESSs and DGs, reducing the dependence on external grid energy and ensuring effective voltage regulation in active distribution systems.