Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge
Traditional economic dispatch algorithms rely on the accuracy of all parameters and also lack the adaptability to the high uncertainties brought by the dynamic changes happening in the current power systems. Its computing efficiency also needs to be improved with the increased operational complexiti...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-08-01
|
| Series: | International Journal of Electrical Power & Energy Systems |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S0142061525002704 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849421913485475840 |
|---|---|
| author | Yanbo Chen Qintao Du Huayu Dong Tao Huang Jiahao Ma Zitao Xu Zhihao Wang |
| author_facet | Yanbo Chen Qintao Du Huayu Dong Tao Huang Jiahao Ma Zitao Xu Zhihao Wang |
| author_sort | Yanbo Chen |
| collection | DOAJ |
| description | Traditional economic dispatch algorithms rely on the accuracy of all parameters and also lack the adaptability to the high uncertainties brought by the dynamic changes happening in the current power systems. Its computing efficiency also needs to be improved with the increased operational complexities. In recent years, due to high self-learning and self-optimization ability, reinforcement learning has emerged in the field of economic dispatch, which can solve model-free dynamic programming problems that cannot be effectively solved by traditional optimization methods. In this paper, we construct a reinforcement agent for intra-day dispatch to optimize generator output, using a twin delayed deep deterministic policy gradient algorithm based on pre-training and expert knowledge (PEK-TD3). Aiming at solving the problems of long exploration time and poor convergence of conventional deep reinforcement learning, we propose an initial policy network training method based on pre-training with supervised learning, which significantly speeds up the training process of deep reinforcement learning and greatly reduces the model development cycle. At the same time, expert knowledge is embedded in the deep reinforcement learning to guide the training of the agent. With the guidance of expert knowledge, on the one hand, the agent quickly learns to limit the search direction to the feasible region of the power system operation so as to improve the convergence. On the other hand, in order to obtain higher rewards, agent learns to prioritize the renewable energy utilization which significantly reduces the curtailment rate of renewable energy. Finally, the modify IEEE 118-node system is used to verify the performance of the proposed method. |
| format | Article |
| id | doaj-art-e0398032dfea4797a941a48421e6ee96 |
| institution | Kabale University |
| issn | 0142-0615 |
| language | English |
| publishDate | 2025-08-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Electrical Power & Energy Systems |
| spelling | doaj-art-e0398032dfea4797a941a48421e6ee962025-08-20T03:31:20ZengElsevierInternational Journal of Electrical Power & Energy Systems0142-06152025-08-0116911071910.1016/j.ijepes.2025.110719Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledgeYanbo Chen0Qintao Du1Huayu Dong2Tao Huang3Jiahao Ma4Zitao Xu5Zhihao Wang6The State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources and School of Electrical & Electronic Engineering, North China Electric Power University, 102206 Beijing, China; Corresponding author.The Key Laboratory of Control of Power Transmission and Conversion, Ministry of Education, and Shanghai Non-Carbon Energy Conversion and Utilization Institute, Shanghai Jiao Tong University, Shanghai 200240, ChinaThe State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources and School of Electrical & Electronic Engineering, North China Electric Power University, 102206 Beijing, ChinaThe State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources and School of Electrical & Electronic Engineering, North China Electric Power University, 102206 Beijing, ChinaThe State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources and School of Electrical & Electronic Engineering, North China Electric Power University, 102206 Beijing, ChinaThe State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources and School of Electrical & Electronic Engineering, North China Electric Power University, 102206 Beijing, ChinaThe State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources and School of Electrical & Electronic Engineering, North China Electric Power University, 102206 Beijing, ChinaTraditional economic dispatch algorithms rely on the accuracy of all parameters and also lack the adaptability to the high uncertainties brought by the dynamic changes happening in the current power systems. Its computing efficiency also needs to be improved with the increased operational complexities. In recent years, due to high self-learning and self-optimization ability, reinforcement learning has emerged in the field of economic dispatch, which can solve model-free dynamic programming problems that cannot be effectively solved by traditional optimization methods. In this paper, we construct a reinforcement agent for intra-day dispatch to optimize generator output, using a twin delayed deep deterministic policy gradient algorithm based on pre-training and expert knowledge (PEK-TD3). Aiming at solving the problems of long exploration time and poor convergence of conventional deep reinforcement learning, we propose an initial policy network training method based on pre-training with supervised learning, which significantly speeds up the training process of deep reinforcement learning and greatly reduces the model development cycle. At the same time, expert knowledge is embedded in the deep reinforcement learning to guide the training of the agent. With the guidance of expert knowledge, on the one hand, the agent quickly learns to limit the search direction to the feasible region of the power system operation so as to improve the convergence. On the other hand, in order to obtain higher rewards, agent learns to prioritize the renewable energy utilization which significantly reduces the curtailment rate of renewable energy. Finally, the modify IEEE 118-node system is used to verify the performance of the proposed method.http://www.sciencedirect.com/science/article/pii/S0142061525002704Intra-day dispatchSL–TD3Expert knowledgePre-training with supervised learningRenewable energy utilization |
| spellingShingle | Yanbo Chen Qintao Du Huayu Dong Tao Huang Jiahao Ma Zitao Xu Zhihao Wang Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge International Journal of Electrical Power & Energy Systems Intra-day dispatch SL–TD3 Expert knowledge Pre-training with supervised learning Renewable energy utilization |
| title | Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge |
| title_full | Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge |
| title_fullStr | Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge |
| title_full_unstemmed | Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge |
| title_short | Intra-day dispatch method via deep reinforcement learning based on pre-training and expert knowledge |
| title_sort | intra day dispatch method via deep reinforcement learning based on pre training and expert knowledge |
| topic | Intra-day dispatch SL–TD3 Expert knowledge Pre-training with supervised learning Renewable energy utilization |
| url | http://www.sciencedirect.com/science/article/pii/S0142061525002704 |
| work_keys_str_mv | AT yanbochen intradaydispatchmethodviadeepreinforcementlearningbasedonpretrainingandexpertknowledge AT qintaodu intradaydispatchmethodviadeepreinforcementlearningbasedonpretrainingandexpertknowledge AT huayudong intradaydispatchmethodviadeepreinforcementlearningbasedonpretrainingandexpertknowledge AT taohuang intradaydispatchmethodviadeepreinforcementlearningbasedonpretrainingandexpertknowledge AT jiahaoma intradaydispatchmethodviadeepreinforcementlearningbasedonpretrainingandexpertknowledge AT zitaoxu intradaydispatchmethodviadeepreinforcementlearningbasedonpretrainingandexpertknowledge AT zhihaowang intradaydispatchmethodviadeepreinforcementlearningbasedonpretrainingandexpertknowledge |