Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition
In power market with incomplete information, a generation company only knows its own relevant information, while biddings of other market members and market environment may affect the market clearing result, which impacts the generation company’s revenue, so its bidding strategy should consider mult...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
State Grid Energy Research Institute
2021-11-01
|
| Series: | Zhongguo dianli |
| Subjects: | |
| Online Access: | https://www.electricpower.com.cn/CN/10.11930/j.issn.1004-9649.202103163 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850053179375353856 |
|---|---|
| author | Pengpeng YANG Beibei WANG Peng XU Gaoqin WANG Yaxian ZHENG |
| author_facet | Pengpeng YANG Beibei WANG Peng XU Gaoqin WANG Yaxian ZHENG |
| author_sort | Pengpeng YANG |
| collection | DOAJ |
| description | In power market with incomplete information, a generation company only knows its own relevant information, while biddings of other market members and market environment may affect the market clearing result, which impacts the generation company’s revenue, so its bidding strategy should consider multi-dimensional market information. On the basis of deep learning reinforcement method, this paper proposes a framework based on the multi-agent DDQN (Double Deep Q-Network) algorithm to simulate the bidding strategy of generation company in the spot market. Firstly, the elements of the Markov Decision Process and action-value function in the model is defined. Secondly, the framework of the generator’s double deep Q network is established and the ε-greedy algorithm and Experience Replay Memory is adopted to train the neural network. The proposed model can make decisions based on multi-dimensional continuous states such as the market clearing price and load levels. Finally, a PJM 5-bus test case is used to compare the rewards obtained by DDQN and traditional Q-learning algorithm. The results shows that the DDQN algorithm can make appropriate decisions according to the complex state while the Q-learning algorithm has poor performance. This paper also analyzes the effectiveness of the generation company’s adoption of the DDQN algorithm for generating market strategy in terms of selection of different state vector, network generalization ability and adaptability to larger-scale calculation examples. |
| format | Article |
| id | doaj-art-0a6ac5e41bef427b8bd3cb023d83b2e7 |
| institution | DOAJ |
| issn | 1004-9649 |
| language | zho |
| publishDate | 2021-11-01 |
| publisher | State Grid Energy Research Institute |
| record_format | Article |
| series | Zhongguo dianli |
| spelling | doaj-art-0a6ac5e41bef427b8bd3cb023d83b2e72025-08-20T02:52:37ZzhoState Grid Energy Research InstituteZhongguo dianli1004-96492021-11-015411475810.11930/j.issn.1004-9649.202103163zgdl-54-11-yangpengpengThree-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information ConditionPengpeng YANG0Beibei WANG1Peng XU2Gaoqin WANG3Yaxian ZHENG4School of Electrical Engineering, Southeast University, Nanjing 210096, ChinaSchool of Electrical Engineering, Southeast University, Nanjing 210096, ChinaSchool of Electrical Engineering, Southeast University, Nanjing 210096, ChinaChina Electric Power Research Institute, Nanjing 210003, ChinaChina Electric Power Research Institute, Nanjing 210003, ChinaIn power market with incomplete information, a generation company only knows its own relevant information, while biddings of other market members and market environment may affect the market clearing result, which impacts the generation company’s revenue, so its bidding strategy should consider multi-dimensional market information. On the basis of deep learning reinforcement method, this paper proposes a framework based on the multi-agent DDQN (Double Deep Q-Network) algorithm to simulate the bidding strategy of generation company in the spot market. Firstly, the elements of the Markov Decision Process and action-value function in the model is defined. Secondly, the framework of the generator’s double deep Q network is established and the ε-greedy algorithm and Experience Replay Memory is adopted to train the neural network. The proposed model can make decisions based on multi-dimensional continuous states such as the market clearing price and load levels. Finally, a PJM 5-bus test case is used to compare the rewards obtained by DDQN and traditional Q-learning algorithm. The results shows that the DDQN algorithm can make appropriate decisions according to the complex state while the Q-learning algorithm has poor performance. This paper also analyzes the effectiveness of the generation company’s adoption of the DDQN algorithm for generating market strategy in terms of selection of different state vector, network generalization ability and adaptability to larger-scale calculation examples.https://www.electricpower.com.cn/CN/10.11930/j.issn.1004-9649.202103163deep reinforcement learningbidding strategy of generatorthree stage quotation rulesddqn |
| spellingShingle | Pengpeng YANG Beibei WANG Peng XU Gaoqin WANG Yaxian ZHENG Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition Zhongguo dianli deep reinforcement learning bidding strategy of generator three stage quotation rules ddqn |
| title | Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition |
| title_full | Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition |
| title_fullStr | Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition |
| title_full_unstemmed | Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition |
| title_short | Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition |
| title_sort | three stage bidding strategy of generation company based on double deep q network under incomplete information condition |
| topic | deep reinforcement learning bidding strategy of generator three stage quotation rules ddqn |
| url | https://www.electricpower.com.cn/CN/10.11930/j.issn.1004-9649.202103163 |
| work_keys_str_mv | AT pengpengyang threestagebiddingstrategyofgenerationcompanybasedondoubledeepqnetworkunderincompleteinformationcondition AT beibeiwang threestagebiddingstrategyofgenerationcompanybasedondoubledeepqnetworkunderincompleteinformationcondition AT pengxu threestagebiddingstrategyofgenerationcompanybasedondoubledeepqnetworkunderincompleteinformationcondition AT gaoqinwang threestagebiddingstrategyofgenerationcompanybasedondoubledeepqnetworkunderincompleteinformationcondition AT yaxianzheng threestagebiddingstrategyofgenerationcompanybasedondoubledeepqnetworkunderincompleteinformationcondition |