Three-Stage Bidding Strategy of Generation Company Based on Double Deep Q-Network under Incomplete Information Condition - Pengpeng YANG; Beibei WANG; Peng XU; Gaoqin WANG; Yaxian ZHENG (Zhongguo dianli)
In power market with incomplete information, a generation company only knows its own relevant information, while biddings of other market members and market environment may affect the market clearing result, which impacts the generation company’s revenue, so its bidding strategy should consider multi-dimensional market information. On the basis of deep learning reinforcement method, this paper proposes a framework based on the multi-agent DDQN (Double Deep Q-Network) algorithm to simulate the bidding strategy of generation company in the spot market. Firstly, the elements of the Markov Decision Process and action-value function in the model is defined. Secondly, the framework of the generator’s double deep Q network is established and the ε-greedy algorithm and Experience Replay Memory is adopted to train the neural network. The proposed model can make decisions based on multi-dimensional continuous states such as the market clearing price and load levels. Finally, a PJM 5-bus test case is used to compare the rewards obtained by DDQN and traditional Q-learning algorithm. The results shows that the DDQN algorithm can make appropriate decisions according to the complex state while the Q-learning algorithm has poor performance. This paper also analyzes the effectiveness of the generation company’s adoption of the DDQN algorithm for generating market strategy in terms of selection of different state vector, network generalization ability and adaptability to larger-scale calculation examples.