A group scheduling algorithm for massive heterogeneous data in the “dual carbon” digital intelligence monitoring center considering time-varying characteristics and priorities - Wenni Kang; Dongge Zhu; Shuang Zhang; Jia Liu; Rui Ma (Sustainable Energy Research)
Abstract In view of the fact that the data of the “double carbon” digital intelligent monitoring center has the characteristics of constantly changing with time and that there are key tasks with high real-time requirements in the massive heterogeneous data, a “double carbon” that takes into account time-varying characteristics and priorities is proposed. The massive heterogeneous data grouping scheduling algorithm of the digital monitoring center schedules data according to the urgency of the task. The functional data analysis (FDA) method is used to convert the massive multi-source heterogeneous data of the “double carbon” digital intelligence monitoring center into continuous functions to solve the problem of frequency inconsistency and unify the data format; through the CNN–LSTM based on the Attention mechanism. The model extracts time-varying features from the data that eliminates heterogeneous characteristics, and implements data grouping in the “dual carbon” digital intelligence monitoring center; by setting differentiated priorities for different groups of data, it combines the data scheduling demand estimation model and delayed response time (RTT) factor and congestion factor, calculate the data priority-oriented data scheduling link similarity (DPLS), allocate the data to be scheduled to the scheduling link with the highest DPLS value for transmission, and realize the “double carbon” digital intelligence monitoring center data group scheduling. Experimental results show that this algorithm can unify heterogeneous data to the form of functional expression and improve data consistency. The absolute value of the Pearson correlation coefficient of data grouping reaches 0.953, and the grouping effect is good. High-priority data can be scheduled to the best transmission link to improve the efficiency and reliability of data transmission and realize the optimal allocation of resources.