Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases
The development of electricity spot markets necessitates more refined and accurate load forecasting capabilities to enable precise dispatch control and the creation of new trading products. Accurate load forecasting relies on high-quality historical load data, with complete load data serving as the...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/18/1/176 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549238916349952 |
|---|---|
| author | Jingjiao Li Yifan Lv Zhou Zhou Zhiwen Du Qiang Wei Ke Xu |
| author_facet | Jingjiao Li Yifan Lv Zhou Zhou Zhiwen Du Qiang Wei Ke Xu |
| author_sort | Jingjiao Li |
| collection | DOAJ |
| description | The development of electricity spot markets necessitates more refined and accurate load forecasting capabilities to enable precise dispatch control and the creation of new trading products. Accurate load forecasting relies on high-quality historical load data, with complete load data serving as the cornerstone for both forecasting and transactions in electricity spot markets. However, historical load data at the distribution network or user level often suffers from anomalies and missing values. Data-driven methods have been widely adopted for anomaly detection due to their independence from prior expert knowledge and precise physical models. Nevertheless, single network architectures struggle to adapt to the diverse load characteristics of distribution networks or users, hindering the effective capture of anomaly patterns. This paper proposes a PLS-VAE-BiLSTM-based method for anomaly identification and correction in load data by combining the strengths of Variational Autoencoders (VAE) and Bidirectional Long Short-Term Memory Networks (BiLSTM). This method begins with data preprocessing, including normalization and preliminary missing value imputation based on Partial Least Squares (PLS). Subsequently, a hybrid VAE-BiLSTM model is constructed and trained on a loaded dataset incorporating influencing factors to learn the relationships between different data features. Anomalies are identified and corrected by calculating the deviation between the model’s reconstructed values and the actual values. Finally, validation on both public and private datasets demonstrates that the PLS-VAE-BiLSTM model achieves average performance metrics of 98.44% precision, 94% recall rate, and 96.05% F1 score. Compared with VAE-LSTM, PSO-PFCM, and WTRR models, the proposed method exhibits superior overall anomaly detection performance. |
| format | Article |
| id | doaj-art-f11308f4a33940f9ac8822eec1c493c4 |
| institution | Kabale University |
| issn | 1996-1073 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj-art-f11308f4a33940f9ac8822eec1c493c42025-01-10T13:17:19ZengMDPI AGEnergies1996-10732025-01-0118117610.3390/en18010176Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market DatabasesJingjiao Li0Yifan Lv1Zhou Zhou2Zhiwen Du3Qiang Wei4Ke Xu5School of Electric Power Engineering, Nanjing Institute of Technology, Nanjing 211167, ChinaSchool of Electric Power Engineering, Nanjing Institute of Technology, Nanjing 211167, ChinaSchool of Electric Power Engineering, Nanjing Institute of Technology, Nanjing 211167, ChinaSchool of Electric Power Engineering, Nanjing Institute of Technology, Nanjing 211167, ChinaSchool of Electric Power Engineering, Nanjing Institute of Technology, Nanjing 211167, ChinaSchool of Electric Power Engineering, Nanjing Institute of Technology, Nanjing 211167, ChinaThe development of electricity spot markets necessitates more refined and accurate load forecasting capabilities to enable precise dispatch control and the creation of new trading products. Accurate load forecasting relies on high-quality historical load data, with complete load data serving as the cornerstone for both forecasting and transactions in electricity spot markets. However, historical load data at the distribution network or user level often suffers from anomalies and missing values. Data-driven methods have been widely adopted for anomaly detection due to their independence from prior expert knowledge and precise physical models. Nevertheless, single network architectures struggle to adapt to the diverse load characteristics of distribution networks or users, hindering the effective capture of anomaly patterns. This paper proposes a PLS-VAE-BiLSTM-based method for anomaly identification and correction in load data by combining the strengths of Variational Autoencoders (VAE) and Bidirectional Long Short-Term Memory Networks (BiLSTM). This method begins with data preprocessing, including normalization and preliminary missing value imputation based on Partial Least Squares (PLS). Subsequently, a hybrid VAE-BiLSTM model is constructed and trained on a loaded dataset incorporating influencing factors to learn the relationships between different data features. Anomalies are identified and corrected by calculating the deviation between the model’s reconstructed values and the actual values. Finally, validation on both public and private datasets demonstrates that the PLS-VAE-BiLSTM model achieves average performance metrics of 98.44% precision, 94% recall rate, and 96.05% F1 score. Compared with VAE-LSTM, PSO-PFCM, and WTRR models, the proposed method exhibits superior overall anomaly detection performance.https://www.mdpi.com/1996-1073/18/1/176anomaly identification and correctionbidirectional long short-term memory networkpower load datapartial least squarevariational auto-encoders |
| spellingShingle | Jingjiao Li Yifan Lv Zhou Zhou Zhiwen Du Qiang Wei Ke Xu Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases Energies anomaly identification and correction bidirectional long short-term memory network power load data partial least square variational auto-encoders |
| title | Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases |
| title_full | Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases |
| title_fullStr | Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases |
| title_full_unstemmed | Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases |
| title_short | Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases |
| title_sort | identification and correction of abnormal incomplete power load data in electricity spot market databases |
| topic | anomaly identification and correction bidirectional long short-term memory network power load data partial least square variational auto-encoders |
| url | https://www.mdpi.com/1996-1073/18/1/176 |
| work_keys_str_mv | AT jingjiaoli identificationandcorrectionofabnormalincompletepowerloaddatainelectricityspotmarketdatabases AT yifanlv identificationandcorrectionofabnormalincompletepowerloaddatainelectricityspotmarketdatabases AT zhouzhou identificationandcorrectionofabnormalincompletepowerloaddatainelectricityspotmarketdatabases AT zhiwendu identificationandcorrectionofabnormalincompletepowerloaddatainelectricityspotmarketdatabases AT qiangwei identificationandcorrectionofabnormalincompletepowerloaddatainelectricityspotmarketdatabases AT kexu identificationandcorrectionofabnormalincompletepowerloaddatainelectricityspotmarketdatabases |