Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysis
IntroductionClimate change isone of the major challenges facing the world today, causing frequent extreme weather events that significantly impact human production, life, and the ecological environment. Traditional climate prediction models largely rely on the simulation of physical processes. While...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Environmental Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fenvs.2024.1464241/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832591657744203776 |
|---|---|
| author | Shangke Liu Ke Liu Zheng Wang Yuanyuan Liu Bin Bai Rui Zhao |
| author_facet | Shangke Liu Ke Liu Zheng Wang Yuanyuan Liu Bin Bai Rui Zhao |
| author_sort | Shangke Liu |
| collection | DOAJ |
| description | IntroductionClimate change isone of the major challenges facing the world today, causing frequent extreme weather events that significantly impact human production, life, and the ecological environment. Traditional climate prediction models largely rely on the simulation of physical processes. While they have achieved some success, these models still face issues such as complexity, high computational cost, and insufficient handling of multivariable nonlinear relationships.MethodsIn light of this, this paper proposes a hybrid deep learning model based on Transformer-Convolutional Neural Network (CNN)-Long Short-Term Memory (LSTM) to improve the accuracy of climate predictions. Firstly, the Transformer model is introduced to capture the complex patterns in cimate data time series through its powerful sequence modeling capabilities. Secondly, CNN is utilized to extract local features and capture short-term changes. Lastly, LSTM is adept at handling long-term dependencies, ensuring the model can remember and utilize information over extended time spans.Results and DiscussionExperiments conducted on temperature data from Guangdong Province in China validate the performance of the proposed model. Compared to four different climate prediction decomposition methods, the proposed hybrid model with the Transformer method performs the best. The resuts also show that the Transformer-CNN-LSTM hybrid model outperforms other hybrid models on five evaluation metrics, indicating that the proposed model provides more accurate predictions and more stable fitting results. |
| format | Article |
| id | doaj-art-9d4779daf70e42ddb044c47f8ac81d3f |
| institution | Kabale University |
| issn | 2296-665X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Environmental Science |
| spelling | doaj-art-9d4779daf70e42ddb044c47f8ac81d3f2025-01-22T07:14:29ZengFrontiers Media S.A.Frontiers in Environmental Science2296-665X2025-01-011210.3389/fenvs.2024.14642411464241Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysisShangke Liu0Ke Liu1Zheng Wang2Yuanyuan Liu3Bin Bai4Rui Zhao5State Grid Ningxia Electric Power Co. Ltd., Eco-tech Research Institute, Yinchuan, Ningxia, ChinaShandong Chengxin Engineering Construction & Consulting Co. Ltd., Jinan, Shangdong, ChinaState Grid Ningxia Electric Power Co. Ltd., Eco-tech Research Institute, Yinchuan, Ningxia, ChinaState Grid Ningxia Electric Power Co. Ltd., Eco-tech Research Institute, Yinchuan, Ningxia, ChinaState Grid Ningxia Electric Power Co. Ltd., Eco-tech Research Institute, Yinchuan, Ningxia, ChinaState Grid Ningxia Electric Power Co. Ltd., Eco-tech Research Institute, Yinchuan, Ningxia, ChinaIntroductionClimate change isone of the major challenges facing the world today, causing frequent extreme weather events that significantly impact human production, life, and the ecological environment. Traditional climate prediction models largely rely on the simulation of physical processes. While they have achieved some success, these models still face issues such as complexity, high computational cost, and insufficient handling of multivariable nonlinear relationships.MethodsIn light of this, this paper proposes a hybrid deep learning model based on Transformer-Convolutional Neural Network (CNN)-Long Short-Term Memory (LSTM) to improve the accuracy of climate predictions. Firstly, the Transformer model is introduced to capture the complex patterns in cimate data time series through its powerful sequence modeling capabilities. Secondly, CNN is utilized to extract local features and capture short-term changes. Lastly, LSTM is adept at handling long-term dependencies, ensuring the model can remember and utilize information over extended time spans.Results and DiscussionExperiments conducted on temperature data from Guangdong Province in China validate the performance of the proposed model. Compared to four different climate prediction decomposition methods, the proposed hybrid model with the Transformer method performs the best. The resuts also show that the Transformer-CNN-LSTM hybrid model outperforms other hybrid models on five evaluation metrics, indicating that the proposed model provides more accurate predictions and more stable fitting results.https://www.frontiersin.org/articles/10.3389/fenvs.2024.1464241/fullclimate predictionsequentiallyhybrid deep learningCNNLSTMtransformer |
| spellingShingle | Shangke Liu Ke Liu Zheng Wang Yuanyuan Liu Bin Bai Rui Zhao Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysis Frontiers in Environmental Science climate prediction sequentially hybrid deep learning CNN LSTM transformer |
| title | Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysis |
| title_full | Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysis |
| title_fullStr | Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysis |
| title_full_unstemmed | Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysis |
| title_short | Investigation of a transformer-based hybrid artificial neural networks for climate data prediction and analysis |
| title_sort | investigation of a transformer based hybrid artificial neural networks for climate data prediction and analysis |
| topic | climate prediction sequentially hybrid deep learning CNN LSTM transformer |
| url | https://www.frontiersin.org/articles/10.3389/fenvs.2024.1464241/full |
| work_keys_str_mv | AT shangkeliu investigationofatransformerbasedhybridartificialneuralnetworksforclimatedatapredictionandanalysis AT keliu investigationofatransformerbasedhybridartificialneuralnetworksforclimatedatapredictionandanalysis AT zhengwang investigationofatransformerbasedhybridartificialneuralnetworksforclimatedatapredictionandanalysis AT yuanyuanliu investigationofatransformerbasedhybridartificialneuralnetworksforclimatedatapredictionandanalysis AT binbai investigationofatransformerbasedhybridartificialneuralnetworksforclimatedatapredictionandanalysis AT ruizhao investigationofatransformerbasedhybridartificialneuralnetworksforclimatedatapredictionandanalysis |