Multi-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images
Abstract Hydrometeorological forecasting and early warning involve many hazardous elements, with the estimation of intensity and center location of tropical cyclones (TCs) being key. This paper proposes a new multitask deep learning model with attention gate mechanisms to work with satellite images...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-12733-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849388263738966016 |
|---|---|
| author | Thanh-Ha Do Son-The Phan Duc-Tien Du Dinh-Quan Dang Khanh-Hung Mai Lars R. Hole |
| author_facet | Thanh-Ha Do Son-The Phan Duc-Tien Du Dinh-Quan Dang Khanh-Hung Mai Lars R. Hole |
| author_sort | Thanh-Ha Do |
| collection | DOAJ |
| description | Abstract Hydrometeorological forecasting and early warning involve many hazardous elements, with the estimation of intensity and center location of tropical cyclones (TCs) being key. This paper proposes a new multitask deep learning model with attention gate mechanisms to work with satellite images and construct heatmaps for TC’s centering and classification. The multi-head keypoint design (MHKD) with the spatial attention mechanism (SAM) is fitted to the decoder layer using multi-resolution inputs from the encoder. In addition, the new loss function is employed with an Euclidean distance to guide centers of heatmaps from lower decoder layers toward higher ones, thereby refining keypoints during the early decoding stage. Experimental results, done on a constructed dataset for the Western North Pacific for 2015-2023 collected from the Japanese Himawari 8/9 geostationary satellite and the best track of the World Meteorological Organization (WMO) Regional Specialized Meteorological Center (RSMC) Tokyo - Typhoon Center, indicate that the proposed model successfully detects most TC existences on combined images from three infrared channels. The model’s accuracy can reach over 72% of the Tropical Depression (TD) grade and over 90% for really strong TCs (Severe Tropical Storm (STS) and Typhoon (TY)). Compared to a typical detecting object problem, the main issues come from the complexity of TC cloud patterns, which are nonlinear with actual TC grades or discrimination between TC grades (transition between TD to Tropical Storm (TS), TS to STS, and upgrading and progress of TCs). The proposed MHKD can help reduce the over-estimate rate for the TD grade and under-estimate rates for TS and STS grades, and most notably, the TC center localization yielded an average error of approximately 34 km with a single keypoint or one head attention network (One ATTN) and around 27 km when using three head attention network (Three ATTN). |
| format | Article |
| id | doaj-art-e94405b9201640b6a9f115e3e4ed12c0 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-e94405b9201640b6a9f115e3e4ed12c02025-08-20T03:42:22ZengNature PortfolioScientific Reports2045-23222025-07-0115111110.1038/s41598-025-12733-wMulti-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite imagesThanh-Ha Do0Son-The Phan1Duc-Tien Du2Dinh-Quan Dang3Khanh-Hung Mai4Lars R. Hole5Posts and Telecommunications Institute of TechnologyVNU University of ScienceNational Center for Hydro-Meteorological ForecastingNational Center for Hydro-Meteorological ForecastingNational Center for Hydro-Meteorological ForecastingNorwegian Meteorological InstituteAbstract Hydrometeorological forecasting and early warning involve many hazardous elements, with the estimation of intensity and center location of tropical cyclones (TCs) being key. This paper proposes a new multitask deep learning model with attention gate mechanisms to work with satellite images and construct heatmaps for TC’s centering and classification. The multi-head keypoint design (MHKD) with the spatial attention mechanism (SAM) is fitted to the decoder layer using multi-resolution inputs from the encoder. In addition, the new loss function is employed with an Euclidean distance to guide centers of heatmaps from lower decoder layers toward higher ones, thereby refining keypoints during the early decoding stage. Experimental results, done on a constructed dataset for the Western North Pacific for 2015-2023 collected from the Japanese Himawari 8/9 geostationary satellite and the best track of the World Meteorological Organization (WMO) Regional Specialized Meteorological Center (RSMC) Tokyo - Typhoon Center, indicate that the proposed model successfully detects most TC existences on combined images from three infrared channels. The model’s accuracy can reach over 72% of the Tropical Depression (TD) grade and over 90% for really strong TCs (Severe Tropical Storm (STS) and Typhoon (TY)). Compared to a typical detecting object problem, the main issues come from the complexity of TC cloud patterns, which are nonlinear with actual TC grades or discrimination between TC grades (transition between TD to Tropical Storm (TS), TS to STS, and upgrading and progress of TCs). The proposed MHKD can help reduce the over-estimate rate for the TD grade and under-estimate rates for TS and STS grades, and most notably, the TC center localization yielded an average error of approximately 34 km with a single keypoint or one head attention network (One ATTN) and around 27 km when using three head attention network (Three ATTN).https://doi.org/10.1038/s41598-025-12733-w |
| spellingShingle | Thanh-Ha Do Son-The Phan Duc-Tien Du Dinh-Quan Dang Khanh-Hung Mai Lars R. Hole Multi-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images Scientific Reports |
| title | Multi-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images |
| title_full | Multi-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images |
| title_fullStr | Multi-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images |
| title_full_unstemmed | Multi-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images |
| title_short | Multi-heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images |
| title_sort | multi heat keypoint incorporation in deep learning model to tropical cyclone centering and intensity classifying from geostationary satellite images |
| url | https://doi.org/10.1038/s41598-025-12733-w |
| work_keys_str_mv | AT thanhhado multiheatkeypointincorporationindeeplearningmodeltotropicalcyclonecenteringandintensityclassifyingfromgeostationarysatelliteimages AT sonthephan multiheatkeypointincorporationindeeplearningmodeltotropicalcyclonecenteringandintensityclassifyingfromgeostationarysatelliteimages AT ductiendu multiheatkeypointincorporationindeeplearningmodeltotropicalcyclonecenteringandintensityclassifyingfromgeostationarysatelliteimages AT dinhquandang multiheatkeypointincorporationindeeplearningmodeltotropicalcyclonecenteringandintensityclassifyingfromgeostationarysatelliteimages AT khanhhungmai multiheatkeypointincorporationindeeplearningmodeltotropicalcyclonecenteringandintensityclassifyingfromgeostationarysatelliteimages AT larsrhole multiheatkeypointincorporationindeeplearningmodeltotropicalcyclonecenteringandintensityclassifyingfromgeostationarysatelliteimages |