A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) dataset
The NASA Harmonized Landsat Sentinel-2 (HLS) data provides global coverage atmospherically corrected surface reflectance with a 30m cloud and cloud shadow mask derived using the Fmask algorithm applied to top-of-atmosphere (TOA) reflectance. In this study we demonstrate, as have other researchers, l...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-06-01
|
| Series: | Science of Remote Sensing |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666017225000197 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849329203633192960 |
|---|---|
| author | Haiyan Huang David P. Roy Hugo De Lemos Yuean Qiu Hankui K. Zhang |
| author_facet | Haiyan Huang David P. Roy Hugo De Lemos Yuean Qiu Hankui K. Zhang |
| author_sort | Haiyan Huang |
| collection | DOAJ |
| description | The NASA Harmonized Landsat Sentinel-2 (HLS) data provides global coverage atmospherically corrected surface reflectance with a 30m cloud and cloud shadow mask derived using the Fmask algorithm applied to top-of-atmosphere (TOA) reflectance. In this study we demonstrate, as have other researchers, low Sentinel-2 Fmask performance, and present a solution that applies a deep learning Swin-Unet model to the HLS surface reflectance to provide unambiguously improved cloud and cloud shadow detection. The model was trained and assessed using 30m HLS surface reflectance for the 13 Sentinel-2 bands and corresponding CloudSEN12+ annotations, that define cloud, thin cloud, clear, and cloud shadow, and is the largest publicly available expert annotation set. All the CloudSEN12 annotations with coincident HLS Sentinel-2 data were considered. A total of 8672 globally distributed 5 × 5 km data sets were used, 7362 to train the model, 464 for internal model validation, and 846 to independently assess the classification accuracy. The HLS Sentinel-2 Fmask had F1-scores of 0.832 (cloud), 0.546 (cloud shadow), and 0.873 (clear), and the Swin-Unet model had higher performance with F1-scores of 0.891 (cloud and thin cloud combined), 0.710 (cloud shadow), and 0.923 (clear) despite the use of surface and not TOA reflectance. The Swin-Unet thin cloud class had low accuracy (0.604 F1-score) likely due to atmospheric correction issues and thin cloud variability that are discussed. The comprehensively trained model provides a solution for users who wish to improve the HLS Sentinel-2 cloud and cloud shadow masking using the available HLS Sentinel-2 surface reflectance data. |
| format | Article |
| id | doaj-art-097bbeb74c29490dbe17c05cde85f604 |
| institution | Kabale University |
| issn | 2666-0172 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Science of Remote Sensing |
| spelling | doaj-art-097bbeb74c29490dbe17c05cde85f6042025-08-20T03:47:20ZengElsevierScience of Remote Sensing2666-01722025-06-011110021310.1016/j.srs.2025.100213A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) datasetHaiyan Huang0David P. Roy1Hugo De Lemos2Yuean Qiu3Hankui K. Zhang4Center for Global Change and Earth Observations, Michigan State University, East Lansing, MI, 48824, USACenter for Global Change and Earth Observations, Michigan State University, East Lansing, MI, 48824, USA; Department of Geography, Environment, & Spatial Sciences, Michigan State University, East Lansing, MI, 48824, USA; Corresponding author. Center for Global Change and Earth Observations, Michigan State University, East Lansing, MI, 48824, USA.Center for Global Change and Earth Observations, Michigan State University, East Lansing, MI, 48824, USACenter for Global Change and Earth Observations, Michigan State University, East Lansing, MI, 48824, USAGeospatial Sciences Center of Excellence, Department of Geography and Geospatial Sciences, South Dakota State University, Brookings, SD, 57007, USAThe NASA Harmonized Landsat Sentinel-2 (HLS) data provides global coverage atmospherically corrected surface reflectance with a 30m cloud and cloud shadow mask derived using the Fmask algorithm applied to top-of-atmosphere (TOA) reflectance. In this study we demonstrate, as have other researchers, low Sentinel-2 Fmask performance, and present a solution that applies a deep learning Swin-Unet model to the HLS surface reflectance to provide unambiguously improved cloud and cloud shadow detection. The model was trained and assessed using 30m HLS surface reflectance for the 13 Sentinel-2 bands and corresponding CloudSEN12+ annotations, that define cloud, thin cloud, clear, and cloud shadow, and is the largest publicly available expert annotation set. All the CloudSEN12 annotations with coincident HLS Sentinel-2 data were considered. A total of 8672 globally distributed 5 × 5 km data sets were used, 7362 to train the model, 464 for internal model validation, and 846 to independently assess the classification accuracy. The HLS Sentinel-2 Fmask had F1-scores of 0.832 (cloud), 0.546 (cloud shadow), and 0.873 (clear), and the Swin-Unet model had higher performance with F1-scores of 0.891 (cloud and thin cloud combined), 0.710 (cloud shadow), and 0.923 (clear) despite the use of surface and not TOA reflectance. The Swin-Unet thin cloud class had low accuracy (0.604 F1-score) likely due to atmospheric correction issues and thin cloud variability that are discussed. The comprehensively trained model provides a solution for users who wish to improve the HLS Sentinel-2 cloud and cloud shadow masking using the available HLS Sentinel-2 surface reflectance data.http://www.sciencedirect.com/science/article/pii/S2666017225000197CloudCloud shadowDeep learningHLSSentinel-2Swin-Unet |
| spellingShingle | Haiyan Huang David P. Roy Hugo De Lemos Yuean Qiu Hankui K. Zhang A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) dataset Science of Remote Sensing Cloud Cloud shadow Deep learning HLS Sentinel-2 Swin-Unet |
| title | A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) dataset |
| title_full | A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) dataset |
| title_fullStr | A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) dataset |
| title_full_unstemmed | A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) dataset |
| title_short | A global Swin-Unet Sentinel-2 surface reflectance-based cloud and cloud shadow detection algorithm for the NASA Harmonized Landsat Sentinel-2 (HLS) dataset |
| title_sort | global swin unet sentinel 2 surface reflectance based cloud and cloud shadow detection algorithm for the nasa harmonized landsat sentinel 2 hls dataset |
| topic | Cloud Cloud shadow Deep learning HLS Sentinel-2 Swin-Unet |
| url | http://www.sciencedirect.com/science/article/pii/S2666017225000197 |
| work_keys_str_mv | AT haiyanhuang aglobalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT davidproy aglobalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT hugodelemos aglobalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT yueanqiu aglobalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT hankuikzhang aglobalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT haiyanhuang globalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT davidproy globalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT hugodelemos globalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT yueanqiu globalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset AT hankuikzhang globalswinunetsentinel2surfacereflectancebasedcloudandcloudshadowdetectionalgorithmforthenasaharmonizedlandsatsentinel2hlsdataset |