Improving Satellite Imagery Masking Using Multitask and Transfer Learning
Many remote sensing applications require masking of pixels in satellite imagery for further analysis. For instance, estimating water quality variables such as suspended sediment concentration (SSC) requires isolating pixels depicting water bodies unaffected by clouds, their shadows, terrain shadows,...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10925631/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850271643009548288 |
|---|---|
| author | Rangel Daroya Luisa Vieira Lucchese Travis Simmons Punwath Prum Tamlin Pavelsky John Gardner Colin J. Gleason Subhransu Maji |
| author_facet | Rangel Daroya Luisa Vieira Lucchese Travis Simmons Punwath Prum Tamlin Pavelsky John Gardner Colin J. Gleason Subhransu Maji |
| author_sort | Rangel Daroya |
| collection | DOAJ |
| description | Many remote sensing applications require masking of pixels in satellite imagery for further analysis. For instance, estimating water quality variables such as suspended sediment concentration (SSC) requires isolating pixels depicting water bodies unaffected by clouds, their shadows, terrain shadows, and snow and ice formation. A significant bottleneck is the reliance on multiple data products (e.g., satellite imagery and elevation maps) and lack of precision in individual processing steps, which degrade estimation accuracy. We propose a unified masking system that predicts all necessary masks from harmonized landsat and sentinel (HLS) imagery. Our model leverages multitask learning to improve accuracy while sharing computation across tasks for added efficiency. In this article, we explore recent deep learning architectures, demonstrating that masking performance benefits from pretraining on large satellite imagery datasets. We present a range of models offering different speed/accuracy tradeoffs: MobileNet variants provide the fastest inference while maintaining competitive accuracy, whereas transformer-based architectures achieve the highest accuracy, particularly when pretrained on large-scale satellite datasets. Our models provide a 9% <inline-formula><tex-math notation="LaTeX">$F1$</tex-math></inline-formula> score improvement compared to previous work on water pixel identification. When integrated with an SSC estimation system, our models result in a 30× speedup while reducing estimation error by 2.64 mg/L, allowing for global-scale analysis. We also evaluate our model on a recently proposed cloud and cloud shadow estimation benchmark, where we outperform the current state-of-the-art model by at least 6% in <inline-formula><tex-math notation="LaTeX">$F1$</tex-math></inline-formula> score. |
| format | Article |
| id | doaj-art-41fcf4df68a24df9a90756882c8d6719 |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-41fcf4df68a24df9a90756882c8d67192025-08-20T01:52:10ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01188777879610.1109/JSTARS.2025.355162010925631Improving Satellite Imagery Masking Using Multitask and Transfer LearningRangel Daroya0https://orcid.org/0009-0007-5309-6359Luisa Vieira Lucchese1https://orcid.org/0000-0002-8737-7171Travis Simmons2Punwath Prum3https://orcid.org/0009-0009-7085-7695Tamlin Pavelsky4John Gardner5https://orcid.org/0000-0002-1454-5074Colin J. Gleason6Subhransu Maji7https://orcid.org/0000-0002-3869-9334College of Information and Computer Sciences, University of Massachusetts, Amherst, MA, USADepartment of Geology and Environmental Science, University of Pittsburgh, Pittsburgh, PA, USADepartment of Civil and Environmental Engineering, University of Massachusetts, Amherst, MA, USADepartment of Geology and Environmental Science, University of Pittsburgh, Pittsburgh, PA, USADepartment of Earth, Marine and Environmental Science, University of North Carolina, Chapel Hill, USADepartment of Geology and Environmental Science, University of Pittsburgh, Pittsburgh, PA, USADepartment of Civil and Environmental Engineering, University of Massachusetts, Amherst, MA, USACollege of Information and Computer Sciences, University of Massachusetts, Amherst, MA, USAMany remote sensing applications require masking of pixels in satellite imagery for further analysis. For instance, estimating water quality variables such as suspended sediment concentration (SSC) requires isolating pixels depicting water bodies unaffected by clouds, their shadows, terrain shadows, and snow and ice formation. A significant bottleneck is the reliance on multiple data products (e.g., satellite imagery and elevation maps) and lack of precision in individual processing steps, which degrade estimation accuracy. We propose a unified masking system that predicts all necessary masks from harmonized landsat and sentinel (HLS) imagery. Our model leverages multitask learning to improve accuracy while sharing computation across tasks for added efficiency. In this article, we explore recent deep learning architectures, demonstrating that masking performance benefits from pretraining on large satellite imagery datasets. We present a range of models offering different speed/accuracy tradeoffs: MobileNet variants provide the fastest inference while maintaining competitive accuracy, whereas transformer-based architectures achieve the highest accuracy, particularly when pretrained on large-scale satellite datasets. Our models provide a 9% <inline-formula><tex-math notation="LaTeX">$F1$</tex-math></inline-formula> score improvement compared to previous work on water pixel identification. When integrated with an SSC estimation system, our models result in a 30× speedup while reducing estimation error by 2.64 mg/L, allowing for global-scale analysis. We also evaluate our model on a recently proposed cloud and cloud shadow estimation benchmark, where we outperform the current state-of-the-art model by at least 6% in <inline-formula><tex-math notation="LaTeX">$F1$</tex-math></inline-formula> score.https://ieeexplore.ieee.org/document/10925631/Deep learningglobal surface water detectionmultitask learningsuspended sedimenttransfer learning |
| spellingShingle | Rangel Daroya Luisa Vieira Lucchese Travis Simmons Punwath Prum Tamlin Pavelsky John Gardner Colin J. Gleason Subhransu Maji Improving Satellite Imagery Masking Using Multitask and Transfer Learning IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Deep learning global surface water detection multitask learning suspended sediment transfer learning |
| title | Improving Satellite Imagery Masking Using Multitask and Transfer Learning |
| title_full | Improving Satellite Imagery Masking Using Multitask and Transfer Learning |
| title_fullStr | Improving Satellite Imagery Masking Using Multitask and Transfer Learning |
| title_full_unstemmed | Improving Satellite Imagery Masking Using Multitask and Transfer Learning |
| title_short | Improving Satellite Imagery Masking Using Multitask and Transfer Learning |
| title_sort | improving satellite imagery masking using multitask and transfer learning |
| topic | Deep learning global surface water detection multitask learning suspended sediment transfer learning |
| url | https://ieeexplore.ieee.org/document/10925631/ |
| work_keys_str_mv | AT rangeldaroya improvingsatelliteimagerymaskingusingmultitaskandtransferlearning AT luisavieiralucchese improvingsatelliteimagerymaskingusingmultitaskandtransferlearning AT travissimmons improvingsatelliteimagerymaskingusingmultitaskandtransferlearning AT punwathprum improvingsatelliteimagerymaskingusingmultitaskandtransferlearning AT tamlinpavelsky improvingsatelliteimagerymaskingusingmultitaskandtransferlearning AT johngardner improvingsatelliteimagerymaskingusingmultitaskandtransferlearning AT colinjgleason improvingsatelliteimagerymaskingusingmultitaskandtransferlearning AT subhransumaji improvingsatelliteimagerymaskingusingmultitaskandtransferlearning |