Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late Integration
The accurate identification of rice fields through remote sensing is critical for agricultural monitoring and global food security. While optical and Synthetic Aperture Radar (SAR) data offer complementary advantages for crop mapping—spectral richness from optical imagery and all-weather capabilitie...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Agriculture |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-0472/15/7/706 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850184587577131008 |
|---|---|
| author | Chenyang He Jia Song Huiyao Xu |
| author_facet | Chenyang He Jia Song Huiyao Xu |
| author_sort | Chenyang He |
| collection | DOAJ |
| description | The accurate identification of rice fields through remote sensing is critical for agricultural monitoring and global food security. While optical and Synthetic Aperture Radar (SAR) data offer complementary advantages for crop mapping—spectral richness from optical imagery and all-weather capabilities from SAR—their integration remains challenging due to heterogeneous data characteristics and environmental variability. This study systematically evaluates three Transformer-based fusion strategies for rice identification: Early Fusion Transformer (EFT), Feature Fusion Transformer (FFT), and Decision Fusion Transformer (DFT), designed to integrate optical-SAR data at the input level, feature level, and decision level, respectively. Experiments conducted in Arkansas, USA—a major rice-producing region with complex agroclimatic conditions—demonstrate that EFT achieves superior performance, with an overall accuracy (OA) of 98.33% and rice-specific Intersection over Union (IoU_rice) of 83.47%, surpassing single-modality baselines (optical: IoU_rice = 75.78%; SAR: IoU_rice = 66.81%) and alternative fusion approaches. The model exhibits exceptional robustness in cloud-obstructed regions and diverse field patterns, effectively balancing precision (90.98%) and recall (90.35%). These results highlight the superiority of early-stage fusion in preserving complementary spectral–structural information, while revealing limitations of delayed integration strategies. Our work advances multi-modal remote sensing methodologies, offering a scalable framework for operational agricultural monitoring in challenging environments. |
| format | Article |
| id | doaj-art-b3f50cafb9384e13b7d236fd0d3c1b53 |
| institution | OA Journals |
| issn | 2077-0472 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Agriculture |
| spelling | doaj-art-b3f50cafb9384e13b7d236fd0d3c1b532025-08-20T02:17:00ZengMDPI AGAgriculture2077-04722025-03-0115770610.3390/agriculture15070706Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late IntegrationChenyang He0Jia Song1Huiyao Xu2State Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, ChinaState Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, ChinaState Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, ChinaThe accurate identification of rice fields through remote sensing is critical for agricultural monitoring and global food security. While optical and Synthetic Aperture Radar (SAR) data offer complementary advantages for crop mapping—spectral richness from optical imagery and all-weather capabilities from SAR—their integration remains challenging due to heterogeneous data characteristics and environmental variability. This study systematically evaluates three Transformer-based fusion strategies for rice identification: Early Fusion Transformer (EFT), Feature Fusion Transformer (FFT), and Decision Fusion Transformer (DFT), designed to integrate optical-SAR data at the input level, feature level, and decision level, respectively. Experiments conducted in Arkansas, USA—a major rice-producing region with complex agroclimatic conditions—demonstrate that EFT achieves superior performance, with an overall accuracy (OA) of 98.33% and rice-specific Intersection over Union (IoU_rice) of 83.47%, surpassing single-modality baselines (optical: IoU_rice = 75.78%; SAR: IoU_rice = 66.81%) and alternative fusion approaches. The model exhibits exceptional robustness in cloud-obstructed regions and diverse field patterns, effectively balancing precision (90.98%) and recall (90.35%). These results highlight the superiority of early-stage fusion in preserving complementary spectral–structural information, while revealing limitations of delayed integration strategies. Our work advances multi-modal remote sensing methodologies, offering a scalable framework for operational agricultural monitoring in challenging environments.https://www.mdpi.com/2077-0472/15/7/706fusiontransformerrice identificationopticalsynthetic aperture radar (SAR) |
| spellingShingle | Chenyang He Jia Song Huiyao Xu Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late Integration Agriculture fusion transformer rice identification optical synthetic aperture radar (SAR) |
| title | Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late Integration |
| title_full | Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late Integration |
| title_fullStr | Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late Integration |
| title_full_unstemmed | Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late Integration |
| title_short | Optical and SAR Data Fusion Based on Transformer for Rice Identification: A Comparative Analysis from Early to Late Integration |
| title_sort | optical and sar data fusion based on transformer for rice identification a comparative analysis from early to late integration |
| topic | fusion transformer rice identification optical synthetic aperture radar (SAR) |
| url | https://www.mdpi.com/2077-0472/15/7/706 |
| work_keys_str_mv | AT chenyanghe opticalandsardatafusionbasedontransformerforriceidentificationacomparativeanalysisfromearlytolateintegration AT jiasong opticalandsardatafusionbasedontransformerforriceidentificationacomparativeanalysisfromearlytolateintegration AT huiyaoxu opticalandsardatafusionbasedontransformerforriceidentificationacomparativeanalysisfromearlytolateintegration |