Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization Information
Synthetic aperture radar (SAR) ship classification is crucial for maritime surveillance. Most existing methods primarily focus on visual or polarimetric features, often constrained by a limited feature set and facing challenges in data diversity and multimodal information integration. This study int...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10925632/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850271632217604096 |
|---|---|
| author | Jinyue Chen Youming Wu Wei Dai Wenhui Diao Yang Li Xin Gao Xian Sun |
| author_facet | Jinyue Chen Youming Wu Wei Dai Wenhui Diao Yang Li Xin Gao Xian Sun |
| author_sort | Jinyue Chen |
| collection | DOAJ |
| description | Synthetic aperture radar (SAR) ship classification is crucial for maritime surveillance. Most existing methods primarily focus on visual or polarimetric features, often constrained by a limited feature set and facing challenges in data diversity and multimodal information integration. This study introduces a text-enhanced multimodal framework for SAR ship classification (TeMSC), an extensible and unified approach that integrates multimodal information related to SAR ships. It consists of text-form geometry information embedding, polarization and visual information embedding, and a multimodal prediction module. By incorporating ship geometry information in text format, TeMSC leverages text representation to enhance feature expressiveness, compensating for the limited discriminative power of traditional visual and polarization features, especially in low-resolution scenarios. TeMSC effectively processes complementary multimodal information through a multimodal prediction module, while avoiding the complexity associated with traditional decision-level feature fusion strategies. In addition, a classification token mechanism is introduced to streamline the classification process. Through a two-stage training strategy, TeMSC captures information across multiple SAR datasets, enhancing its generalization and adaptability. Extensive experiments on the FUSAR-Ship and OpenSARShip datasets demonstrate the superior performance of TeMSC and highlight the benefits of multimodal integration for SAR ship classification. TeMSC provides a foundation for future research on SAR-focused multimodal learning applications. |
| format | Article |
| id | doaj-art-0789ca224f874a4c8b9ea4e67e92c46a |
| institution | OA Journals |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-0789ca224f874a4c8b9ea4e67e92c46a2025-08-20T01:52:10ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01188659867110.1109/JSTARS.2025.355123910925632Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization InformationJinyue Chen0https://orcid.org/0009-0000-1821-9818Youming Wu1https://orcid.org/0000-0002-5927-364XWei Dai2https://orcid.org/0000-0002-9010-2101Wenhui Diao3https://orcid.org/0000-0002-3931-3974Yang Li4https://orcid.org/0000-0002-2479-2538Xin Gao5Xian Sun6https://orcid.org/0000-0002-0038-9816Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaSynthetic aperture radar (SAR) ship classification is crucial for maritime surveillance. Most existing methods primarily focus on visual or polarimetric features, often constrained by a limited feature set and facing challenges in data diversity and multimodal information integration. This study introduces a text-enhanced multimodal framework for SAR ship classification (TeMSC), an extensible and unified approach that integrates multimodal information related to SAR ships. It consists of text-form geometry information embedding, polarization and visual information embedding, and a multimodal prediction module. By incorporating ship geometry information in text format, TeMSC leverages text representation to enhance feature expressiveness, compensating for the limited discriminative power of traditional visual and polarization features, especially in low-resolution scenarios. TeMSC effectively processes complementary multimodal information through a multimodal prediction module, while avoiding the complexity associated with traditional decision-level feature fusion strategies. In addition, a classification token mechanism is introduced to streamline the classification process. Through a two-stage training strategy, TeMSC captures information across multiple SAR datasets, enhancing its generalization and adaptability. Extensive experiments on the FUSAR-Ship and OpenSARShip datasets demonstrate the superior performance of TeMSC and highlight the benefits of multimodal integration for SAR ship classification. TeMSC provides a foundation for future research on SAR-focused multimodal learning applications.https://ieeexplore.ieee.org/document/10925632/Dual-polarizationgeometry information embeddingmultimodal learningship classificationsynthetic aperture radar (SAR) |
| spellingShingle | Jinyue Chen Youming Wu Wei Dai Wenhui Diao Yang Li Xin Gao Xian Sun Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization Information IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Dual-polarization geometry information embedding multimodal learning ship classification synthetic aperture radar (SAR) |
| title | Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization Information |
| title_full | Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization Information |
| title_fullStr | Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization Information |
| title_full_unstemmed | Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization Information |
| title_short | Text-Enhanced Multimodal Method for SAR Ship Classification With Geometry and Polarization Information |
| title_sort | text enhanced multimodal method for sar ship classification with geometry and polarization information |
| topic | Dual-polarization geometry information embedding multimodal learning ship classification synthetic aperture radar (SAR) |
| url | https://ieeexplore.ieee.org/document/10925632/ |
| work_keys_str_mv | AT jinyuechen textenhancedmultimodalmethodforsarshipclassificationwithgeometryandpolarizationinformation AT youmingwu textenhancedmultimodalmethodforsarshipclassificationwithgeometryandpolarizationinformation AT weidai textenhancedmultimodalmethodforsarshipclassificationwithgeometryandpolarizationinformation AT wenhuidiao textenhancedmultimodalmethodforsarshipclassificationwithgeometryandpolarizationinformation AT yangli textenhancedmultimodalmethodforsarshipclassificationwithgeometryandpolarizationinformation AT xingao textenhancedmultimodalmethodforsarshipclassificationwithgeometryandpolarizationinformation AT xiansun textenhancedmultimodalmethodforsarshipclassificationwithgeometryandpolarizationinformation |