Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image Captioning
Remote sensing image captioning aims to generate semantically accurate descriptions that are closely linked to the visual features of remote sensing images. Existing approaches typically emphasize fine-grained extraction of visual features and capturing global information. However, they often overlo...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11039674/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849422298965082112 |
|---|---|
| author | Maofu Liu Jiahui Liu Xiaokang Zhang |
| author_facet | Maofu Liu Jiahui Liu Xiaokang Zhang |
| author_sort | Maofu Liu |
| collection | DOAJ |
| description | Remote sensing image captioning aims to generate semantically accurate descriptions that are closely linked to the visual features of remote sensing images. Existing approaches typically emphasize fine-grained extraction of visual features and capturing global information. However, they often overlook the complementary role of textual information in enhancing visual semantics and face challenges in precisely locating objects that are most relevant to the image context. To address these challenges, this article presents a semantic–spatial feature fusion with dynamic graph refinement (SFDR) method, which integrates the semantic–spatial feature fusion (SSFF) and dynamic graph feature refinement (DGFR) modules. The SSFF module utilizes a multilevel feature representation strategy by leveraging pretrained CLIP features, grid features, and ROI features to integrate rich semantic and spatial information. In the DGFR module, a graph attention network captures the relationships between feature nodes, while a dynamic weighting mechanism prioritizes objects that are most relevant to the current scene and suppresses less significant ones. Therefore, the proposed SFDR method significantly enhances the quality of the generated descriptions. Experimental results on three benchmark datasets demonstrate the effectiveness of the proposed method. |
| format | Article |
| id | doaj-art-e0754bcfb3d64c7ba19db2ccf1ce9424 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-e0754bcfb3d64c7ba19db2ccf1ce94242025-08-20T03:31:10ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118154421545510.1109/JSTARS.2025.358068611039674Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image CaptioningMaofu Liu0https://orcid.org/0000-0002-3732-4354Jiahui Liu1https://orcid.org/0009-0008-9693-0734Xiaokang Zhang2https://orcid.org/0000-0002-6127-4801School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, ChinaSchool of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, ChinaSchool of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan, ChinaRemote sensing image captioning aims to generate semantically accurate descriptions that are closely linked to the visual features of remote sensing images. Existing approaches typically emphasize fine-grained extraction of visual features and capturing global information. However, they often overlook the complementary role of textual information in enhancing visual semantics and face challenges in precisely locating objects that are most relevant to the image context. To address these challenges, this article presents a semantic–spatial feature fusion with dynamic graph refinement (SFDR) method, which integrates the semantic–spatial feature fusion (SSFF) and dynamic graph feature refinement (DGFR) modules. The SSFF module utilizes a multilevel feature representation strategy by leveraging pretrained CLIP features, grid features, and ROI features to integrate rich semantic and spatial information. In the DGFR module, a graph attention network captures the relationships between feature nodes, while a dynamic weighting mechanism prioritizes objects that are most relevant to the current scene and suppresses less significant ones. Therefore, the proposed SFDR method significantly enhances the quality of the generated descriptions. Experimental results on three benchmark datasets demonstrate the effectiveness of the proposed method.https://ieeexplore.ieee.org/document/11039674/Dynamic weighting mechanismfeature fusiongraph attention (GAT)remote sensing image captioning |
| spellingShingle | Maofu Liu Jiahui Liu Xiaokang Zhang Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image Captioning IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Dynamic weighting mechanism feature fusion graph attention (GAT) remote sensing image captioning |
| title | Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image Captioning |
| title_full | Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image Captioning |
| title_fullStr | Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image Captioning |
| title_full_unstemmed | Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image Captioning |
| title_short | Semantic–Spatial Feature Fusion With Dynamic Graph Refinement for Remote Sensing Image Captioning |
| title_sort | semantic x2013 spatial feature fusion with dynamic graph refinement for remote sensing image captioning |
| topic | Dynamic weighting mechanism feature fusion graph attention (GAT) remote sensing image captioning |
| url | https://ieeexplore.ieee.org/document/11039674/ |
| work_keys_str_mv | AT maofuliu semanticx2013spatialfeaturefusionwithdynamicgraphrefinementforremotesensingimagecaptioning AT jiahuiliu semanticx2013spatialfeaturefusionwithdynamicgraphrefinementforremotesensingimagecaptioning AT xiaokangzhang semanticx2013spatialfeaturefusionwithdynamicgraphrefinementforremotesensingimagecaptioning |