FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing Images
Fine-grained retrieval of remote sensing images is an image interpretation task that is still in its infancy. With the rapid development of convolutional neural networks (CNN) in the field of remote sensing, it has become possible for remote sensing image retrieval tasks to move toward more fine-gra...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10904305/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849344633204637696 |
|---|---|
| author | Yong-Qiang Mao Zhizhuo Jiang Yu Liu Yiming Zhang Kehan Qi Hanbo Bi You He |
| author_facet | Yong-Qiang Mao Zhizhuo Jiang Yu Liu Yiming Zhang Kehan Qi Hanbo Bi You He |
| author_sort | Yong-Qiang Mao |
| collection | DOAJ |
| description | Fine-grained retrieval of remote sensing images is an image interpretation task that is still in its infancy. With the rapid development of convolutional neural networks (CNN) in the field of remote sensing, it has become possible for remote sensing image retrieval tasks to move toward more fine-grained classes. However, since current methods focus on how to construct similarity metrics between sample pairs, the model ignores the learning of fine-grained intraclass heterogeneity and interclass commonality features, which poses a huge challenge to fine-grained retrieval. To solve this problem, we propose a novel fine-grained retrieval framework of optical remote sensing (FRORS) images, which aims to improve fine-grained retrieval capabilities by constructing interaction and matching between intraclass heterogeneity features, interclass commonality features, and image features. Specifically, we first construct a fine-grained prototype memory (FPM) module, and continuously update the local prototype storage unit through a lightweight CNN to achieve a refined representation of fine-grained heterogeneity features. Furthermore, to learn interclass commonality, we propose a gram learning (GraL) strategy, which is achieved by learning the correlation between feature dimensions. On this basis, we introduce a gram-based metric match (GMM) mechanism, which fuses the prototype features representing intraclass heterogeneity and the gram vector representing interclass commonality through an embedding manner, thereby achieving the purpose of fully interactive matching between image features and fine-grained class features. With FPM, GraL, and GMM, our FRORS can better learn deep features representing fine-grained classes and promote the improvement of the network's fine-grained retrieval ability. Extensive experiments conducted on a self-constructed THUFG-OPT dataset prove that the proposed FRORS achieves state-of-the-art fine-grained retrieval performance, which is 5.75% higher than the baseline method on <inline-formula><tex-math notation="LaTeX">$\mathrm{mAP@10}$</tex-math></inline-formula>. |
| format | Article |
| id | doaj-art-55b5f33edc2946908e0a52329f7eecf5 |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-55b5f33edc2946908e0a52329f7eecf52025-08-20T03:42:37ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01187406741910.1109/JSTARS.2025.354582810904305FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing ImagesYong-Qiang Mao0https://orcid.org/0000-0001-9256-3668Zhizhuo Jiang1https://orcid.org/0000-0002-5269-2753Yu Liu2https://orcid.org/0000-0002-5216-3181Yiming Zhang3Kehan Qi4https://orcid.org/0009-0009-3344-3784Hanbo Bi5https://orcid.org/0009-0001-4209-5461You He6https://orcid.org/0000-0002-6111-340XDepartment of Electronic Engineering, Tsinghua University, Beijing, ChinaShenzhen International Graduate School, Tsinghua University, Shenzhen, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing, ChinaShenzhen International Graduate School, Tsinghua University, Shenzhen, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing, ChinaFine-grained retrieval of remote sensing images is an image interpretation task that is still in its infancy. With the rapid development of convolutional neural networks (CNN) in the field of remote sensing, it has become possible for remote sensing image retrieval tasks to move toward more fine-grained classes. However, since current methods focus on how to construct similarity metrics between sample pairs, the model ignores the learning of fine-grained intraclass heterogeneity and interclass commonality features, which poses a huge challenge to fine-grained retrieval. To solve this problem, we propose a novel fine-grained retrieval framework of optical remote sensing (FRORS) images, which aims to improve fine-grained retrieval capabilities by constructing interaction and matching between intraclass heterogeneity features, interclass commonality features, and image features. Specifically, we first construct a fine-grained prototype memory (FPM) module, and continuously update the local prototype storage unit through a lightweight CNN to achieve a refined representation of fine-grained heterogeneity features. Furthermore, to learn interclass commonality, we propose a gram learning (GraL) strategy, which is achieved by learning the correlation between feature dimensions. On this basis, we introduce a gram-based metric match (GMM) mechanism, which fuses the prototype features representing intraclass heterogeneity and the gram vector representing interclass commonality through an embedding manner, thereby achieving the purpose of fully interactive matching between image features and fine-grained class features. With FPM, GraL, and GMM, our FRORS can better learn deep features representing fine-grained classes and promote the improvement of the network's fine-grained retrieval ability. Extensive experiments conducted on a self-constructed THUFG-OPT dataset prove that the proposed FRORS achieves state-of-the-art fine-grained retrieval performance, which is 5.75% higher than the baseline method on <inline-formula><tex-math notation="LaTeX">$\mathrm{mAP@10}$</tex-math></inline-formula>.https://ieeexplore.ieee.org/document/10904305/Fine-grainedimage retrievaloptical imagesremote sensing |
| spellingShingle | Yong-Qiang Mao Zhizhuo Jiang Yu Liu Yiming Zhang Kehan Qi Hanbo Bi You He FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing Images IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Fine-grained image retrieval optical images remote sensing |
| title | FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing Images |
| title_full | FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing Images |
| title_fullStr | FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing Images |
| title_full_unstemmed | FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing Images |
| title_short | FRORS: An Effective Fine-Grained Retrieval Framework for Optical Remote Sensing Images |
| title_sort | frors an effective fine grained retrieval framework for optical remote sensing images |
| topic | Fine-grained image retrieval optical images remote sensing |
| url | https://ieeexplore.ieee.org/document/10904305/ |
| work_keys_str_mv | AT yongqiangmao frorsaneffectivefinegrainedretrievalframeworkforopticalremotesensingimages AT zhizhuojiang frorsaneffectivefinegrainedretrievalframeworkforopticalremotesensingimages AT yuliu frorsaneffectivefinegrainedretrievalframeworkforopticalremotesensingimages AT yimingzhang frorsaneffectivefinegrainedretrievalframeworkforopticalremotesensingimages AT kehanqi frorsaneffectivefinegrainedretrievalframeworkforopticalremotesensingimages AT hanbobi frorsaneffectivefinegrainedretrievalframeworkforopticalremotesensingimages AT youhe frorsaneffectivefinegrainedretrievalframeworkforopticalremotesensingimages |