Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning Methods
The 3-D reconstruction of complex urban areas is becoming increasingly important for various applications. To achieve precise and complete 3-D reconstruction, current approaches aim to combine aerial and terrestrial images. The main challenge is achieving reliable feature matching of aerial and terr...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10976566/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850092017933090816 |
|---|---|
| author | Hui Wang Jiangxue Yu San Jiang Dejin Zhang Qingquan Li |
| author_facet | Hui Wang Jiangxue Yu San Jiang Dejin Zhang Qingquan Li |
| author_sort | Hui Wang |
| collection | DOAJ |
| description | The 3-D reconstruction of complex urban areas is becoming increasingly important for various applications. To achieve precise and complete 3-D reconstruction, current approaches aim to combine aerial and terrestrial images. The main challenge is achieving reliable feature matching of aerial and terrestrial images under large viewing angles and varying scene illuminations. Traditional handcrafted methods experience a significant decline in matching performance. In this context, deep-learning-based feature matching methods have developed rapidly and gained extensive attention. However, their performance in handling challenging large-angle aerial–terrestrial datasets still needs to be evaluated. To assess their performance for aerial–terrestrial images, this study has reviewed and evaluated four types of recent deep-learning-based feature matching networks and selected four sets of aerial–terrestrial datasets for experimental tests. Extensive experiments and evaluations have been conducted in terms of feature matching and image orientation based on structure from motion (SfM). The results demonstrate that graph-neural-network-based methods and detector-free methods exhibit significant advantages in feature matching of aerial–terrestrial datasets, which can generate effective and correct matches for aerial–terrestrial images with large-scale and viewpoint differences. In particular, the combination of SuperPoint and LightGlue achieves the best performance, which can generate approximately ten times the number of aerial–terrestrial feature matches when compared with scale invariant feature transform (SIFT). In addition, all images can be registered in SfM reconstruction using its matching results. However, the precision of deep-learning-based methods is still inferior to the classical handcrafted method in SfM reconstruction. Thus, there is still significant room for improvement to enhance their performance further. |
| format | Article |
| id | doaj-art-1492b64310fc4d1babfe67e8a651b55a |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-1492b64310fc4d1babfe67e8a651b55a2025-08-20T02:42:12ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118156881570610.1109/JSTARS.2025.356432610976566Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning MethodsHui Wang0Jiangxue Yu1San Jiang2https://orcid.org/0000-0002-7799-650XDejin Zhang3https://orcid.org/0000-0002-7423-2328Qingquan Li4https://orcid.org/0000-0002-2438-6046School of Computer Science, China University of Geosciences, Wuhan, ChinaSchool of Computer Science, China University of Geosciences, Wuhan, ChinaGuangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen, ChinaGuangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen, ChinaGuangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen, ChinaThe 3-D reconstruction of complex urban areas is becoming increasingly important for various applications. To achieve precise and complete 3-D reconstruction, current approaches aim to combine aerial and terrestrial images. The main challenge is achieving reliable feature matching of aerial and terrestrial images under large viewing angles and varying scene illuminations. Traditional handcrafted methods experience a significant decline in matching performance. In this context, deep-learning-based feature matching methods have developed rapidly and gained extensive attention. However, their performance in handling challenging large-angle aerial–terrestrial datasets still needs to be evaluated. To assess their performance for aerial–terrestrial images, this study has reviewed and evaluated four types of recent deep-learning-based feature matching networks and selected four sets of aerial–terrestrial datasets for experimental tests. Extensive experiments and evaluations have been conducted in terms of feature matching and image orientation based on structure from motion (SfM). The results demonstrate that graph-neural-network-based methods and detector-free methods exhibit significant advantages in feature matching of aerial–terrestrial datasets, which can generate effective and correct matches for aerial–terrestrial images with large-scale and viewpoint differences. In particular, the combination of SuperPoint and LightGlue achieves the best performance, which can generate approximately ten times the number of aerial–terrestrial feature matches when compared with scale invariant feature transform (SIFT). In addition, all images can be registered in SfM reconstruction using its matching results. However, the precision of deep-learning-based methods is still inferior to the classical handcrafted method in SfM reconstruction. Thus, there is still significant room for improvement to enhance their performance further.https://ieeexplore.ieee.org/document/10976566/Aerial–terrestrial imagesdeep learningdetector-based methodsdetector-free methodsfeature matchingstructure from motion (SfM) |
| spellingShingle | Hui Wang Jiangxue Yu San Jiang Dejin Zhang Qingquan Li Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning Methods IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Aerial–terrestrial images deep learning detector-based methods detector-free methods feature matching structure from motion (SfM) |
| title | Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning Methods |
| title_full | Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning Methods |
| title_fullStr | Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning Methods |
| title_full_unstemmed | Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning Methods |
| title_short | Aerial–Terrestrial Image Feature Matching: An Evaluation of Recent Deep Learning Methods |
| title_sort | aerial x2013 terrestrial image feature matching an evaluation of recent deep learning methods |
| topic | Aerial–terrestrial images deep learning detector-based methods detector-free methods feature matching structure from motion (SfM) |
| url | https://ieeexplore.ieee.org/document/10976566/ |
| work_keys_str_mv | AT huiwang aerialx2013terrestrialimagefeaturematchinganevaluationofrecentdeeplearningmethods AT jiangxueyu aerialx2013terrestrialimagefeaturematchinganevaluationofrecentdeeplearningmethods AT sanjiang aerialx2013terrestrialimagefeaturematchinganevaluationofrecentdeeplearningmethods AT dejinzhang aerialx2013terrestrialimagefeaturematchinganevaluationofrecentdeeplearningmethods AT qingquanli aerialx2013terrestrialimagefeaturematchinganevaluationofrecentdeeplearningmethods |