Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments
Mapping pavement types, especially in sidewalks, is essential for urban planning and mobility studies. Identifying pavement materials is a key factor in assessing mobility, such as walkability and wheelchair usability. However, satellite imagery in this scenario is limited, and in situ mapping can b...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | ISPRS International Journal of Geo-Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2220-9964/13/12/422 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850241299022610432 |
|---|---|
| author | Kauê de Moraes Vestena Silvana Phillipi Camboim Maria Antonia Brovelli Daniel Rodrigues dos Santos |
| author_facet | Kauê de Moraes Vestena Silvana Phillipi Camboim Maria Antonia Brovelli Daniel Rodrigues dos Santos |
| author_sort | Kauê de Moraes Vestena |
| collection | DOAJ |
| description | Mapping pavement types, especially in sidewalks, is essential for urban planning and mobility studies. Identifying pavement materials is a key factor in assessing mobility, such as walkability and wheelchair usability. However, satellite imagery in this scenario is limited, and in situ mapping can be costly. A promising solution is to extract such geospatial features from street-level imagery. This study explores using open-vocabulary classification algorithms to segment and identify pavement types and surface materials in this scenario. Our approach uses large language models (LLMs) to improve the accuracy of classifying different pavement types. The methodology involves two experiments: the first uses free prompting with random street-view images, employing Grounding Dino and SAM algorithms to assess performance across categories. The second experiment evaluates standardized pavement classification using the Deep Pavements dataset and a fine-tuned CLIP algorithm optimized for detecting OSM-compliant pavement categories. The study presents open resources, such as the Deep Pavements dataset and a fine-tuned CLIP-based model, demonstrating a significant improvement in the true positive rate (TPR) from 56.04% to 93.5%. Our findings highlight both the potential and limitations of current open-vocabulary algorithms and emphasize the importance of diverse training datasets. This study advances urban feature mapping by offering a more intuitive and accurate approach to geospatial data extraction, enhancing urban accessibility and mobility mapping. |
| format | Article |
| id | doaj-art-bcf09448cdf1465eb25a30d6d291897f |
| institution | OA Journals |
| issn | 2220-9964 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | ISPRS International Journal of Geo-Information |
| spelling | doaj-art-bcf09448cdf1465eb25a30d6d291897f2025-08-20T02:00:38ZengMDPI AGISPRS International Journal of Geo-Information2220-99642024-11-01131242210.3390/ijgi13120422Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban EnvironmentsKauê de Moraes Vestena0Silvana Phillipi Camboim1Maria Antonia Brovelli2Daniel Rodrigues dos Santos3Department of Geomatics, Federal University of Paraná, Curitiba 80060-000, BrazilDepartment of Geomatics, Federal University of Paraná, Curitiba 80060-000, BrazilPolitecnico di Milano, Department of Civil and Environmental Engineering, 20133 Milan, ItalyMilitary Institute of Engineering, Cartography Section, Rio de Janeiro 22290-270, BrazilMapping pavement types, especially in sidewalks, is essential for urban planning and mobility studies. Identifying pavement materials is a key factor in assessing mobility, such as walkability and wheelchair usability. However, satellite imagery in this scenario is limited, and in situ mapping can be costly. A promising solution is to extract such geospatial features from street-level imagery. This study explores using open-vocabulary classification algorithms to segment and identify pavement types and surface materials in this scenario. Our approach uses large language models (LLMs) to improve the accuracy of classifying different pavement types. The methodology involves two experiments: the first uses free prompting with random street-view images, employing Grounding Dino and SAM algorithms to assess performance across categories. The second experiment evaluates standardized pavement classification using the Deep Pavements dataset and a fine-tuned CLIP algorithm optimized for detecting OSM-compliant pavement categories. The study presents open resources, such as the Deep Pavements dataset and a fine-tuned CLIP-based model, demonstrating a significant improvement in the true positive rate (TPR) from 56.04% to 93.5%. Our findings highlight both the potential and limitations of current open-vocabulary algorithms and emphasize the importance of diverse training datasets. This study advances urban feature mapping by offering a more intuitive and accurate approach to geospatial data extraction, enhancing urban accessibility and mobility mapping.https://www.mdpi.com/2220-9964/13/12/422open-vocabulary algorithmspavement segmentationsurface material detectionstreet-view imageryurban mobility |
| spellingShingle | Kauê de Moraes Vestena Silvana Phillipi Camboim Maria Antonia Brovelli Daniel Rodrigues dos Santos Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments ISPRS International Journal of Geo-Information open-vocabulary algorithms pavement segmentation surface material detection street-view imagery urban mobility |
| title | Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments |
| title_full | Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments |
| title_fullStr | Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments |
| title_full_unstemmed | Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments |
| title_short | Investigating the Performance of Open-Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments |
| title_sort | investigating the performance of open vocabulary classification algorithms for pathway and surface material detection in urban environments |
| topic | open-vocabulary algorithms pavement segmentation surface material detection street-view imagery urban mobility |
| url | https://www.mdpi.com/2220-9964/13/12/422 |
| work_keys_str_mv | AT kauedemoraesvestena investigatingtheperformanceofopenvocabularyclassificationalgorithmsforpathwayandsurfacematerialdetectioninurbanenvironments AT silvanaphillipicamboim investigatingtheperformanceofopenvocabularyclassificationalgorithmsforpathwayandsurfacematerialdetectioninurbanenvironments AT mariaantoniabrovelli investigatingtheperformanceofopenvocabularyclassificationalgorithmsforpathwayandsurfacematerialdetectioninurbanenvironments AT danielrodriguesdossantos investigatingtheperformanceofopenvocabularyclassificationalgorithmsforpathwayandsurfacematerialdetectioninurbanenvironments |