Visual Prompt Learning of Foundation Models for Post-Disaster Damage Evaluation
In response to the urgent need for rapid and precise post-disaster damage evaluation, this study introduces the Visual Prompt Damage Evaluation (ViPDE) framework, a novel contrastive learning-based approach that leverages the embedded knowledge within the Segment Anything Model (SAM) and pairs of re...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-05-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/17/10/1664 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850256057951059968 |
|---|---|
| author | Fei Zhao Chengcui Zhang Runlin Zhang Tianyang Wang |
| author_facet | Fei Zhao Chengcui Zhang Runlin Zhang Tianyang Wang |
| author_sort | Fei Zhao |
| collection | DOAJ |
| description | In response to the urgent need for rapid and precise post-disaster damage evaluation, this study introduces the Visual Prompt Damage Evaluation (ViPDE) framework, a novel contrastive learning-based approach that leverages the embedded knowledge within the Segment Anything Model (SAM) and pairs of remote sensing images to enhance building damage assessment. In this framework, we propose a learnable cascaded Visual Prompt Generator (VPG) that provides semantic visual prompts, guiding SAM to effectively analyze pre- and post-disaster image pairs and construct a nuanced representation of the affected areas at different stages. By keeping the foundation model’s parameters frozen, ViPDE significantly enhances training efficiency compared with traditional full-model fine-tuning methods. This parameter-efficient approach reduces computational costs and accelerates deployment in emergency scenarios. Moreover, our model demonstrates robustness across diverse disaster types and geographic locations. Beyond mere binary assessments, our model distinguishes damage levels with a finer granularity, categorizing them on a scale from 1 (no damage) to 4 (destroyed). Extensive experiments validate the effectiveness of ViPDE, showcasing its superior performance over existing methods. Comparative evaluations demonstrate that ViPDE achieves an F1 score of 0.7014. This foundation model-based approach sets a new benchmark in disaster management. It also pioneers a new practical architectural paradigm for foundation model-based contrastive learning focused on specific objects of interest. |
| format | Article |
| id | doaj-art-e0045bc8c569431fb823490b1e45c5f5 |
| institution | OA Journals |
| issn | 2072-4292 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-e0045bc8c569431fb823490b1e45c5f52025-08-20T01:56:42ZengMDPI AGRemote Sensing2072-42922025-05-011710166410.3390/rs17101664Visual Prompt Learning of Foundation Models for Post-Disaster Damage EvaluationFei Zhao0Chengcui Zhang1Runlin Zhang2Tianyang Wang3Department of Computer Science, The University of Alabama at Birmingham, Birmingham, AL 35294, USADepartment of Computer Science, The University of Alabama at Birmingham, Birmingham, AL 35294, USADavid R. Cheriton School of Computer Science, University of Waterloo, Waterloo, ON N2L 3G1, CanadaDepartment of Computer Science, The University of Alabama at Birmingham, Birmingham, AL 35294, USAIn response to the urgent need for rapid and precise post-disaster damage evaluation, this study introduces the Visual Prompt Damage Evaluation (ViPDE) framework, a novel contrastive learning-based approach that leverages the embedded knowledge within the Segment Anything Model (SAM) and pairs of remote sensing images to enhance building damage assessment. In this framework, we propose a learnable cascaded Visual Prompt Generator (VPG) that provides semantic visual prompts, guiding SAM to effectively analyze pre- and post-disaster image pairs and construct a nuanced representation of the affected areas at different stages. By keeping the foundation model’s parameters frozen, ViPDE significantly enhances training efficiency compared with traditional full-model fine-tuning methods. This parameter-efficient approach reduces computational costs and accelerates deployment in emergency scenarios. Moreover, our model demonstrates robustness across diverse disaster types and geographic locations. Beyond mere binary assessments, our model distinguishes damage levels with a finer granularity, categorizing them on a scale from 1 (no damage) to 4 (destroyed). Extensive experiments validate the effectiveness of ViPDE, showcasing its superior performance over existing methods. Comparative evaluations demonstrate that ViPDE achieves an F1 score of 0.7014. This foundation model-based approach sets a new benchmark in disaster management. It also pioneers a new practical architectural paradigm for foundation model-based contrastive learning focused on specific objects of interest.https://www.mdpi.com/2072-4292/17/10/1664deep learningvisual prompt learningvision foundation modelsatellite imagerybuilding damage evaluationcontrastive learning |
| spellingShingle | Fei Zhao Chengcui Zhang Runlin Zhang Tianyang Wang Visual Prompt Learning of Foundation Models for Post-Disaster Damage Evaluation Remote Sensing deep learning visual prompt learning vision foundation model satellite imagery building damage evaluation contrastive learning |
| title | Visual Prompt Learning of Foundation Models for Post-Disaster Damage Evaluation |
| title_full | Visual Prompt Learning of Foundation Models for Post-Disaster Damage Evaluation |
| title_fullStr | Visual Prompt Learning of Foundation Models for Post-Disaster Damage Evaluation |
| title_full_unstemmed | Visual Prompt Learning of Foundation Models for Post-Disaster Damage Evaluation |
| title_short | Visual Prompt Learning of Foundation Models for Post-Disaster Damage Evaluation |
| title_sort | visual prompt learning of foundation models for post disaster damage evaluation |
| topic | deep learning visual prompt learning vision foundation model satellite imagery building damage evaluation contrastive learning |
| url | https://www.mdpi.com/2072-4292/17/10/1664 |
| work_keys_str_mv | AT feizhao visualpromptlearningoffoundationmodelsforpostdisasterdamageevaluation AT chengcuizhang visualpromptlearningoffoundationmodelsforpostdisasterdamageevaluation AT runlinzhang visualpromptlearningoffoundationmodelsforpostdisasterdamageevaluation AT tianyangwang visualpromptlearningoffoundationmodelsforpostdisasterdamageevaluation |