<italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information
Window segmentation and vectorization remains a significant challenge, particularly in the absence of clean facade images. To extract complete window segments from building façade images with occlusions, this article proposes an occlusion-aware window segmentation <italic>(OA-WinSeg...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10923720/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849766863243837440 |
|---|---|
| author | Manuela F. Ceron-Viveros Wolfgang Maass Jiaojiao Tian |
| author_facet | Manuela F. Ceron-Viveros Wolfgang Maass Jiaojiao Tian |
| author_sort | Manuela F. Ceron-Viveros |
| collection | DOAJ |
| description | Window segmentation and vectorization remains a significant challenge, particularly in the absence of clean facade images. To extract complete window segments from building façade images with occlusions, this article proposes an occlusion-aware window segmentation <italic>(OA-WinSeg)</italic> network with conditional adversarial training guided by prior structural information. This architecture combines the power of image segmentation and generative capabilities to handle occlusions. First, <italic>OA-WinSeg</italic> automatically detects occlusions and generates a rectangular boundary guidance from a coarse window segmentation, which incorporates structural information about the building layout into the process. Subsequently, the network refines the coarse segmentation and generates window segments in the missing regions by attending to contextual information of the nonoccluded parts of the façade. Finally, our approach generates accurate vector representations, information needed for building modeling systems. Experimental results demonstrate the effectiveness of our model with simulated and occluded real-world datasets. In addition, we evaluate our model on various ablation studies to explore the contribution of the different modules. Finally, we have analyzed the potential applications of the proposed segmentation network and the completed window segments, including building façade inpainting. |
| format | Article |
| id | doaj-art-06e23d31ac3e412e84760ebf888edab3 |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-06e23d31ac3e412e84760ebf888edab32025-08-20T03:04:26ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01188490850310.1109/JSTARS.2025.355063210923720<italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior InformationManuela F. Ceron-Viveros0Wolfgang Maass1Jiaojiao Tian2https://orcid.org/0000-0002-8407-5098Department of Computer Science, Saarland University, Saarbrücken, GermanyGerman Research Center for Artificial Intelligence, Kaiserslautern, GermanyRemote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Wessling, GermanyWindow segmentation and vectorization remains a significant challenge, particularly in the absence of clean facade images. To extract complete window segments from building façade images with occlusions, this article proposes an occlusion-aware window segmentation <italic>(OA-WinSeg)</italic> network with conditional adversarial training guided by prior structural information. This architecture combines the power of image segmentation and generative capabilities to handle occlusions. First, <italic>OA-WinSeg</italic> automatically detects occlusions and generates a rectangular boundary guidance from a coarse window segmentation, which incorporates structural information about the building layout into the process. Subsequently, the network refines the coarse segmentation and generates window segments in the missing regions by attending to contextual information of the nonoccluded parts of the façade. Finally, our approach generates accurate vector representations, information needed for building modeling systems. Experimental results demonstrate the effectiveness of our model with simulated and occluded real-world datasets. In addition, we evaluate our model on various ablation studies to explore the contribution of the different modules. Finally, we have analyzed the potential applications of the proposed segmentation network and the completed window segments, including building façade inpainting.https://ieeexplore.ieee.org/document/10923720/Attention modeldeep learningfaçade imagesinpaintingocclusionswindow segmentation |
| spellingShingle | Manuela F. Ceron-Viveros Wolfgang Maass Jiaojiao Tian <italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Attention model deep learning façade images inpainting occlusions window segmentation |
| title | <italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information |
| title_full | <italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information |
| title_fullStr | <italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information |
| title_full_unstemmed | <italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information |
| title_short | <italic>OA-WinSeg:</italic> Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information |
| title_sort | italic oa winseg italic occlusion aware window segmentation with conditional adversarial training guided by structural prior information |
| topic | Attention model deep learning façade images inpainting occlusions window segmentation |
| url | https://ieeexplore.ieee.org/document/10923720/ |
| work_keys_str_mv | AT manuelafceronviveros italicoawinsegitalicocclusionawarewindowsegmentationwithconditionaladversarialtrainingguidedbystructuralpriorinformation AT wolfgangmaass italicoawinsegitalicocclusionawarewindowsegmentationwithconditionaladversarialtrainingguidedbystructuralpriorinformation AT jiaojiaotian italicoawinsegitalicocclusionawarewindowsegmentationwithconditionaladversarialtrainingguidedbystructuralpriorinformation |