3D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour Points
Real-world 3D reconstructions of building façades in LoD3 and beyond are not yet widely available on the mass market due to financial and technological barriers, as well as the challenges of automated modeling. We propose a novel method for extracting 3D façade elements using i...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2025-07-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://isprs-archives.copernicus.org/articles/XLVIII-G-2025/473/2025/isprs-archives-XLVIII-G-2025-473-2025.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849245789751083008 |
|---|---|
| author | F. Frank V. Shah S. Worbis L. Hoegner |
| author_facet | F. Frank V. Shah S. Worbis L. Hoegner |
| author_sort | F. Frank |
| collection | DOAJ |
| description | Real-world 3D reconstructions of building façades in LoD3 and beyond are not yet widely available on the mass market due to financial and technological barriers, as well as the challenges of automated modeling. We propose a novel method for extracting 3D façade elements using image-based instance segmentation and scale-invariant object contour points (SIOCP). Our methodology uses RGB images, camera parameters, absolute 6DoF pose and position, as well as LoD2 building information. The images are processed using instance segmentation with YOLOv8 and SAM, complemented by classical and enhanced algorithms for line and edge detection. The SIOCP method refines object contour lines from instance segmentation by incorporating LoD2 building data and 6DoF information. Subsequently, the keypoints are estimated and the single-camera image 6DoF pose is reconstructed using a PnP solver. From these 6DoF poses a photogrammetrically point cloud is generated, and semantically- and instance-segmented with SuperCluster. The segmentation results are intended for future comparisons with other point clouds and LoD3 reconstructions. The presented approach is still under development, so the current results are limited. In summary, this paper introduces a key component of our vision for LoD3 reconstruction by using handheld devices. |
| format | Article |
| id | doaj-art-4a3e1abb320340ccafc75916acfed07a |
| institution | Kabale University |
| issn | 1682-1750 2194-9034 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj-art-4a3e1abb320340ccafc75916acfed07a2025-08-20T03:58:41ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342025-07-01XLVIII-G-202547348010.5194/isprs-archives-XLVIII-G-2025-473-20253D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour PointsF. Frank0V. Shah1S. Worbis2L. Hoegner3IWT – Institute of Economic and Technical Sciences, Friedrichshafen, GermanyIWT – Institute of Economic and Technical Sciences, Friedrichshafen, GermanyDepartment of Geoinformatics, Hochschule München University of Applied Sciences, Munich, GermanyDepartment of Geoinformatics, Hochschule München University of Applied Sciences, Munich, GermanyReal-world 3D reconstructions of building façades in LoD3 and beyond are not yet widely available on the mass market due to financial and technological barriers, as well as the challenges of automated modeling. We propose a novel method for extracting 3D façade elements using image-based instance segmentation and scale-invariant object contour points (SIOCP). Our methodology uses RGB images, camera parameters, absolute 6DoF pose and position, as well as LoD2 building information. The images are processed using instance segmentation with YOLOv8 and SAM, complemented by classical and enhanced algorithms for line and edge detection. The SIOCP method refines object contour lines from instance segmentation by incorporating LoD2 building data and 6DoF information. Subsequently, the keypoints are estimated and the single-camera image 6DoF pose is reconstructed using a PnP solver. From these 6DoF poses a photogrammetrically point cloud is generated, and semantically- and instance-segmented with SuperCluster. The segmentation results are intended for future comparisons with other point clouds and LoD3 reconstructions. The presented approach is still under development, so the current results are limited. In summary, this paper introduces a key component of our vision for LoD3 reconstruction by using handheld devices.https://isprs-archives.copernicus.org/articles/XLVIII-G-2025/473/2025/isprs-archives-XLVIII-G-2025-473-2025.pdf |
| spellingShingle | F. Frank V. Shah S. Worbis L. Hoegner 3D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour Points The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | 3D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour Points |
| title_full | 3D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour Points |
| title_fullStr | 3D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour Points |
| title_full_unstemmed | 3D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour Points |
| title_short | 3D Façade Element Extraction from Image-based Instance Segmentation and Scale-Invariant Object Contour Points |
| title_sort | 3d facade element extraction from image based instance segmentation and scale invariant object contour points |
| url | https://isprs-archives.copernicus.org/articles/XLVIII-G-2025/473/2025/isprs-archives-XLVIII-G-2025-473-2025.pdf |
| work_keys_str_mv | AT ffrank 3dfacadeelementextractionfromimagebasedinstancesegmentationandscaleinvariantobjectcontourpoints AT vshah 3dfacadeelementextractionfromimagebasedinstancesegmentationandscaleinvariantobjectcontourpoints AT sworbis 3dfacadeelementextractionfromimagebasedinstancesegmentationandscaleinvariantobjectcontourpoints AT lhoegner 3dfacadeelementextractionfromimagebasedinstancesegmentationandscaleinvariantobjectcontourpoints |