Pushing the limit to near real-time indoor LiDAR-based semantic segmentation
Semantic segmentation of indoor 3D point clouds is a critical technology for understanding three dimensional indoor environments, with significant applications in indoor navigation, positioning, and intelligent robotics. While real-time semantic segmentation is already a reality for images, existing...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2024-12-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://isprs-archives.copernicus.org/articles/XLVIII-2-W8-2024/45/2024/isprs-archives-XLVIII-2-W8-2024-45-2024.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850111098747879424 |
|---|---|
| author | P. Bournez J. Salzinger M. Cella F. Vultaggio F. d’Apolito P. Fanta-Jende |
| author_facet | P. Bournez J. Salzinger M. Cella F. Vultaggio F. d’Apolito P. Fanta-Jende |
| author_sort | P. Bournez |
| collection | DOAJ |
| description | Semantic segmentation of indoor 3D point clouds is a critical technology for understanding three dimensional indoor environments, with significant applications in indoor navigation, positioning, and intelligent robotics. While real-time semantic segmentation is already a reality for images, existing classification pipelines for LiDAR point clouds assume a pre-existing map which relies on data collected from accurate but heavy sensors. However, this approach is impractical for high-level task planning and autonomous exploration, which benefits from a rapid 3D structure understanding of the environment. Furthermore, while RGB cameras remain a popular choice in good visibility conditions, such sensors are inefficient in environments where visibility is hindered. Consequently, LiDAR point clouds emerge as a rather reliable source of environmental information in such circumstances. In this paper, we adapt an existing semantic segmentation model, Superpoint Transformer, to LiDAR-based situation where RGB inputs are not available and near real-time processing is attempted. To this end, we simulated our robot’s trajectory and leveraged Hidden Point Removal using the open-source dataset S3DIS to train the model. We investigated various strategies such as modifying the interval prediction and thoroughly study its influence on the prediction intervals. Our model demonstrates an improvement from 40 to 67.6 mean Intersection over Union (mIoU) compared to the baseline on simple (floor, ceiling, walls) and complex (doors, windows) classes. |
| format | Article |
| id | doaj-art-3e6f1ffc71704e5d87ffe90e179a4808 |
| institution | OA Journals |
| issn | 1682-1750 2194-9034 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj-art-3e6f1ffc71704e5d87ffe90e179a48082025-08-20T02:37:41ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342024-12-01XLVIII-2-W8-2024455210.5194/isprs-archives-XLVIII-2-W8-2024-45-2024Pushing the limit to near real-time indoor LiDAR-based semantic segmentationP. Bournez0J. Salzinger1M. Cella2F. Vultaggio3F. d’Apolito4P. Fanta-Jende5AIT, Austrian Institute of Technology - Center for Vision, Automation and Control, Unit Assistive and Autonomous Systems, AustriaAIT, Austrian Institute of Technology - Center for Vision, Automation and Control, Unit Assistive and Autonomous Systems, AustriaAIT, Austrian Institute of Technology - Center for Vision, Automation and Control, Unit Assistive and Autonomous Systems, AustriaAIT, Austrian Institute of Technology - Center for Vision, Automation and Control, Unit Assistive and Autonomous Systems, AustriaAIT, Austrian Institute of Technology - Center for Vision, Automation and Control, Unit Assistive and Autonomous Systems, AustriaAIT, Austrian Institute of Technology - Center for Vision, Automation and Control, Unit Assistive and Autonomous Systems, AustriaSemantic segmentation of indoor 3D point clouds is a critical technology for understanding three dimensional indoor environments, with significant applications in indoor navigation, positioning, and intelligent robotics. While real-time semantic segmentation is already a reality for images, existing classification pipelines for LiDAR point clouds assume a pre-existing map which relies on data collected from accurate but heavy sensors. However, this approach is impractical for high-level task planning and autonomous exploration, which benefits from a rapid 3D structure understanding of the environment. Furthermore, while RGB cameras remain a popular choice in good visibility conditions, such sensors are inefficient in environments where visibility is hindered. Consequently, LiDAR point clouds emerge as a rather reliable source of environmental information in such circumstances. In this paper, we adapt an existing semantic segmentation model, Superpoint Transformer, to LiDAR-based situation where RGB inputs are not available and near real-time processing is attempted. To this end, we simulated our robot’s trajectory and leveraged Hidden Point Removal using the open-source dataset S3DIS to train the model. We investigated various strategies such as modifying the interval prediction and thoroughly study its influence on the prediction intervals. Our model demonstrates an improvement from 40 to 67.6 mean Intersection over Union (mIoU) compared to the baseline on simple (floor, ceiling, walls) and complex (doors, windows) classes.https://isprs-archives.copernicus.org/articles/XLVIII-2-W8-2024/45/2024/isprs-archives-XLVIII-2-W8-2024-45-2024.pdf |
| spellingShingle | P. Bournez J. Salzinger M. Cella F. Vultaggio F. d’Apolito P. Fanta-Jende Pushing the limit to near real-time indoor LiDAR-based semantic segmentation The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | Pushing the limit to near real-time indoor LiDAR-based semantic segmentation |
| title_full | Pushing the limit to near real-time indoor LiDAR-based semantic segmentation |
| title_fullStr | Pushing the limit to near real-time indoor LiDAR-based semantic segmentation |
| title_full_unstemmed | Pushing the limit to near real-time indoor LiDAR-based semantic segmentation |
| title_short | Pushing the limit to near real-time indoor LiDAR-based semantic segmentation |
| title_sort | pushing the limit to near real time indoor lidar based semantic segmentation |
| url | https://isprs-archives.copernicus.org/articles/XLVIII-2-W8-2024/45/2024/isprs-archives-XLVIII-2-W8-2024-45-2024.pdf |
| work_keys_str_mv | AT pbournez pushingthelimittonearrealtimeindoorlidarbasedsemanticsegmentation AT jsalzinger pushingthelimittonearrealtimeindoorlidarbasedsemanticsegmentation AT mcella pushingthelimittonearrealtimeindoorlidarbasedsemanticsegmentation AT fvultaggio pushingthelimittonearrealtimeindoorlidarbasedsemanticsegmentation AT fdapolito pushingthelimittonearrealtimeindoorlidarbasedsemanticsegmentation AT pfantajende pushingthelimittonearrealtimeindoorlidarbasedsemanticsegmentation |