DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation
While most existing advanced large-scale point cloud semantic segmentation methods can accurately identify most large-scale objects, there is still room for improvement in the recognition accuracy of small-scale, low-proportion objects. Compared to point clouds, digital orthophoto maps (DOMs) has a...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11048913/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850069932426919936 |
|---|---|
| author | Zhenzhen Song Mingqiang Guo Liang Wu Heng Liu Ying Huang Zheng Liu |
| author_facet | Zhenzhen Song Mingqiang Guo Liang Wu Heng Liu Ying Huang Zheng Liu |
| author_sort | Zhenzhen Song |
| collection | DOAJ |
| description | While most existing advanced large-scale point cloud semantic segmentation methods can accurately identify most large-scale objects, there is still room for improvement in the recognition accuracy of small-scale, low-proportion objects. Compared to point clouds, digital orthophoto maps (DOMs) has a more structured data format, allowing for better recognition of small-scale surface features. However, in existing projection-based methods, directly mapping images onto point clouds leads to occlusion issues. If image and point cloud features are simply concatenated, it results in feature blurring. Based on this observation, this article proposes a DAPSS network for point cloud semantic segmentation, assisted by prior knowledge constructed from DOM. The pretrained DOM features can provide a broader receptive field as guidance for learning the local context features of point clouds. Vertical occlusion has an issue, making ray-based mapping methods unsuitable. We propose a method that search for the nearest mapped point cloud in spherical space to fill in the occluded point cloud based on the already mapped point cloud. The traditional approach of directly concatenating point cloud features with image features often leads to feature blurring. Therefore, we propose a plug-and-play multimodal feature adaptive fusion module, which can adaptively select and aggregate features from different modalities to reduce redundant information further. In addition, we designed a cascaded multimodal feature deep fusion module to promote deep fusion between different modal features. Experiments on two large datasets demonstrate that DAPSS outperforms current mainstream methods, achieving mean Intersection-over-Union scores of 65.9% and 82.9% on the SansetUrban and SUM-Helsinki datasets, respectively. DAPSS not only effectively addresses the recognition of small-scale surface features, but also resolves the occlusion problems associated with projection-based methods. |
| format | Article |
| id | doaj-art-012bdedfeed34b4baf694767f141ac8b |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-012bdedfeed34b4baf694767f141ac8b2025-08-20T02:47:39ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118182641828010.1109/JSTARS.2025.358280411048913DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic SegmentationZhenzhen Song0https://orcid.org/0009-0004-7949-4038Mingqiang Guo1https://orcid.org/0000-0003-4097-4814Liang Wu2https://orcid.org/0000-0003-1877-7179Heng Liu3Ying Huang4Zheng Liu5https://orcid.org/0000-0001-6713-6680School of Geography and Information Engineering, China University of Geosciences, Wuhan, ChinaSchool of Computer Science, China University of Geosciences, Wuhan, ChinaSchool of Computer Science and the School of Geography and Information Engineering, China University of Geosciences, Wuhan, ChinaSchool of Geography and Information Engineering, China University of Geosciences, Wuhan, ChinaWuhan Zondy Cyber Technology Company Ltd., Wuhan, ChinaSchool of Computer Science and the School of Geography and Information Engineering, China University of Geosciences, Wuhan, ChinaWhile most existing advanced large-scale point cloud semantic segmentation methods can accurately identify most large-scale objects, there is still room for improvement in the recognition accuracy of small-scale, low-proportion objects. Compared to point clouds, digital orthophoto maps (DOMs) has a more structured data format, allowing for better recognition of small-scale surface features. However, in existing projection-based methods, directly mapping images onto point clouds leads to occlusion issues. If image and point cloud features are simply concatenated, it results in feature blurring. Based on this observation, this article proposes a DAPSS network for point cloud semantic segmentation, assisted by prior knowledge constructed from DOM. The pretrained DOM features can provide a broader receptive field as guidance for learning the local context features of point clouds. Vertical occlusion has an issue, making ray-based mapping methods unsuitable. We propose a method that search for the nearest mapped point cloud in spherical space to fill in the occluded point cloud based on the already mapped point cloud. The traditional approach of directly concatenating point cloud features with image features often leads to feature blurring. Therefore, we propose a plug-and-play multimodal feature adaptive fusion module, which can adaptively select and aggregate features from different modalities to reduce redundant information further. In addition, we designed a cascaded multimodal feature deep fusion module to promote deep fusion between different modal features. Experiments on two large datasets demonstrate that DAPSS outperforms current mainstream methods, achieving mean Intersection-over-Union scores of 65.9% and 82.9% on the SansetUrban and SUM-Helsinki datasets, respectively. DAPSS not only effectively addresses the recognition of small-scale surface features, but also resolves the occlusion problems associated with projection-based methods.https://ieeexplore.ieee.org/document/11048913/Digital orthophoto map (DOM)feature fusionoblique photogrammetrypoint cloud semantic segmentation |
| spellingShingle | Zhenzhen Song Mingqiang Guo Liang Wu Heng Liu Ying Huang Zheng Liu DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Digital orthophoto map (DOM) feature fusion oblique photogrammetry point cloud semantic segmentation |
| title | DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation |
| title_full | DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation |
| title_fullStr | DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation |
| title_full_unstemmed | DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation |
| title_short | DAPSS: A Novel Network for DOM Assisted Oblique Photography Point Cloud Semantic Segmentation |
| title_sort | dapss a novel network for dom assisted oblique photography point cloud semantic segmentation |
| topic | Digital orthophoto map (DOM) feature fusion oblique photogrammetry point cloud semantic segmentation |
| url | https://ieeexplore.ieee.org/document/11048913/ |
| work_keys_str_mv | AT zhenzhensong dapssanovelnetworkfordomassistedobliquephotographypointcloudsemanticsegmentation AT mingqiangguo dapssanovelnetworkfordomassistedobliquephotographypointcloudsemanticsegmentation AT liangwu dapssanovelnetworkfordomassistedobliquephotographypointcloudsemanticsegmentation AT hengliu dapssanovelnetworkfordomassistedobliquephotographypointcloudsemanticsegmentation AT yinghuang dapssanovelnetworkfordomassistedobliquephotographypointcloudsemanticsegmentation AT zhengliu dapssanovelnetworkfordomassistedobliquephotographypointcloudsemanticsegmentation |