DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point Cloud
High-resolution remote sensing image segmentation has advanced significantly with 2-D convolutional neural networks and transformer-based models like SegFormer and Swin Transformer. Concurrently, the rapid development of 3-D convolution techniques has driven advancements in methods like PointNet and...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10778434/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846118376714272768 |
|---|---|
| author | Wei Liu He Wang Yicheng Qiao Haopeng Zhang Junli Yang |
| author_facet | Wei Liu He Wang Yicheng Qiao Haopeng Zhang Junli Yang |
| author_sort | Wei Liu |
| collection | DOAJ |
| description | High-resolution remote sensing image segmentation has advanced significantly with 2-D convolutional neural networks and transformer-based models like SegFormer and Swin Transformer. Concurrently, the rapid development of 3-D convolution techniques has driven advancements in methods like PointNet and Kernel Point Convolution for 3-D LiDAR point cloud segmentation. Traditional fusion of aerial imagery and LiDAR data often relies on digital surface models or other features extracted from LiDAR point clouds, incorporating them as depth channels into image data. In this article, we propose a novel approach called Direct LiDAR-Aerial Fusion Network, which directly integrates multispectral images (RGB) and LiDAR point cloud data for semantic segmentation. Experiments on the modified GRSS18 dataset demonstrate that our method achieves an overall accuracy (OA) of 79.88%, outperforming conventional approaches. By fusing RGB and LiDAR features, our technique improves OA by 1.77% and mean Intersection over Union by 0.83%. |
| format | Article |
| id | doaj-art-cd7c8707f30241ddbb0519a6217e79ac |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-cd7c8707f30241ddbb0519a6217e79ac2024-12-18T00:00:29ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-01181864187510.1109/JSTARS.2024.351151710778434DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point CloudWei Liu0He Wang1Yicheng Qiao2Haopeng Zhang3https://orcid.org/0000-0003-1981-8307Junli Yang4https://orcid.org/0000-0001-8370-7105School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, ChinaSino-French Engineer School, Beihang University, Beijing, ChinaSchool of Sports Engineering, Beijing Sport University, Beijing, ChinaSchool of Astronautics, Beihang University, Beijing, ChinaInternational School, Beijing University of Posts and Telecommunications, Beijing, ChinaHigh-resolution remote sensing image segmentation has advanced significantly with 2-D convolutional neural networks and transformer-based models like SegFormer and Swin Transformer. Concurrently, the rapid development of 3-D convolution techniques has driven advancements in methods like PointNet and Kernel Point Convolution for 3-D LiDAR point cloud segmentation. Traditional fusion of aerial imagery and LiDAR data often relies on digital surface models or other features extracted from LiDAR point clouds, incorporating them as depth channels into image data. In this article, we propose a novel approach called Direct LiDAR-Aerial Fusion Network, which directly integrates multispectral images (RGB) and LiDAR point cloud data for semantic segmentation. Experiments on the modified GRSS18 dataset demonstrate that our method achieves an overall accuracy (OA) of 79.88%, outperforming conventional approaches. By fusing RGB and LiDAR features, our technique improves OA by 1.77% and mean Intersection over Union by 0.83%.https://ieeexplore.ieee.org/document/10778434/Aerial imagedata fusionlight detection and ranging (LiDAR)point cloudsemantic segmentation |
| spellingShingle | Wei Liu He Wang Yicheng Qiao Haopeng Zhang Junli Yang DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point Cloud IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Aerial image data fusion light detection and ranging (LiDAR) point cloud semantic segmentation |
| title | DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point Cloud |
| title_full | DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point Cloud |
| title_fullStr | DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point Cloud |
| title_full_unstemmed | DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point Cloud |
| title_short | DLAFNet: Direct LiDAR-Aerial Fusion Network for Semantic Segmentation of 2-D Aerial Image and 3-D LiDAR Point Cloud |
| title_sort | dlafnet direct lidar aerial fusion network for semantic segmentation of 2 d aerial image and 3 d lidar point cloud |
| topic | Aerial image data fusion light detection and ranging (LiDAR) point cloud semantic segmentation |
| url | https://ieeexplore.ieee.org/document/10778434/ |
| work_keys_str_mv | AT weiliu dlafnetdirectlidaraerialfusionnetworkforsemanticsegmentationof2daerialimageand3dlidarpointcloud AT hewang dlafnetdirectlidaraerialfusionnetworkforsemanticsegmentationof2daerialimageand3dlidarpointcloud AT yichengqiao dlafnetdirectlidaraerialfusionnetworkforsemanticsegmentationof2daerialimageand3dlidarpointcloud AT haopengzhang dlafnetdirectlidaraerialfusionnetworkforsemanticsegmentationof2daerialimageand3dlidarpointcloud AT junliyang dlafnetdirectlidaraerialfusionnetworkforsemanticsegmentationof2daerialimageand3dlidarpointcloud |