Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving
We explore the use of radiance fields (RFs) to reconstruct photorealistic 3D urban scenes, creating digital twins (DTs) for autonomous driving (AD) by leveraging Nerfacto and Splatfacto models integrated with the CARLA simulator. Our research demonstrates that publicly available RFs can be utilized...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Sciendo
2024-12-01
|
| Series: | Acta Electrotechnica et Informatica |
| Subjects: | |
| Online Access: | https://doi.org/10.2478/aei-2024-0015 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850127956941209600 |
|---|---|
| author | Dopiriak Matúš Gerec Jakub Gazda Juraj |
| author_facet | Dopiriak Matúš Gerec Jakub Gazda Juraj |
| author_sort | Dopiriak Matúš |
| collection | DOAJ |
| description | We explore the use of radiance fields (RFs) to reconstruct photorealistic 3D urban scenes, creating digital twins (DTs) for autonomous driving (AD) by leveraging Nerfacto and Splatfacto models integrated with the CARLA simulator. Our research demonstrates that publicly available RFs can be utilized through Nerfstudio library to create photorealistic urban scenes and extract arbitrary images based on the camera pose. These scenes can serve as simulations for AD or as DT repositories for static environments within the vehicular metaverse. Additionally, we quantitatively evaluate RF models and use masking to remove dynamic objects, successfully simulating real-world scenarios. Quantitative evaluation shows that the Splatfacto model achieves a peak signal-to-noise ratio (PSNR) of up to 26.40, a structural similarity index measure (SSIM) of 0.84, and a learned perceptual image patch similarity (LPIPS) score of 0.21, consistently outperforming the Nerfacto model. |
| format | Article |
| id | doaj-art-3d2022a02b3f452cbbe3bfad13a12589 |
| institution | OA Journals |
| issn | 1338-3957 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Sciendo |
| record_format | Article |
| series | Acta Electrotechnica et Informatica |
| spelling | doaj-art-3d2022a02b3f452cbbe3bfad13a125892025-08-20T02:33:31ZengSciendoActa Electrotechnica et Informatica1338-39572024-12-01244273410.2478/aei-2024-0015Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous DrivingDopiriak Matúš0Gerec Jakub1Gazda Juraj2Department of Computers and Informatics, Faculty of Electrical Engineering and Informatics, Technical University of Košice, Letná 9, 042 00 Košice, Slovak Republic, Tel. +421 55 602 3175Department of Computers and Informatics, Faculty of Electrical Engineering and Informatics, Technical University of Košice, Letná 9, 042 00 Košice, Slovak Republic, Tel. +421 55 602 3175Department of Computers and Informatics, Faculty of Electrical Engineering and Informatics, Technical University of Košice, Letná 9, 042 00 Košice, Slovak Republic, Tel. +421 55 602 3175We explore the use of radiance fields (RFs) to reconstruct photorealistic 3D urban scenes, creating digital twins (DTs) for autonomous driving (AD) by leveraging Nerfacto and Splatfacto models integrated with the CARLA simulator. Our research demonstrates that publicly available RFs can be utilized through Nerfstudio library to create photorealistic urban scenes and extract arbitrary images based on the camera pose. These scenes can serve as simulations for AD or as DT repositories for static environments within the vehicular metaverse. Additionally, we quantitatively evaluate RF models and use masking to remove dynamic objects, successfully simulating real-world scenarios. Quantitative evaluation shows that the Splatfacto model achieves a peak signal-to-noise ratio (PSNR) of up to 26.40, a structural similarity index measure (SSIM) of 0.84, and a learned perceptual image patch similarity (LPIPS) score of 0.21, consistently outperforming the Nerfacto model.https://doi.org/10.2478/aei-2024-0015autonomous drivingcarla simulatordigital twinradiance fieldsvehicular metaverse |
| spellingShingle | Dopiriak Matúš Gerec Jakub Gazda Juraj Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving Acta Electrotechnica et Informatica autonomous driving carla simulator digital twin radiance fields vehicular metaverse |
| title | Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving |
| title_full | Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving |
| title_fullStr | Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving |
| title_full_unstemmed | Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving |
| title_short | Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving |
| title_sort | reconstruction of photorealistic 3d urban scenes using radiance fields as digital twins for autonomous driving |
| topic | autonomous driving carla simulator digital twin radiance fields vehicular metaverse |
| url | https://doi.org/10.2478/aei-2024-0015 |
| work_keys_str_mv | AT dopiriakmatus reconstructionofphotorealistic3durbanscenesusingradiancefieldsasdigitaltwinsforautonomousdriving AT gerecjakub reconstructionofphotorealistic3durbanscenesusingradiancefieldsasdigitaltwinsforautonomousdriving AT gazdajuraj reconstructionofphotorealistic3durbanscenesusingradiancefieldsasdigitaltwinsforautonomousdriving |