Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal Discretization
Whereas emerging learning-based scene representations are predominantly evaluated based on image quality metrics such as PSNR, SSIM or LPIPS, only a few investigations focus on the evaluation of geometric accuracy of the underlying model. In contrast to only demonstrating the geometric deviations of...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2024-12-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://isprs-archives.copernicus.org/articles/XLVIII-2-W7-2024/49/2024/isprs-archives-XLVIII-2-W7-2024-49-2024.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850248072853979136 |
|---|---|
| author | D. Haitz B. Kıvılcım M. Ulrich M. Weinmann M. Weinmann |
| author_facet | D. Haitz B. Kıvılcım M. Ulrich M. Weinmann M. Weinmann |
| author_sort | D. Haitz |
| collection | DOAJ |
| description | Whereas emerging learning-based scene representations are predominantly evaluated based on image quality metrics such as PSNR, SSIM or LPIPS, only a few investigations focus on the evaluation of geometric accuracy of the underlying model. In contrast to only demonstrating the geometric deviations of models for the fully optimized scene model, our work aims at investigating the geometric convergence behavior during the optimization. For this purpose, we analyze the geometric convergence of discretized density fields by leveraging respectively derived point cloud representations for different training steps during the optimization of the scene representation and their comparison based on established point cloud metrics, thereby allowing insights regarding which scene parts are already represented well within the scene representation at a certain time during the optimization. By demonstrating that certain regions reach convergence earlier than other regions in the scene, we provide the motivation regarding future developments on locally-guided optimization approaches to shift the computational burden to the adjustment of regions that still need to converge while leaving converged regions unchanged which might help to further reduce training time and improve the achieved quality. |
| format | Article |
| id | doaj-art-142b756b5ef44ff89e662df60d72c47d |
| institution | OA Journals |
| issn | 1682-1750 2194-9034 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj-art-142b756b5ef44ff89e662df60d72c47d2025-08-20T01:58:48ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342024-12-01XLVIII-2-W7-2024495610.5194/isprs-archives-XLVIII-2-W7-2024-49-2024Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal DiscretizationD. Haitz0B. Kıvılcım1M. Ulrich2M. Weinmann3M. Weinmann4Karlsruhe Institute of Technology, Institute of Photogrammetry and Remote Sensing, GermanyKarlsruhe Institute of Technology, Institute of Photogrammetry and Remote Sensing, GermanyKarlsruhe Institute of Technology, Institute of Photogrammetry and Remote Sensing, GermanyKarlsruhe Institute of Technology, Institute of Photogrammetry and Remote Sensing, GermanyDelft University of Technology, Department of Intelligent Systems, The NetherlandsWhereas emerging learning-based scene representations are predominantly evaluated based on image quality metrics such as PSNR, SSIM or LPIPS, only a few investigations focus on the evaluation of geometric accuracy of the underlying model. In contrast to only demonstrating the geometric deviations of models for the fully optimized scene model, our work aims at investigating the geometric convergence behavior during the optimization. For this purpose, we analyze the geometric convergence of discretized density fields by leveraging respectively derived point cloud representations for different training steps during the optimization of the scene representation and their comparison based on established point cloud metrics, thereby allowing insights regarding which scene parts are already represented well within the scene representation at a certain time during the optimization. By demonstrating that certain regions reach convergence earlier than other regions in the scene, we provide the motivation regarding future developments on locally-guided optimization approaches to shift the computational burden to the adjustment of regions that still need to converge while leaving converged regions unchanged which might help to further reduce training time and improve the achieved quality.https://isprs-archives.copernicus.org/articles/XLVIII-2-W7-2024/49/2024/isprs-archives-XLVIII-2-W7-2024-49-2024.pdf |
| spellingShingle | D. Haitz B. Kıvılcım M. Ulrich M. Weinmann M. Weinmann Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal Discretization The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal Discretization |
| title_full | Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal Discretization |
| title_fullStr | Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal Discretization |
| title_full_unstemmed | Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal Discretization |
| title_short | Density-based Geometric Convergence of NeRFs at Training Time: Insights from Spatio-temporal Discretization |
| title_sort | density based geometric convergence of nerfs at training time insights from spatio temporal discretization |
| url | https://isprs-archives.copernicus.org/articles/XLVIII-2-W7-2024/49/2024/isprs-archives-XLVIII-2-W7-2024-49-2024.pdf |
| work_keys_str_mv | AT dhaitz densitybasedgeometricconvergenceofnerfsattrainingtimeinsightsfromspatiotemporaldiscretization AT bkıvılcım densitybasedgeometricconvergenceofnerfsattrainingtimeinsightsfromspatiotemporaldiscretization AT mulrich densitybasedgeometricconvergenceofnerfsattrainingtimeinsightsfromspatiotemporaldiscretization AT mweinmann densitybasedgeometricconvergenceofnerfsattrainingtimeinsightsfromspatiotemporaldiscretization AT mweinmann densitybasedgeometricconvergenceofnerfsattrainingtimeinsightsfromspatiotemporaldiscretization |