Rugularizing generalizable neural radiance field with limited-view images
Abstract We present a novel learning model with attention and prior guidance for view synthesis. In contrast to previous works that focus on optimizing for specific scenes with densely captured views, our model explores a generic deep neural framework to reconstruct radiance fields from a limited nu...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-12-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01696-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571159631101952 |
|---|---|
| author | Wei Sun Ruijia Cui Qianzhou Wang Xianguang Kong Yanning Zhang |
| author_facet | Wei Sun Ruijia Cui Qianzhou Wang Xianguang Kong Yanning Zhang |
| author_sort | Wei Sun |
| collection | DOAJ |
| description | Abstract We present a novel learning model with attention and prior guidance for view synthesis. In contrast to previous works that focus on optimizing for specific scenes with densely captured views, our model explores a generic deep neural framework to reconstruct radiance fields from a limited number of input views. To address challenges arising from under-constrained conditions, our approach employs cost volumes for geometry-aware scene reasoning, and integrates relevant knowledge from the ray-cast space and the surrounding-view space using an attention model. Additionally, a denoising diffusion model learns a prior over scene color, facilitating regularization of the training process and enabling high-quality radiance field reconstruction. Experimental results on diverse benchmark datasets demonstrate that our approach can generalize across scenes and produce realistic view synthesis results using only three input images, surpassing the performance of previous state-of-the-art methods. Moreover, our reconstructed radiance field can be effectively optimized by fine-tuning the target scene to achieve higher quality results with reduced optimization time. The code will be released at https://github.com/dsdefv/nerf . |
| format | Article |
| id | doaj-art-41a5ceee835d434a835d97dc22373e5c |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-41a5ceee835d434a835d97dc22373e5c2025-02-02T12:49:40ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-12-0111111210.1007/s40747-024-01696-6Rugularizing generalizable neural radiance field with limited-view imagesWei Sun0Ruijia Cui1Qianzhou Wang2Xianguang Kong3Yanning Zhang4School of Computer Science and Technology, Xi’an University of Posts and TelecommunicationsSchool of Computer Science and Technology, Xi’an University of Posts and TelecommunicationsSchool of Computer Science and Technology, Xi’an University of Posts and TelecommunicationsSchool of Mechano-Electronic Engineering, Xidian UniversityNational Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application TechnologyAbstract We present a novel learning model with attention and prior guidance for view synthesis. In contrast to previous works that focus on optimizing for specific scenes with densely captured views, our model explores a generic deep neural framework to reconstruct radiance fields from a limited number of input views. To address challenges arising from under-constrained conditions, our approach employs cost volumes for geometry-aware scene reasoning, and integrates relevant knowledge from the ray-cast space and the surrounding-view space using an attention model. Additionally, a denoising diffusion model learns a prior over scene color, facilitating regularization of the training process and enabling high-quality radiance field reconstruction. Experimental results on diverse benchmark datasets demonstrate that our approach can generalize across scenes and produce realistic view synthesis results using only three input images, surpassing the performance of previous state-of-the-art methods. Moreover, our reconstructed radiance field can be effectively optimized by fine-tuning the target scene to achieve higher quality results with reduced optimization time. The code will be released at https://github.com/dsdefv/nerf .https://doi.org/10.1007/s40747-024-01696-6Neural radiance fieldLimited viewsGeneralizable networkDiffusion model |
| spellingShingle | Wei Sun Ruijia Cui Qianzhou Wang Xianguang Kong Yanning Zhang Rugularizing generalizable neural radiance field with limited-view images Complex & Intelligent Systems Neural radiance field Limited views Generalizable network Diffusion model |
| title | Rugularizing generalizable neural radiance field with limited-view images |
| title_full | Rugularizing generalizable neural radiance field with limited-view images |
| title_fullStr | Rugularizing generalizable neural radiance field with limited-view images |
| title_full_unstemmed | Rugularizing generalizable neural radiance field with limited-view images |
| title_short | Rugularizing generalizable neural radiance field with limited-view images |
| title_sort | rugularizing generalizable neural radiance field with limited view images |
| topic | Neural radiance field Limited views Generalizable network Diffusion model |
| url | https://doi.org/10.1007/s40747-024-01696-6 |
| work_keys_str_mv | AT weisun rugularizinggeneralizableneuralradiancefieldwithlimitedviewimages AT ruijiacui rugularizinggeneralizableneuralradiancefieldwithlimitedviewimages AT qianzhouwang rugularizinggeneralizableneuralradiancefieldwithlimitedviewimages AT xianguangkong rugularizinggeneralizableneuralradiancefieldwithlimitedviewimages AT yanningzhang rugularizinggeneralizableneuralradiancefieldwithlimitedviewimages |