Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo Supervision
Neural implicit surface reconstruction has recently emerged as a prominent paradigm in multi-view 3D reconstruction using deep learning. In contrast to traditional multi-view stereo methods, signed distance function (SDF)-based approaches leverage neural networks to effectively represent 3D scenes....
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10900377/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850023459432693760 |
|---|---|
| author | Wongyeom Kim Jisun Park Kyungeun Cho |
| author_facet | Wongyeom Kim Jisun Park Kyungeun Cho |
| author_sort | Wongyeom Kim |
| collection | DOAJ |
| description | Neural implicit surface reconstruction has recently emerged as a prominent paradigm in multi-view 3D reconstruction using deep learning. In contrast to traditional multi-view stereo methods, signed distance function (SDF)-based approaches leverage neural networks to effectively represent 3D scenes. Furthermore, to reconstruct scenes and individual objects separately, some studies have extended the framework for object-compositional neural implicit surface reconstruction by utilizing 2D instance masks to supervise the SDF of each object. Nonetheless, these methods often reconstruct objects as partial shapes in scenes captured from sparse viewpoints or in complex scenes containing multiple objects. This issue primarily stems from the absence of a 3D prior, which fails to provide sufficient geometry for partially observed and occluded regions. We propose a framework for completing the partial object shapes of object-compositional neural implicit representation utilizing a diffusion-based 3D mesh generation model. The existing diffusion model, trained only on large-scale 3D object datasets, generates complete shapes from partial shapes; however, their results differ significantly from the objects in the scene. To complete the representation of partial shapes while ensuring shape consistency across multi-view images, we combine the SDF values, output by the diffusion model, with the object-compositional neural implicit representation. The combined representation is then volume-rendered to fine-tune the diffusion model utilizing a 2D prior. Furthermore, the complete shape generated by our method can serve as pseudo 3D priors to provide the geometry for the unobserved regions in object-compositional representation. Extensive experiments demonstrate that our novel framework significantly improves the reconstruction quality of unobserved regions. |
| format | Article |
| id | doaj-art-1ed0f8a777d147b78f4610afa1a90eb4 |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-1ed0f8a777d147b78f4610afa1a90eb42025-08-20T03:01:22ZengIEEEIEEE Access2169-35362025-01-0113361513616110.1109/ACCESS.2025.354470510900377Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo SupervisionWongyeom Kim0https://orcid.org/0009-0008-4788-935XJisun Park1https://orcid.org/0000-0002-4304-1780Kyungeun Cho2https://orcid.org/0000-0003-2219-0848Department of Computer Science and Artificial Intelligence, Dongguk University, Jung-gu, Seoul, Republic of KoreaNUI/NUX Platform Research Center, Dongguk University, Jung-gu, Seoul, Republic of KoreaDivision of AI Software Convergence, Dongguk University, Jung-gu, Seoul, Republic of KoreaNeural implicit surface reconstruction has recently emerged as a prominent paradigm in multi-view 3D reconstruction using deep learning. In contrast to traditional multi-view stereo methods, signed distance function (SDF)-based approaches leverage neural networks to effectively represent 3D scenes. Furthermore, to reconstruct scenes and individual objects separately, some studies have extended the framework for object-compositional neural implicit surface reconstruction by utilizing 2D instance masks to supervise the SDF of each object. Nonetheless, these methods often reconstruct objects as partial shapes in scenes captured from sparse viewpoints or in complex scenes containing multiple objects. This issue primarily stems from the absence of a 3D prior, which fails to provide sufficient geometry for partially observed and occluded regions. We propose a framework for completing the partial object shapes of object-compositional neural implicit representation utilizing a diffusion-based 3D mesh generation model. The existing diffusion model, trained only on large-scale 3D object datasets, generates complete shapes from partial shapes; however, their results differ significantly from the objects in the scene. To complete the representation of partial shapes while ensuring shape consistency across multi-view images, we combine the SDF values, output by the diffusion model, with the object-compositional neural implicit representation. The combined representation is then volume-rendered to fine-tune the diffusion model utilizing a 2D prior. Furthermore, the complete shape generated by our method can serve as pseudo 3D priors to provide the geometry for the unobserved regions in object-compositional representation. Extensive experiments demonstrate that our novel framework significantly improves the reconstruction quality of unobserved regions.https://ieeexplore.ieee.org/document/10900377/Deep learningmesh generationsurface reconstruction |
| spellingShingle | Wongyeom Kim Jisun Park Kyungeun Cho Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo Supervision IEEE Access Deep learning mesh generation surface reconstruction |
| title | Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo Supervision |
| title_full | Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo Supervision |
| title_fullStr | Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo Supervision |
| title_full_unstemmed | Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo Supervision |
| title_short | Complete Object-Compositional Neural Implicit Surfaces With 3D Pseudo Supervision |
| title_sort | complete object compositional neural implicit surfaces with 3d pseudo supervision |
| topic | Deep learning mesh generation surface reconstruction |
| url | https://ieeexplore.ieee.org/document/10900377/ |
| work_keys_str_mv | AT wongyeomkim completeobjectcompositionalneuralimplicitsurfaceswith3dpseudosupervision AT jisunpark completeobjectcompositionalneuralimplicitsurfaceswith3dpseudosupervision AT kyungeuncho completeobjectcompositionalneuralimplicitsurfaceswith3dpseudosupervision |