A 3D reconstruction platform for complex plants using OB-NeRF
IntroductionApplying 3D reconstruction techniques to individual plants has enhanced high-throughput phenotyping and provided accurate data support for developing "digital twins" in the agricultural domain. High costs, slow processing times, intricate workflows, and limited automation often...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-03-01
|
| Series: | Frontiers in Plant Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpls.2025.1449626/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850250398711939072 |
|---|---|
| author | Sixiao Wu Changhao Hu Boyuan Tian Yuan Huang Shuo Yang Shanjun Li Shengyong Xu |
| author_facet | Sixiao Wu Changhao Hu Boyuan Tian Yuan Huang Shuo Yang Shanjun Li Shengyong Xu |
| author_sort | Sixiao Wu |
| collection | DOAJ |
| description | IntroductionApplying 3D reconstruction techniques to individual plants has enhanced high-throughput phenotyping and provided accurate data support for developing "digital twins" in the agricultural domain. High costs, slow processing times, intricate workflows, and limited automation often constrain the application of existing 3D reconstruction platforms.MethodsWe develop a 3D reconstruction platform for complex plants to overcome these issues. Initially, a video acquisition system is built based on "camera to plant" mode. Then, we extract the keyframes in the videos. After that, Zhang Zhengyou's calibration method and Structure from Motion(SfM)are utilized to estimate the camera parameters. Next, Camera poses estimated from SfM were automatically calibrated using camera imaging trajectories as prior knowledge. Finally, Object-Based NeRF we proposed is utilized for the fine-scale reconstruction of plants. The OB-NeRF algorithm introduced a new ray sampling strategy that improved the efficiency and quality of target plant reconstruction without segmenting the background of images. Furthermore, the precision of the reconstruction was enhanced by optimizing camera poses. An exposure adjustment phase was integrated to improve the algorithm's robustness in uneven lighting conditions. The training process was significantly accelerated through the use of shallow MLP and multi-resolution hash encoding. Lastly, the camera imaging trajectories contributed to the automatic localization of target plants within the scene, enabling the automated extraction of Mesh. Results and discussionOur pipeline reconstructed high-quality neural radiance fields of the target plant from captured videos in just 250 seconds, enabling the synthesis of novel viewpoint images and the extraction of Mesh. OB-NeRF surpasses NeRF in PSNR evaluation and reduces the reconstruction time from over 10 hours to just 30 Seconds. Compared to Instant-NGP, NeRFacto, and NeuS, OB-NeRF achieves higher reconstruction quality in a shorter reconstruction time. Moreover, Our reconstructed 3D model demonstrated superior texture and geometric fidelity compared to those generated by COLMAP and Kinect-based reconstruction methods. The $R^2$ was 0.9933,0.9881 and 0.9883 for plant height, leaf length, and leaf width, respectively. The MAE was 2.0947, 0.1898, and 0.1199 cm. The 3D reconstruction platform introduced in this study provides a robust foundation for high-throughput phenotyping and the creation of agricultural “digital twins”. |
| format | Article |
| id | doaj-art-042fd70c5a304b39ab6b0d2ce7d2925b |
| institution | OA Journals |
| issn | 1664-462X |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Plant Science |
| spelling | doaj-art-042fd70c5a304b39ab6b0d2ce7d2925b2025-08-20T01:58:12ZengFrontiers Media S.A.Frontiers in Plant Science1664-462X2025-03-011610.3389/fpls.2025.14496261449626A 3D reconstruction platform for complex plants using OB-NeRFSixiao Wu0Changhao Hu1Boyuan Tian2Yuan Huang3Shuo Yang4Shanjun Li5Shengyong Xu6China College of Engineering/Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Huazhong Agricultural University, Wuhan, ChinaChina College of Engineering/Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Huazhong Agricultural University, Wuhan, ChinaChina College of Engineering/Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Huazhong Agricultural University, Wuhan, ChinaCollege of Horticulture and Forestry Sciences, National Key Laboratory for Germplasm Innovation & Utilization of Horticultural Crops, Huazhong Agricultural University, Wuhan, ChinaXianning Academy of Agricultural Science, Xianning, ChinaChina College of Engineering/Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Huazhong Agricultural University, Wuhan, ChinaChina College of Engineering/Key Laboratory of Agricultural Equipment for the Middle and Lower Reaches of the Yangtze River, Ministry of Agriculture, Huazhong Agricultural University, Wuhan, ChinaIntroductionApplying 3D reconstruction techniques to individual plants has enhanced high-throughput phenotyping and provided accurate data support for developing "digital twins" in the agricultural domain. High costs, slow processing times, intricate workflows, and limited automation often constrain the application of existing 3D reconstruction platforms.MethodsWe develop a 3D reconstruction platform for complex plants to overcome these issues. Initially, a video acquisition system is built based on "camera to plant" mode. Then, we extract the keyframes in the videos. After that, Zhang Zhengyou's calibration method and Structure from Motion(SfM)are utilized to estimate the camera parameters. Next, Camera poses estimated from SfM were automatically calibrated using camera imaging trajectories as prior knowledge. Finally, Object-Based NeRF we proposed is utilized for the fine-scale reconstruction of plants. The OB-NeRF algorithm introduced a new ray sampling strategy that improved the efficiency and quality of target plant reconstruction without segmenting the background of images. Furthermore, the precision of the reconstruction was enhanced by optimizing camera poses. An exposure adjustment phase was integrated to improve the algorithm's robustness in uneven lighting conditions. The training process was significantly accelerated through the use of shallow MLP and multi-resolution hash encoding. Lastly, the camera imaging trajectories contributed to the automatic localization of target plants within the scene, enabling the automated extraction of Mesh. Results and discussionOur pipeline reconstructed high-quality neural radiance fields of the target plant from captured videos in just 250 seconds, enabling the synthesis of novel viewpoint images and the extraction of Mesh. OB-NeRF surpasses NeRF in PSNR evaluation and reduces the reconstruction time from over 10 hours to just 30 Seconds. Compared to Instant-NGP, NeRFacto, and NeuS, OB-NeRF achieves higher reconstruction quality in a shorter reconstruction time. Moreover, Our reconstructed 3D model demonstrated superior texture and geometric fidelity compared to those generated by COLMAP and Kinect-based reconstruction methods. The $R^2$ was 0.9933,0.9881 and 0.9883 for plant height, leaf length, and leaf width, respectively. The MAE was 2.0947, 0.1898, and 0.1199 cm. The 3D reconstruction platform introduced in this study provides a robust foundation for high-throughput phenotyping and the creation of agricultural “digital twins”.https://www.frontiersin.org/articles/10.3389/fpls.2025.1449626/fullneural radiance fields3D reconstructionplant phenotypingdigital twinsmesh |
| spellingShingle | Sixiao Wu Changhao Hu Boyuan Tian Yuan Huang Shuo Yang Shanjun Li Shengyong Xu A 3D reconstruction platform for complex plants using OB-NeRF Frontiers in Plant Science neural radiance fields 3D reconstruction plant phenotyping digital twins mesh |
| title | A 3D reconstruction platform for complex plants using OB-NeRF |
| title_full | A 3D reconstruction platform for complex plants using OB-NeRF |
| title_fullStr | A 3D reconstruction platform for complex plants using OB-NeRF |
| title_full_unstemmed | A 3D reconstruction platform for complex plants using OB-NeRF |
| title_short | A 3D reconstruction platform for complex plants using OB-NeRF |
| title_sort | 3d reconstruction platform for complex plants using ob nerf |
| topic | neural radiance fields 3D reconstruction plant phenotyping digital twins mesh |
| url | https://www.frontiersin.org/articles/10.3389/fpls.2025.1449626/full |
| work_keys_str_mv | AT sixiaowu a3dreconstructionplatformforcomplexplantsusingobnerf AT changhaohu a3dreconstructionplatformforcomplexplantsusingobnerf AT boyuantian a3dreconstructionplatformforcomplexplantsusingobnerf AT yuanhuang a3dreconstructionplatformforcomplexplantsusingobnerf AT shuoyang a3dreconstructionplatformforcomplexplantsusingobnerf AT shanjunli a3dreconstructionplatformforcomplexplantsusingobnerf AT shengyongxu a3dreconstructionplatformforcomplexplantsusingobnerf AT sixiaowu 3dreconstructionplatformforcomplexplantsusingobnerf AT changhaohu 3dreconstructionplatformforcomplexplantsusingobnerf AT boyuantian 3dreconstructionplatformforcomplexplantsusingobnerf AT yuanhuang 3dreconstructionplatformforcomplexplantsusingobnerf AT shuoyang 3dreconstructionplatformforcomplexplantsusingobnerf AT shanjunli 3dreconstructionplatformforcomplexplantsusingobnerf AT shengyongxu 3dreconstructionplatformforcomplexplantsusingobnerf |