Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks
With the rapid advancements in computer vision and deep learning, multi-view stereo (MVS) based on conventional RGB cameras has emerged as a promising and cost-effective tool for botanical research. However, existing methods often struggle to capture the intricate textures and fine edges of plants,...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/7/2177 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850188311862181888 |
|---|---|
| author | Yang Cheng Zhen Liu Gongpu Lan Jingjiang Xu Ren Chen Yanping Huang |
| author_facet | Yang Cheng Zhen Liu Gongpu Lan Jingjiang Xu Ren Chen Yanping Huang |
| author_sort | Yang Cheng |
| collection | DOAJ |
| description | With the rapid advancements in computer vision and deep learning, multi-view stereo (MVS) based on conventional RGB cameras has emerged as a promising and cost-effective tool for botanical research. However, existing methods often struggle to capture the intricate textures and fine edges of plants, resulting in suboptimal 3D reconstruction accuracy. To overcome this challenge, we proposed Edge_MVSFormer on the basis of TransMVSNet, which particularly focuses on enhancing the accuracy of plant leaf edge reconstruction. This model integrates an edge detection algorithm to augment edge information as input to the network and introduces an edge-aware loss function to focus the network’s attention on a more accurate reconstruction of edge regions, where depth estimation errors are obviously more significant. Edge_MVSFormer was pre-trained on two public MVS datasets and fine-tuned with our private data of 10 model plants collected for this study. Experimental results on 10 test model plants demonstrated that for depth images, the proposed algorithm reduces the edge error and overall reconstruction error by 2.20 ± 0.36 mm and 0.46 ± 0.07 mm, respectively. For point clouds, the edge and overall reconstruction errors were reduced by 0.13 ± 0.02 mm and 0.05 ± 0.02 mm, respectively. This study underscores the critical role of edge information in the precise reconstruction of plant MVS data. |
| format | Article |
| id | doaj-art-fd2762261eb34f178de15e11dc22e3e6 |
| institution | OA Journals |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-fd2762261eb34f178de15e11dc22e3e62025-08-20T02:15:54ZengMDPI AGSensors1424-82202025-03-01257217710.3390/s25072177Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer NetworksYang Cheng0Zhen Liu1Gongpu Lan2Jingjiang Xu3Ren Chen4Yanping Huang5Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, ChinaGuangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, ChinaGuangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, ChinaGuangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, ChinaGuangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, ChinaGuangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, ChinaWith the rapid advancements in computer vision and deep learning, multi-view stereo (MVS) based on conventional RGB cameras has emerged as a promising and cost-effective tool for botanical research. However, existing methods often struggle to capture the intricate textures and fine edges of plants, resulting in suboptimal 3D reconstruction accuracy. To overcome this challenge, we proposed Edge_MVSFormer on the basis of TransMVSNet, which particularly focuses on enhancing the accuracy of plant leaf edge reconstruction. This model integrates an edge detection algorithm to augment edge information as input to the network and introduces an edge-aware loss function to focus the network’s attention on a more accurate reconstruction of edge regions, where depth estimation errors are obviously more significant. Edge_MVSFormer was pre-trained on two public MVS datasets and fine-tuned with our private data of 10 model plants collected for this study. Experimental results on 10 test model plants demonstrated that for depth images, the proposed algorithm reduces the edge error and overall reconstruction error by 2.20 ± 0.36 mm and 0.46 ± 0.07 mm, respectively. For point clouds, the edge and overall reconstruction errors were reduced by 0.13 ± 0.02 mm and 0.05 ± 0.02 mm, respectively. This study underscores the critical role of edge information in the precise reconstruction of plant MVS data.https://www.mdpi.com/1424-8220/25/7/2177plant 3D reconstructionmulti-view stereodepth imagepoint cloudedge regionsedge detection |
| spellingShingle | Yang Cheng Zhen Liu Gongpu Lan Jingjiang Xu Ren Chen Yanping Huang Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks Sensors plant 3D reconstruction multi-view stereo depth image point cloud edge regions edge detection |
| title | Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks |
| title_full | Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks |
| title_fullStr | Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks |
| title_full_unstemmed | Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks |
| title_short | Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks |
| title_sort | edge mvsformer edge aware multi view stereo plant reconstruction based on transformer networks |
| topic | plant 3D reconstruction multi-view stereo depth image point cloud edge regions edge detection |
| url | https://www.mdpi.com/1424-8220/25/7/2177 |
| work_keys_str_mv | AT yangcheng edgemvsformeredgeawaremultiviewstereoplantreconstructionbasedontransformernetworks AT zhenliu edgemvsformeredgeawaremultiviewstereoplantreconstructionbasedontransformernetworks AT gongpulan edgemvsformeredgeawaremultiviewstereoplantreconstructionbasedontransformernetworks AT jingjiangxu edgemvsformeredgeawaremultiviewstereoplantreconstructionbasedontransformernetworks AT renchen edgemvsformeredgeawaremultiviewstereoplantreconstructionbasedontransformernetworks AT yanpinghuang edgemvsformeredgeawaremultiviewstereoplantreconstructionbasedontransformernetworks |