Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion
Abstract Tapping line detection and rubber tapping pose estimation are challenging tasks in rubber plantation environments for rubber tapping robots. This study proposed a method for tapping line detection and rubber tapping pose estimation based on improved YOLOv8 and RGB-D information fusion. Firs...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-11-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-79132-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850129190333972480 |
|---|---|
| author | Yaya Chen Hui Yang Junxiao Liu Zhifu Zhang Xirui Zhang |
| author_facet | Yaya Chen Hui Yang Junxiao Liu Zhifu Zhang Xirui Zhang |
| author_sort | Yaya Chen |
| collection | DOAJ |
| description | Abstract Tapping line detection and rubber tapping pose estimation are challenging tasks in rubber plantation environments for rubber tapping robots. This study proposed a method for tapping line detection and rubber tapping pose estimation based on improved YOLOv8 and RGB-D information fusion. Firstly, YOLOv8n was improved by introducing the CFB module into the backbone, adding an output layer into the neck, fusing the EMA attention mechanism into the neck, and modifying the loss function as NWD to realize multi-object detection and segmentation. Secondly, the trunk skeleton line was extracted by combining level set and ellipse fitting. Then, the new tapping line was located by combining edge detection and geometric analysis. Finally, the rubber tapping pose was estimated based on the trunk skeleton line and the new tapping line. The detection results from 597 test images showed the improved YOLOv8n’s detection mAP0.5, segmentation mAP0.5, and model size were 81.9%, 72.9%, and 6.06 MB, respectively. The improved YOLOv8n’s effect and efficiency were superior compared to other networks, and it could better detect and segment natural rubber tree image targets in different scenes. The pose estimation results from 300 new tapping lines showed the average success rate and average time consumed for rubber tapping pose estimation were 96% and 0.2818 s, respectively. The positioning errors in x, y, and z directions were 0.69 ± 0.51 mm, 0.73 ± 0.4 mm, and 1.07 ± 0.56 mm, respectively. The error angles in a, o, and n directions were 1.65° ± 0.68°, 2.53° ± 0.88°, and 2.26° ± 0.89°, respectively. Therefore, this method offers an effective solution for rubber tapping pose estimation and provides theoretical support for the development of rubber tapping robots. |
| format | Article |
| id | doaj-art-1b8b5f8ae29846648f82c00a98376390 |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-1b8b5f8ae29846648f82c00a983763902025-08-20T02:33:05ZengNature PortfolioScientific Reports2045-23222024-11-0114112310.1038/s41598-024-79132-5Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusionYaya Chen0Hui Yang1Junxiao Liu2Zhifu Zhang3Xirui Zhang4School of Information and Communication Engineering, Hainan UniversityMechanical and Electrical Engineering College, Hainan UniversityMechanical and Electrical Engineering College, Hainan UniversityMechanical and Electrical Engineering College, Hainan UniversitySchool of Information and Communication Engineering, Hainan UniversityAbstract Tapping line detection and rubber tapping pose estimation are challenging tasks in rubber plantation environments for rubber tapping robots. This study proposed a method for tapping line detection and rubber tapping pose estimation based on improved YOLOv8 and RGB-D information fusion. Firstly, YOLOv8n was improved by introducing the CFB module into the backbone, adding an output layer into the neck, fusing the EMA attention mechanism into the neck, and modifying the loss function as NWD to realize multi-object detection and segmentation. Secondly, the trunk skeleton line was extracted by combining level set and ellipse fitting. Then, the new tapping line was located by combining edge detection and geometric analysis. Finally, the rubber tapping pose was estimated based on the trunk skeleton line and the new tapping line. The detection results from 597 test images showed the improved YOLOv8n’s detection mAP0.5, segmentation mAP0.5, and model size were 81.9%, 72.9%, and 6.06 MB, respectively. The improved YOLOv8n’s effect and efficiency were superior compared to other networks, and it could better detect and segment natural rubber tree image targets in different scenes. The pose estimation results from 300 new tapping lines showed the average success rate and average time consumed for rubber tapping pose estimation were 96% and 0.2818 s, respectively. The positioning errors in x, y, and z directions were 0.69 ± 0.51 mm, 0.73 ± 0.4 mm, and 1.07 ± 0.56 mm, respectively. The error angles in a, o, and n directions were 1.65° ± 0.68°, 2.53° ± 0.88°, and 2.26° ± 0.89°, respectively. Therefore, this method offers an effective solution for rubber tapping pose estimation and provides theoretical support for the development of rubber tapping robots.https://doi.org/10.1038/s41598-024-79132-5Natural rubber treeDeep learningObject detectionPoint cloudRubber tapping pose estimation |
| spellingShingle | Yaya Chen Hui Yang Junxiao Liu Zhifu Zhang Xirui Zhang Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion Scientific Reports Natural rubber tree Deep learning Object detection Point cloud Rubber tapping pose estimation |
| title | Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion |
| title_full | Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion |
| title_fullStr | Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion |
| title_full_unstemmed | Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion |
| title_short | Tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved YOLOv8 and RGB-D information fusion |
| title_sort | tapping line detection and rubber tapping pose estimation for natural rubber trees based on improved yolov8 and rgb d information fusion |
| topic | Natural rubber tree Deep learning Object detection Point cloud Rubber tapping pose estimation |
| url | https://doi.org/10.1038/s41598-024-79132-5 |
| work_keys_str_mv | AT yayachen tappinglinedetectionandrubbertappingposeestimationfornaturalrubbertreesbasedonimprovedyolov8andrgbdinformationfusion AT huiyang tappinglinedetectionandrubbertappingposeestimationfornaturalrubbertreesbasedonimprovedyolov8andrgbdinformationfusion AT junxiaoliu tappinglinedetectionandrubbertappingposeestimationfornaturalrubbertreesbasedonimprovedyolov8andrgbdinformationfusion AT zhifuzhang tappinglinedetectionandrubbertappingposeestimationfornaturalrubbertreesbasedonimprovedyolov8andrgbdinformationfusion AT xiruizhang tappinglinedetectionandrubbertappingposeestimationfornaturalrubbertreesbasedonimprovedyolov8andrgbdinformationfusion |