Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot
Harvesting grapes requires a large amount of manual labor. To reduce the labor force for the harvesting job, in this study, we developed a robot harvester for the vine grapes. In this paper, we proposed an algorithm that using multi-cameras, as well as artificial intelligence (AI) object detection m...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/24/8035 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846102767610888192 |
|---|---|
| author | Liangliang Yang Tomoki Noguchi Yohei Hoshino |
| author_facet | Liangliang Yang Tomoki Noguchi Yohei Hoshino |
| author_sort | Liangliang Yang |
| collection | DOAJ |
| description | Harvesting grapes requires a large amount of manual labor. To reduce the labor force for the harvesting job, in this study, we developed a robot harvester for the vine grapes. In this paper, we proposed an algorithm that using multi-cameras, as well as artificial intelligence (AI) object detection methods, to detect the thin stem and decide the cut point. The camera system was constructed by two cameras that include multi-lenses. One camera is mounted at the base of the robot and named the “base camera”; the other camera is mounted at the robot hand and named the “hand camera” to recognize grapes and estimate the stem position. At the first step, the grapes are detected by using a You Only Look Once (YOLO) method, while the stems of the grapes are detected at the second step using a pixel-level semantic segmentation method. Field experiments were conducted at an outdoor grapes field. The experiment results show that the proposed algorithm and the camera system can successfully detect out the cut point, and the correct detection rate is around 98% and 93% in the indoor and outdoor conditions, respectively. The detection system was integrated to a grape-harvesting robot in the experiment, and the experiment results show the system can successfully harvest the grapes in the outdoor conditions. |
| format | Article |
| id | doaj-art-238981ffa292414485daa67ef99db6cd |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-238981ffa292414485daa67ef99db6cd2024-12-27T14:52:47ZengMDPI AGSensors1424-82202024-12-012424803510.3390/s24248035Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting RobotLiangliang Yang0Tomoki Noguchi1Yohei Hoshino2Laboratory of Bio-Mechatronics, Faculty of Engineering, Kitami Institute of Technology, Koentyo 165, Kitami Shi 090-8507, Hokkaido, JapanLaboratory of Bio-Mechatronics, Faculty of Engineering, Kitami Institute of Technology, Koentyo 165, Kitami Shi 090-8507, Hokkaido, JapanLaboratory of Bio-Mechatronics, Faculty of Engineering, Kitami Institute of Technology, Koentyo 165, Kitami Shi 090-8507, Hokkaido, JapanHarvesting grapes requires a large amount of manual labor. To reduce the labor force for the harvesting job, in this study, we developed a robot harvester for the vine grapes. In this paper, we proposed an algorithm that using multi-cameras, as well as artificial intelligence (AI) object detection methods, to detect the thin stem and decide the cut point. The camera system was constructed by two cameras that include multi-lenses. One camera is mounted at the base of the robot and named the “base camera”; the other camera is mounted at the robot hand and named the “hand camera” to recognize grapes and estimate the stem position. At the first step, the grapes are detected by using a You Only Look Once (YOLO) method, while the stems of the grapes are detected at the second step using a pixel-level semantic segmentation method. Field experiments were conducted at an outdoor grapes field. The experiment results show that the proposed algorithm and the camera system can successfully detect out the cut point, and the correct detection rate is around 98% and 93% in the indoor and outdoor conditions, respectively. The detection system was integrated to a grape-harvesting robot in the experiment, and the experiment results show the system can successfully harvest the grapes in the outdoor conditions.https://www.mdpi.com/1424-8220/24/24/8035grape-harvesting robotmulti-camera systemyou only look once (YOLO) based grapes detectionstem detection using semantic segmentation |
| spellingShingle | Liangliang Yang Tomoki Noguchi Yohei Hoshino Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot Sensors grape-harvesting robot multi-camera system you only look once (YOLO) based grapes detection stem detection using semantic segmentation |
| title | Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot |
| title_full | Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot |
| title_fullStr | Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot |
| title_full_unstemmed | Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot |
| title_short | Development of a Grape Cut Point Detection System Using Multi-Cameras for a Grape-Harvesting Robot |
| title_sort | development of a grape cut point detection system using multi cameras for a grape harvesting robot |
| topic | grape-harvesting robot multi-camera system you only look once (YOLO) based grapes detection stem detection using semantic segmentation |
| url | https://www.mdpi.com/1424-8220/24/24/8035 |
| work_keys_str_mv | AT liangliangyang developmentofagrapecutpointdetectionsystemusingmulticamerasforagrapeharvestingrobot AT tomokinoguchi developmentofagrapecutpointdetectionsystemusingmulticamerasforagrapeharvestingrobot AT yoheihoshino developmentofagrapecutpointdetectionsystemusingmulticamerasforagrapeharvestingrobot |