An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robots
Accurate detection and localization of fruits in natural environments is a key step for fruit picking robots to achieve precise harvesting. However, existing banana detection and positioning methods have two main limitations in practical applications: a large number of model parameters that make dep...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | Smart Agricultural Technology |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2772375524001552 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850061525277999104 |
|---|---|
| author | Zhenhui Zheng Ling Chen Lijiao Wei Weihua Huang Dongjie Du Guoquan Qin Zhou Yang Shuo Wang |
| author_facet | Zhenhui Zheng Ling Chen Lijiao Wei Weihua Huang Dongjie Du Guoquan Qin Zhou Yang Shuo Wang |
| author_sort | Zhenhui Zheng |
| collection | DOAJ |
| description | Accurate detection and localization of fruits in natural environments is a key step for fruit picking robots to achieve precise harvesting. However, existing banana detection and positioning methods have two main limitations in practical applications: a large number of model parameters that make deployment difficult, and a need for performance improvement. To tackle the above issues, a high-precision and lightweight banana bunch recognition and localization method was proposed and deployed on edge devices for application. First, a Slim-Banana model was proposed based on the improvement of YOLOv8l. In order to reduce the model calculation amount and maintain high performance, GSConv was introduced in the Slim-Banana model to replace the standard convolution, and combined with grouped convolution and spatial convolution. At the same time, the cross-stage local network (GSCSP) module was designed to reduce the computational complexity and the complexity of the network structure through a single-stage aggregation method. Then, the RealSense depth sensor is combined with TOF technology to perform image registration and 3D localization of the banana. Finally, the pipeline is deployed on the Nvidia Orin NX edge device and its performance and resource consumption in actual work are deeply analyzed. Experimental results show that the detection precision, recall, mAP and inference time of our method are 0.947, 0.948, 0.98 and 113.6 ms respectively, the network memory size required is 4449MiB, and the average localization errors in the X-axis, Y-axis and Z-axis directions are 13.47 mm, 12.87 mm and 13.87 mm respectively. To our knowledge, this is the first work that implements banana detection and localization on edge devices. Experimental results show that compared with existing methods, our method achieves better performance in complex orchard environments, achieving efficient and lightweight banana recognition and localization. |
| format | Article |
| id | doaj-art-8eb328e8b218460f8f000ef9c5d41cab |
| institution | DOAJ |
| issn | 2772-3755 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Smart Agricultural Technology |
| spelling | doaj-art-8eb328e8b218460f8f000ef9c5d41cab2025-08-20T02:50:13ZengElsevierSmart Agricultural Technology2772-37552024-12-01910055010.1016/j.atech.2024.100550An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robotsZhenhui Zheng0Ling Chen1Lijiao Wei2Weihua Huang3Dongjie Du4Guoquan Qin5Zhou Yang6Shuo Wang7Institute of Agricultural Machinery, Chinese Academy of Tropical Agricultural Sciences, Guangdong, China; Key Laboratory of Agricultural Equipment for Tropical Crops, Ministry of Agriculture and Rural Affairs, Guangdong, China; Guangdong Engineering Technology Research Center of Precision Emission Control for Agricultural Particulates, Guangdong, China; Corresponding author.Institute of Agricultural Machinery, Chinese Academy of Tropical Agricultural Sciences, Guangdong, China; College of Mathematics and Informatics, South China Agricultural University, Guangdong, ChinaInstitute of Agricultural Machinery, Chinese Academy of Tropical Agricultural Sciences, Guangdong, ChinaInstitute of Agricultural Machinery, Chinese Academy of Tropical Agricultural Sciences, Guangdong, ChinaInstitute of Agricultural Machinery, Chinese Academy of Tropical Agricultural Sciences, Guangdong, ChinaCollege of Mathematics and Informatics, South China Agricultural University, Guangdong, ChinaCollege of Engineering, South China Agricultural University, Guangdong, ChinaInstitute of Agricultural Machinery, Chinese Academy of Tropical Agricultural Sciences, Guangdong, China; Key Laboratory of Agricultural Equipment for Tropical Crops, Ministry of Agriculture and Rural Affairs, Guangdong, China; Guangdong Engineering Technology Research Center of Precision Emission Control for Agricultural Particulates, Guangdong, ChinaAccurate detection and localization of fruits in natural environments is a key step for fruit picking robots to achieve precise harvesting. However, existing banana detection and positioning methods have two main limitations in practical applications: a large number of model parameters that make deployment difficult, and a need for performance improvement. To tackle the above issues, a high-precision and lightweight banana bunch recognition and localization method was proposed and deployed on edge devices for application. First, a Slim-Banana model was proposed based on the improvement of YOLOv8l. In order to reduce the model calculation amount and maintain high performance, GSConv was introduced in the Slim-Banana model to replace the standard convolution, and combined with grouped convolution and spatial convolution. At the same time, the cross-stage local network (GSCSP) module was designed to reduce the computational complexity and the complexity of the network structure through a single-stage aggregation method. Then, the RealSense depth sensor is combined with TOF technology to perform image registration and 3D localization of the banana. Finally, the pipeline is deployed on the Nvidia Orin NX edge device and its performance and resource consumption in actual work are deeply analyzed. Experimental results show that the detection precision, recall, mAP and inference time of our method are 0.947, 0.948, 0.98 and 113.6 ms respectively, the network memory size required is 4449MiB, and the average localization errors in the X-axis, Y-axis and Z-axis directions are 13.47 mm, 12.87 mm and 13.87 mm respectively. To our knowledge, this is the first work that implements banana detection and localization on edge devices. Experimental results show that compared with existing methods, our method achieves better performance in complex orchard environments, achieving efficient and lightweight banana recognition and localization.http://www.sciencedirect.com/science/article/pii/S2772375524001552BananaVisual detection3d localizationYOLOv8Lightweight |
| spellingShingle | Zhenhui Zheng Ling Chen Lijiao Wei Weihua Huang Dongjie Du Guoquan Qin Zhou Yang Shuo Wang An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robots Smart Agricultural Technology Banana Visual detection 3d localization YOLOv8 Lightweight |
| title | An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robots |
| title_full | An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robots |
| title_fullStr | An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robots |
| title_full_unstemmed | An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robots |
| title_short | An efficient and lightweight banana detection and localization system based on deep CNNs for agricultural robots |
| title_sort | efficient and lightweight banana detection and localization system based on deep cnns for agricultural robots |
| topic | Banana Visual detection 3d localization YOLOv8 Lightweight |
| url | http://www.sciencedirect.com/science/article/pii/S2772375524001552 |
| work_keys_str_mv | AT zhenhuizheng anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT lingchen anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT lijiaowei anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT weihuahuang anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT dongjiedu anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT guoquanqin anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT zhouyang anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT shuowang anefficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT zhenhuizheng efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT lingchen efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT lijiaowei efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT weihuahuang efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT dongjiedu efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT guoquanqin efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT zhouyang efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots AT shuowang efficientandlightweightbananadetectionandlocalizationsystembasedondeepcnnsforagriculturalrobots |