SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical Images
It is challenging to achieve accurate tea bud detection in optical images with complex backgrounds since distinguishing between the foregrounds and backgrounds of these images remains difficult. Although several studies have been proposed to implicitly distinguish foregrounds and backgrounds via var...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Agriculture |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-0472/15/7/712 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850184542802935808 |
|---|---|
| author | Xintong Zhang Dasheng Wu Fengya Xu |
| author_facet | Xintong Zhang Dasheng Wu Fengya Xu |
| author_sort | Xintong Zhang |
| collection | DOAJ |
| description | It is challenging to achieve accurate tea bud detection in optical images with complex backgrounds since distinguishing between the foregrounds and backgrounds of these images remains difficult. Although several studies have been proposed to implicitly distinguish foregrounds and backgrounds via various attention mechanisms, explicit distinction between foregrounds and backgrounds has been seldom explored. Inspired by recent successful applications of the Segment Anything Model (SAM) in computer vision, this study proposes a SAM-assisted dual-branch YOLOv8 model named SD-YOLOv8 for tea bud detection to address the challenges of explicit distinction between foregrounds and backgrounds. The SD-YOLOv8 model mainly consists of two key components: (1) the SAM-based foreground segmenter (SFS) to generate foreground masks of tea bud images without any training, and (2) the heterogeneous feature extractor to parallelly capture both color features in optical images and edge features in foreground masks. The experimental results show that the proposed SD-YOLOv8 significantly improves the performance of tea bud detection based on the explicit distinction between foregrounds and backgrounds. The mean Average Precision of the SD-YOLOv8 model reaches 86.0%, surpassing the YOLOv8 (mAP 81.6%) by 5 percentage points and outperforming recent object detection models, including Faster R-CNN (mAP 60.7%), DETR (mAP 64.6%), YOLOv5 (mAP 72.4%), and YOLOv7 (mAP 80.6%). This demonstrates its superior capability in efficiently detecting tea buds against complex backgrounds. Additionally, this study proposes a self-built tea bud dataset with three seasons to address the data shortages in tea bud detection. |
| format | Article |
| id | doaj-art-6609df2c05ab4811a559a7ecfd21c59c |
| institution | OA Journals |
| issn | 2077-0472 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Agriculture |
| spelling | doaj-art-6609df2c05ab4811a559a7ecfd21c59c2025-08-20T02:17:00ZengMDPI AGAgriculture2077-04722025-03-0115771210.3390/agriculture15070712SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical ImagesXintong Zhang0Dasheng Wu1Fengya Xu2College of Mathematics and Computer Science, Zhejiang A&F University, Hangzhou 311300, ChinaCollege of Mathematics and Computer Science, Zhejiang A&F University, Hangzhou 311300, ChinaCollege of Mathematics and Computer Science, Zhejiang A&F University, Hangzhou 311300, ChinaIt is challenging to achieve accurate tea bud detection in optical images with complex backgrounds since distinguishing between the foregrounds and backgrounds of these images remains difficult. Although several studies have been proposed to implicitly distinguish foregrounds and backgrounds via various attention mechanisms, explicit distinction between foregrounds and backgrounds has been seldom explored. Inspired by recent successful applications of the Segment Anything Model (SAM) in computer vision, this study proposes a SAM-assisted dual-branch YOLOv8 model named SD-YOLOv8 for tea bud detection to address the challenges of explicit distinction between foregrounds and backgrounds. The SD-YOLOv8 model mainly consists of two key components: (1) the SAM-based foreground segmenter (SFS) to generate foreground masks of tea bud images without any training, and (2) the heterogeneous feature extractor to parallelly capture both color features in optical images and edge features in foreground masks. The experimental results show that the proposed SD-YOLOv8 significantly improves the performance of tea bud detection based on the explicit distinction between foregrounds and backgrounds. The mean Average Precision of the SD-YOLOv8 model reaches 86.0%, surpassing the YOLOv8 (mAP 81.6%) by 5 percentage points and outperforming recent object detection models, including Faster R-CNN (mAP 60.7%), DETR (mAP 64.6%), YOLOv5 (mAP 72.4%), and YOLOv7 (mAP 80.6%). This demonstrates its superior capability in efficiently detecting tea buds against complex backgrounds. Additionally, this study proposes a self-built tea bud dataset with three seasons to address the data shortages in tea bud detection.https://www.mdpi.com/2077-0472/15/7/712agricultural computer visionheterogeneous feature extractionzero-shot segmentationdeep learning |
| spellingShingle | Xintong Zhang Dasheng Wu Fengya Xu SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical Images Agriculture agricultural computer vision heterogeneous feature extraction zero-shot segmentation deep learning |
| title | SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical Images |
| title_full | SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical Images |
| title_fullStr | SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical Images |
| title_full_unstemmed | SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical Images |
| title_short | SD-YOLOv8: SAM-Assisted Dual-Branch YOLOv8 Model for Tea Bud Detection on Optical Images |
| title_sort | sd yolov8 sam assisted dual branch yolov8 model for tea bud detection on optical images |
| topic | agricultural computer vision heterogeneous feature extraction zero-shot segmentation deep learning |
| url | https://www.mdpi.com/2077-0472/15/7/712 |
| work_keys_str_mv | AT xintongzhang sdyolov8samassisteddualbranchyolov8modelforteabuddetectiononopticalimages AT dashengwu sdyolov8samassisteddualbranchyolov8modelforteabuddetectiononopticalimages AT fengyaxu sdyolov8samassisteddualbranchyolov8modelforteabuddetectiononopticalimages |