Assisting human annotation of marine images with foundation models
Marine scientists have been leveraging supervised machine learning algorithms to analyze image and video data for nearly two decades. There have been many advances, but the cost of generating expert human annotations to train new models remains extremely high. There is broad recognition both in comp...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-07-01
|
| Series: | Frontiers in Marine Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmars.2025.1469396/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850067842691497984 |
|---|---|
| author | Eric C. Orenstein Eric C. Orenstein Benjamin Woodward Lonny Lundsten Kevin Barnard Brian Schlining Kakani Katjia |
| author_facet | Eric C. Orenstein Eric C. Orenstein Benjamin Woodward Lonny Lundsten Kevin Barnard Brian Schlining Kakani Katjia |
| author_sort | Eric C. Orenstein |
| collection | DOAJ |
| description | Marine scientists have been leveraging supervised machine learning algorithms to analyze image and video data for nearly two decades. There have been many advances, but the cost of generating expert human annotations to train new models remains extremely high. There is broad recognition both in computer and domain sciences that generating training data remains the major bottleneck when developing ML models for targeted tasks. Increasingly, computer scientists are not attempting to produce highly-optimized models from general annotation frameworks, instead focusing on adaptation strategies to tackle new data challenges. Taking inspiration from large language models, computer vision researchers are now thinking in terms of “foundation models” that can yield reasonable zero- and few-shot detection and segmentation performance with human prompting. Here we consider the utility of this approach for ocean imagery, leveraging Meta’s Segment Anything Model to enrich ocean image annotations based on existing labels. This workflow yields promising results, especially for modernizing existing data repositories. Moreover, it suggests that future human annotation efforts could use foundation models to speed progress toward a sufficient training set to address domain specific problems. |
| format | Article |
| id | doaj-art-12c7e49938d44a0492247d00a3526e9e |
| institution | DOAJ |
| issn | 2296-7745 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Marine Science |
| spelling | doaj-art-12c7e49938d44a0492247d00a3526e9e2025-08-20T02:48:12ZengFrontiers Media S.A.Frontiers in Marine Science2296-77452025-07-011210.3389/fmars.2025.14693961469396Assisting human annotation of marine images with foundation modelsEric C. Orenstein0Eric C. Orenstein1Benjamin Woodward2Lonny Lundsten3Kevin Barnard4Brian Schlining5Kakani Katjia6Research and Development, Information and Technology Dissemination, Monterey Bay Aquarium Research Institute, Moss Landing, CA, United StatesResearch and Development, National Oceanography Centre, Southampton, United KingdomResearch and Development, CVision AI, Medford, MA, United StatesResearch and Development, Information and Technology Dissemination, Monterey Bay Aquarium Research Institute, Moss Landing, CA, United StatesResearch and Development, Information and Technology Dissemination, Monterey Bay Aquarium Research Institute, Moss Landing, CA, United StatesResearch and Development, Information and Technology Dissemination, Monterey Bay Aquarium Research Institute, Moss Landing, CA, United StatesResearch and Development, Information and Technology Dissemination, Monterey Bay Aquarium Research Institute, Moss Landing, CA, United StatesMarine scientists have been leveraging supervised machine learning algorithms to analyze image and video data for nearly two decades. There have been many advances, but the cost of generating expert human annotations to train new models remains extremely high. There is broad recognition both in computer and domain sciences that generating training data remains the major bottleneck when developing ML models for targeted tasks. Increasingly, computer scientists are not attempting to produce highly-optimized models from general annotation frameworks, instead focusing on adaptation strategies to tackle new data challenges. Taking inspiration from large language models, computer vision researchers are now thinking in terms of “foundation models” that can yield reasonable zero- and few-shot detection and segmentation performance with human prompting. Here we consider the utility of this approach for ocean imagery, leveraging Meta’s Segment Anything Model to enrich ocean image annotations based on existing labels. This workflow yields promising results, especially for modernizing existing data repositories. Moreover, it suggests that future human annotation efforts could use foundation models to speed progress toward a sufficient training set to address domain specific problems.https://www.frontiersin.org/articles/10.3389/fmars.2025.1469396/fullfoundation modelmarine imagerysegmentationobject detectionhuman-in-the-loop |
| spellingShingle | Eric C. Orenstein Eric C. Orenstein Benjamin Woodward Lonny Lundsten Kevin Barnard Brian Schlining Kakani Katjia Assisting human annotation of marine images with foundation models Frontiers in Marine Science foundation model marine imagery segmentation object detection human-in-the-loop |
| title | Assisting human annotation of marine images with foundation models |
| title_full | Assisting human annotation of marine images with foundation models |
| title_fullStr | Assisting human annotation of marine images with foundation models |
| title_full_unstemmed | Assisting human annotation of marine images with foundation models |
| title_short | Assisting human annotation of marine images with foundation models |
| title_sort | assisting human annotation of marine images with foundation models |
| topic | foundation model marine imagery segmentation object detection human-in-the-loop |
| url | https://www.frontiersin.org/articles/10.3389/fmars.2025.1469396/full |
| work_keys_str_mv | AT ericcorenstein assistinghumanannotationofmarineimageswithfoundationmodels AT ericcorenstein assistinghumanannotationofmarineimageswithfoundationmodels AT benjaminwoodward assistinghumanannotationofmarineimageswithfoundationmodels AT lonnylundsten assistinghumanannotationofmarineimageswithfoundationmodels AT kevinbarnard assistinghumanannotationofmarineimageswithfoundationmodels AT brianschlining assistinghumanannotationofmarineimageswithfoundationmodels AT kakanikatjia assistinghumanannotationofmarineimageswithfoundationmodels |