PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field images
Abstract Plant phenology plays a fundamental role in shaping ecosystems, and global change‐induced shifts in phenology have cascading impacts on species interactions and ecosystem structure and function. Detailed, high‐quality observations of when plants undergo seasonal transitions such as leaf‐out...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2025-08-01
|
| Series: | Methods in Ecology and Evolution |
| Subjects: | |
| Online Access: | https://doi.org/10.1111/2041-210X.70081 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849338092164481024 |
|---|---|
| author | Russell Dinnage Erin Grady Nevyn Neal Jonn Deck Ellen Denny Ramona Walls Carrie Seltzer Robert Guralnick Daijiang Li |
| author_facet | Russell Dinnage Erin Grady Nevyn Neal Jonn Deck Ellen Denny Ramona Walls Carrie Seltzer Robert Guralnick Daijiang Li |
| author_sort | Russell Dinnage |
| collection | DOAJ |
| description | Abstract Plant phenology plays a fundamental role in shaping ecosystems, and global change‐induced shifts in phenology have cascading impacts on species interactions and ecosystem structure and function. Detailed, high‐quality observations of when plants undergo seasonal transitions such as leaf‐out, flowering and fruiting are critical for tracking causes and consequences of phenology shifts, but these data are often sparse and biased globally. These data gaps limit broader generalizations and forecasting improvements in the face of continuing disturbance. One solution to closing such gaps is to document phenology on field images taken by public participants. iNaturalist, in particular, provides global‐scale research‐grade data and is expanding rapidly. Here we utilize over 53 million field images of plants and millions of human annotations from iNaturalist—data spanning all angiosperms and drawn from across the globe—to train a computer vision model (PhenoVision) to detect the presence of fruits and flowers. PhenoVision utilizes a vision transformer architecture pretrained with a masked autoencoder to improve classification success, and it achieves high accuracy on held‐out test images for flower (98.5%) and fruit presence (95%), as well as a high level of agreement with an expert annotator (98.6% for flowers and 90.4% for fruits). Key to producing research‐ready phenology data is post‐calibration tuning and validation focused on reducing noise inherent in field photographs, and maximizing the true positive rate. We also develop a standardized set of quality metrics and metadata so that results can be used effectively by the community. Finally, we showcase how this effort vastly increases phenology data coverage, including regions of the globe where data have been limited before. Our end products are tuned models, new data resources and an application streamlining discovery and use of those data for the broader research and management community. We close by discussing next steps, including automating phenology annotations, adding new phenology targets, for example leaf phenology, and further integration with other resources to form a global central database integrating all in situ plant phenology resources. |
| format | Article |
| id | doaj-art-cebacabcc93f43109792b70f89541ee7 |
| institution | Kabale University |
| issn | 2041-210X |
| language | English |
| publishDate | 2025-08-01 |
| publisher | Wiley |
| record_format | Article |
| series | Methods in Ecology and Evolution |
| spelling | doaj-art-cebacabcc93f43109792b70f89541ee72025-08-20T03:44:31ZengWileyMethods in Ecology and Evolution2041-210X2025-08-011681763178010.1111/2041-210X.70081PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field imagesRussell Dinnage0Erin Grady1Nevyn Neal2Jonn Deck3Ellen Denny4Ramona Walls5Carrie Seltzer6Robert Guralnick7Daijiang Li8Institute of Environment, Department of Biological Sciences Florida International University Miami Florida USAFlorida Museum of Natural History University of Florida Gainesville Florida USADepartment of Ecology and Evolutionary Biology University of Arizona Tucson Arizona USABerkeley Natural History Museums University of California Berkeley California USAUSA National Phenology Network, School of Natural Resources and the Environment University of Arizona Tucson Arizona USAData Collaboration Center Critical Path Institute Tucson Arizona USAiNaturalist San Rafael California USAFlorida Museum of Natural History University of Florida Gainesville Florida USADepartment of Ecology and Evolutionary Biology University of Arizona Tucson Arizona USAAbstract Plant phenology plays a fundamental role in shaping ecosystems, and global change‐induced shifts in phenology have cascading impacts on species interactions and ecosystem structure and function. Detailed, high‐quality observations of when plants undergo seasonal transitions such as leaf‐out, flowering and fruiting are critical for tracking causes and consequences of phenology shifts, but these data are often sparse and biased globally. These data gaps limit broader generalizations and forecasting improvements in the face of continuing disturbance. One solution to closing such gaps is to document phenology on field images taken by public participants. iNaturalist, in particular, provides global‐scale research‐grade data and is expanding rapidly. Here we utilize over 53 million field images of plants and millions of human annotations from iNaturalist—data spanning all angiosperms and drawn from across the globe—to train a computer vision model (PhenoVision) to detect the presence of fruits and flowers. PhenoVision utilizes a vision transformer architecture pretrained with a masked autoencoder to improve classification success, and it achieves high accuracy on held‐out test images for flower (98.5%) and fruit presence (95%), as well as a high level of agreement with an expert annotator (98.6% for flowers and 90.4% for fruits). Key to producing research‐ready phenology data is post‐calibration tuning and validation focused on reducing noise inherent in field photographs, and maximizing the true positive rate. We also develop a standardized set of quality metrics and metadata so that results can be used effectively by the community. Finally, we showcase how this effort vastly increases phenology data coverage, including regions of the globe where data have been limited before. Our end products are tuned models, new data resources and an application streamlining discovery and use of those data for the broader research and management community. We close by discussing next steps, including automating phenology annotations, adding new phenology targets, for example leaf phenology, and further integration with other resources to form a global central database integrating all in situ plant phenology resources.https://doi.org/10.1111/2041-210X.70081automated annotationcommunity science datacomputer visiondata integrationmachine learningmasked autoencoders |
| spellingShingle | Russell Dinnage Erin Grady Nevyn Neal Jonn Deck Ellen Denny Ramona Walls Carrie Seltzer Robert Guralnick Daijiang Li PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field images Methods in Ecology and Evolution automated annotation community science data computer vision data integration machine learning masked autoencoders |
| title | PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field images |
| title_full | PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field images |
| title_fullStr | PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field images |
| title_full_unstemmed | PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field images |
| title_short | PhenoVision: A framework for automating and delivering research‐ready plant phenology data from field images |
| title_sort | phenovision a framework for automating and delivering research ready plant phenology data from field images |
| topic | automated annotation community science data computer vision data integration machine learning masked autoencoders |
| url | https://doi.org/10.1111/2041-210X.70081 |
| work_keys_str_mv | AT russelldinnage phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT eringrady phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT nevynneal phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT jonndeck phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT ellendenny phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT ramonawalls phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT carrieseltzer phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT robertguralnick phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages AT daijiangli phenovisionaframeworkforautomatinganddeliveringresearchreadyplantphenologydatafromfieldimages |