Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology
Abstract Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they ma...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-02-01
|
| Series: | BMC Biology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12915-024-02101-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823861683289849856 |
|---|---|
| author | Elliot Gould Hannah S. Fraser Timothy H. Parker Shinichi Nakagawa Simon C. Griffith Peter A. Vesk Fiona Fidler Daniel G. Hamilton Robin N. Abbey-Lee Jessica K. Abbott Luis A. Aguirre Carles Alcaraz Irith Aloni Drew Altschul Kunal Arekar Jeff W. Atkins Joe Atkinson Christopher M. Baker Meghan Barrett Kristian Bell Suleiman Kehinde Bello Iván Beltrán Bernd J. Berauer Michael Grant Bertram Peter D. Billman Charlie K. Blake Shannon Blake Louis Bliard Andrea Bonisoli-Alquati Timothée Bonnet Camille Nina Marion Bordes Aneesh P. H. Bose Thomas Botterill-James Melissa Anna Boyd Sarah A. Boyle Tom Bradfer-Lawrence Jennifer Bradham Jack A. Brand Martin I. Brengdahl Martin Bulla Luc Bussière Ettore Camerlenghi Sara E. Campbell Leonardo L. F. Campos Anthony Caravaggi Pedro Cardoso Charles J. W. Carroll Therese A. Catanach Xuan Chen Heung Ying Janet Chik Emily Sarah Choy Alec Philip Christie Angela Chuang Amanda J. Chunco Bethany L. Clark Andrea Contina Garth A. Covernton Murray P. Cox Kimberly A. Cressman Marco Crotti Connor Davidson Crouch Pietro B. D’Amelio Alexandra Allison de Sousa Timm Fabian Döbert Ralph Dobler Adam J. Dobson Tim S. Doherty Szymon Marian Drobniak Alexandra Grace Duffy Alison B. Duncan Robert P. Dunn Jamie Dunning Trishna Dutta Luke Eberhart-Hertel Jared Alan Elmore Mahmoud Medhat Elsherif Holly M. English David C. Ensminger Ulrich Rainer Ernst Stephen M. Ferguson Esteban Fernandez-Juricic Thalita Ferreira-Arruda John Fieberg Elizabeth A. Finch Evan A. Fiorenza David N. Fisher Amélie Fontaine Wolfgang Forstmeier Yoan Fourcade Graham S. Frank Cathryn A. Freund Eduardo Fuentes-Lillo Sara L. Gandy Dustin G. Gannon Ana I. García-Cervigón Alexis C. Garretson Xuezhen Ge William L. Geary Charly Géron Marc Gilles Antje Girndt Daniel Gliksman Harrison B. Goldspiel Dylan G. E. Gomes Megan Kate Good Sarah C. Goslee J. Stephen Gosnell Eliza M. Grames Paolo Gratton Nicholas M. Grebe Skye M. Greenler Maaike Griffioen Daniel M. Griffith Frances J. Griffith Jake J. Grossman Ali Güncan Stef Haesen James G. Hagan Heather A. Hager Jonathan Philo Harris Natasha Dean Harrison Sarah Syedia Hasnain Justin Chase Havird Andrew J. Heaton María Laura Herrera-Chaustre Tanner J. Howard Bin-Yan Hsu Fabiola Iannarilli Esperanza C. Iranzo Erik N. K. Iverson Saheed Olaide Jimoh Douglas H. Johnson Martin Johnsson Jesse Jorna Tommaso Jucker Martin Jung Ineta Kačergytė Oliver Kaltz Alison Ke Clint D. Kelly Katharine Keogan Friedrich Wolfgang Keppeler Alexander K. Killion Dongmin Kim David P. Kochan Peter Korsten Shan Kothari Jonas Kuppler Jillian M. Kusch Malgorzata Lagisz Kristen Marianne Lalla Daniel J. Larkin Courtney L. Larson Katherine S. Lauck M. Elise Lauterbur Alan Law Don-Jean Léandri-Breton Jonas J. Lembrechts Kiara L’Herpiniere Eva J. P. Lievens Daniela Oliveira de Lima Shane Lindsay Martin Luquet Ross MacLeod Kirsty H. Macphie Kit Magellan Magdalena M. Mair Lisa E. Malm Stefano Mammola Caitlin P. Mandeville Michael Manhart Laura Milena Manrique-Garzon Elina Mäntylä Philippe Marchand Benjamin Michael Marshall Charles A. Martin Dominic Andreas Martin Jake Mitchell Martin April Robin Martinig Erin S. McCallum Mark McCauley Sabrina M. McNew Scott J. Meiners Thomas Merkling Marcus Michelangeli Maria Moiron Bruno Moreira Jennifer Mortensen Benjamin Mos Taofeek Olatunbosun Muraina Penelope Wrenn Murphy Luca Nelli Petri Niemelä Josh Nightingale Gustav Nilsonne Sergio Nolazco Sabine S. Nooten Jessie Lanterman Novotny Agnes Birgitta Olin Chris L. Organ Kate L. Ostevik Facundo Xavier Palacio Matthieu Paquet Darren James Parker David J. Pascall Valerie J. Pasquarella John Harold Paterson Ana Payo-Payo Karen Marie Pedersen Grégoire Perez Kayla I. Perry Patrice Pottier Michael J. Proulx Raphaël Proulx Jessica L Pruett Veronarindra Ramananjato Finaritra Tolotra Randimbiarison Onja H. Razafindratsima Diana J. Rennison Federico Riva Sepand Riyahi Michael James Roast Felipe Pereira Rocha Dominique G. Roche Cristian Román-Palacios Michael S. Rosenberg Jessica Ross Freya E. Rowland Deusdedith Rugemalila Avery L. Russell Suvi Ruuskanen Patrick Saccone Asaf Sadeh Stephen M. Salazar Kris Sales Pablo Salmón Alfredo Sánchez-Tójar Leticia Pereira Santos Francesca Santostefano Hayden T. Schilling Marcus Schmidt Tim Schmoll Adam C. Schneider Allie E. Schrock Julia Schroeder Nicolas Schtickzelle Nick L. Schultz Drew A. Scott Michael Peter Scroggie Julie Teresa Shapiro Nitika Sharma Caroline L. Shearer Diego Simón Michael I. Sitvarin Fabrício Luiz Skupien Heather Lea Slinn Grania Polly Smith Jeremy A. Smith Rahel Sollmann Kaitlin Stack Whitney Shannon Michael Still Erica F. Stuber Guy F. Sutton Ben Swallow Conor Claverie Taff Elina Takola Andrew J. Tanentzap Rocío Tarjuelo Richard J. Telford Christopher J. Thawley Hugo Thierry Jacqueline Thomson Svenja Tidau Emily M. Tompkins Claire Marie Tortorelli Andrew Trlica Biz R. Turnell Lara Urban Stijn Van de Vondel Jessica Eva Megan van der Wal Jens Van Eeckhoven Francis van Oordt K. Michelle Vanderwel Mark C. Vanderwel Karen J. Vanderwolf Juliana Vélez Diana Carolina Vergara-Florez Brian C. Verrelli Marcus Vinícius Vieira Nora Villamil Valerio Vitali Julien Vollering Jeffrey Walker Xanthe J. Walker Jonathan A. Walter Pawel Waryszak Ryan J. Weaver Ronja E. M. Wedegärtner Daniel L. Weller Shannon Whelan Rachel Louise White David William Wolfson Andrew Wood Scott W. Yanco Jian D. L. Yen Casey Youngflesh Giacomo Zilio Cédric Zimmer Gregory Mark Zimmerman Rachel A. Zitomer |
| author_facet | Elliot Gould Hannah S. Fraser Timothy H. Parker Shinichi Nakagawa Simon C. Griffith Peter A. Vesk Fiona Fidler Daniel G. Hamilton Robin N. Abbey-Lee Jessica K. Abbott Luis A. Aguirre Carles Alcaraz Irith Aloni Drew Altschul Kunal Arekar Jeff W. Atkins Joe Atkinson Christopher M. Baker Meghan Barrett Kristian Bell Suleiman Kehinde Bello Iván Beltrán Bernd J. Berauer Michael Grant Bertram Peter D. Billman Charlie K. Blake Shannon Blake Louis Bliard Andrea Bonisoli-Alquati Timothée Bonnet Camille Nina Marion Bordes Aneesh P. H. Bose Thomas Botterill-James Melissa Anna Boyd Sarah A. Boyle Tom Bradfer-Lawrence Jennifer Bradham Jack A. Brand Martin I. Brengdahl Martin Bulla Luc Bussière Ettore Camerlenghi Sara E. Campbell Leonardo L. F. Campos Anthony Caravaggi Pedro Cardoso Charles J. W. Carroll Therese A. Catanach Xuan Chen Heung Ying Janet Chik Emily Sarah Choy Alec Philip Christie Angela Chuang Amanda J. Chunco Bethany L. Clark Andrea Contina Garth A. Covernton Murray P. Cox Kimberly A. Cressman Marco Crotti Connor Davidson Crouch Pietro B. D’Amelio Alexandra Allison de Sousa Timm Fabian Döbert Ralph Dobler Adam J. Dobson Tim S. Doherty Szymon Marian Drobniak Alexandra Grace Duffy Alison B. Duncan Robert P. Dunn Jamie Dunning Trishna Dutta Luke Eberhart-Hertel Jared Alan Elmore Mahmoud Medhat Elsherif Holly M. English David C. Ensminger Ulrich Rainer Ernst Stephen M. Ferguson Esteban Fernandez-Juricic Thalita Ferreira-Arruda John Fieberg Elizabeth A. Finch Evan A. Fiorenza David N. Fisher Amélie Fontaine Wolfgang Forstmeier Yoan Fourcade Graham S. Frank Cathryn A. Freund Eduardo Fuentes-Lillo Sara L. Gandy Dustin G. Gannon Ana I. García-Cervigón Alexis C. Garretson Xuezhen Ge William L. Geary Charly Géron Marc Gilles Antje Girndt Daniel Gliksman Harrison B. Goldspiel Dylan G. E. Gomes Megan Kate Good Sarah C. Goslee J. Stephen Gosnell Eliza M. Grames Paolo Gratton Nicholas M. Grebe Skye M. Greenler Maaike Griffioen Daniel M. Griffith Frances J. Griffith Jake J. Grossman Ali Güncan Stef Haesen James G. Hagan Heather A. Hager Jonathan Philo Harris Natasha Dean Harrison Sarah Syedia Hasnain Justin Chase Havird Andrew J. Heaton María Laura Herrera-Chaustre Tanner J. Howard Bin-Yan Hsu Fabiola Iannarilli Esperanza C. Iranzo Erik N. K. Iverson Saheed Olaide Jimoh Douglas H. Johnson Martin Johnsson Jesse Jorna Tommaso Jucker Martin Jung Ineta Kačergytė Oliver Kaltz Alison Ke Clint D. Kelly Katharine Keogan Friedrich Wolfgang Keppeler Alexander K. Killion Dongmin Kim David P. Kochan Peter Korsten Shan Kothari Jonas Kuppler Jillian M. Kusch Malgorzata Lagisz Kristen Marianne Lalla Daniel J. Larkin Courtney L. Larson Katherine S. Lauck M. Elise Lauterbur Alan Law Don-Jean Léandri-Breton Jonas J. Lembrechts Kiara L’Herpiniere Eva J. P. Lievens Daniela Oliveira de Lima Shane Lindsay Martin Luquet Ross MacLeod Kirsty H. Macphie Kit Magellan Magdalena M. Mair Lisa E. Malm Stefano Mammola Caitlin P. Mandeville Michael Manhart Laura Milena Manrique-Garzon Elina Mäntylä Philippe Marchand Benjamin Michael Marshall Charles A. Martin Dominic Andreas Martin Jake Mitchell Martin April Robin Martinig Erin S. McCallum Mark McCauley Sabrina M. McNew Scott J. Meiners Thomas Merkling Marcus Michelangeli Maria Moiron Bruno Moreira Jennifer Mortensen Benjamin Mos Taofeek Olatunbosun Muraina Penelope Wrenn Murphy Luca Nelli Petri Niemelä Josh Nightingale Gustav Nilsonne Sergio Nolazco Sabine S. Nooten Jessie Lanterman Novotny Agnes Birgitta Olin Chris L. Organ Kate L. Ostevik Facundo Xavier Palacio Matthieu Paquet Darren James Parker David J. Pascall Valerie J. Pasquarella John Harold Paterson Ana Payo-Payo Karen Marie Pedersen Grégoire Perez Kayla I. Perry Patrice Pottier Michael J. Proulx Raphaël Proulx Jessica L Pruett Veronarindra Ramananjato Finaritra Tolotra Randimbiarison Onja H. Razafindratsima Diana J. Rennison Federico Riva Sepand Riyahi Michael James Roast Felipe Pereira Rocha Dominique G. Roche Cristian Román-Palacios Michael S. Rosenberg Jessica Ross Freya E. Rowland Deusdedith Rugemalila Avery L. Russell Suvi Ruuskanen Patrick Saccone Asaf Sadeh Stephen M. Salazar Kris Sales Pablo Salmón Alfredo Sánchez-Tójar Leticia Pereira Santos Francesca Santostefano Hayden T. Schilling Marcus Schmidt Tim Schmoll Adam C. Schneider Allie E. Schrock Julia Schroeder Nicolas Schtickzelle Nick L. Schultz Drew A. Scott Michael Peter Scroggie Julie Teresa Shapiro Nitika Sharma Caroline L. Shearer Diego Simón Michael I. Sitvarin Fabrício Luiz Skupien Heather Lea Slinn Grania Polly Smith Jeremy A. Smith Rahel Sollmann Kaitlin Stack Whitney Shannon Michael Still Erica F. Stuber Guy F. Sutton Ben Swallow Conor Claverie Taff Elina Takola Andrew J. Tanentzap Rocío Tarjuelo Richard J. Telford Christopher J. Thawley Hugo Thierry Jacqueline Thomson Svenja Tidau Emily M. Tompkins Claire Marie Tortorelli Andrew Trlica Biz R. Turnell Lara Urban Stijn Van de Vondel Jessica Eva Megan van der Wal Jens Van Eeckhoven Francis van Oordt K. Michelle Vanderwel Mark C. Vanderwel Karen J. Vanderwolf Juliana Vélez Diana Carolina Vergara-Florez Brian C. Verrelli Marcus Vinícius Vieira Nora Villamil Valerio Vitali Julien Vollering Jeffrey Walker Xanthe J. Walker Jonathan A. Walter Pawel Waryszak Ryan J. Weaver Ronja E. M. Wedegärtner Daniel L. Weller Shannon Whelan Rachel Louise White David William Wolfson Andrew Wood Scott W. Yanco Jian D. L. Yen Casey Youngflesh Giacomo Zilio Cédric Zimmer Gregory Mark Zimmerman Rachel A. Zitomer |
| author_sort | Elliot Gould |
| collection | DOAJ |
| description | Abstract Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they make regarding statistical analyses. A growing array of studies has explored this analytical variability in different fields and has found substantial variability among results despite analysts having the same data and research question. Many of these studies have been in the social sciences, but one small “many analyst” study found similar variability in ecology. We expanded the scope of this prior work by implementing a large-scale empirical exploration of the variation in effect sizes and model predictions generated by the analytical decisions of different researchers in ecology and evolutionary biology. We used two unpublished datasets, one from evolutionary ecology (blue tit, Cyanistes caeruleus, to compare sibling number and nestling growth) and one from conservation ecology (Eucalyptus, to compare grass cover and tree seedling recruitment). The project leaders recruited 174 analyst teams, comprising 246 analysts, to investigate the answers to prespecified research questions. Analyses conducted by these teams yielded 141 usable effects (compatible with our meta-analyses and with all necessary information provided) for the blue tit dataset, and 85 usable effects for the Eucalyptus dataset. We found substantial heterogeneity among results for both datasets, although the patterns of variation differed between them. For the blue tit analyses, the average effect was convincingly negative, with less growth for nestlings living with more siblings, but there was near continuous variation in effect size from large negative effects to effects near zero, and even effects crossing the traditional threshold of statistical significance in the opposite direction. In contrast, the average relationship between grass cover and Eucalyptus seedling number was only slightly negative and not convincingly different from zero, and most effects ranged from weakly negative to weakly positive, with about a third of effects crossing the traditional threshold of significance in one direction or the other. However, there were also several striking outliers in the Eucalyptus dataset, with effects far from zero. For both datasets, we found substantial variation in the variable selection and random effects structures among analyses, as well as in the ratings of the analytical methods by peer reviewers, but we found no strong relationship between any of these and deviation from the meta-analytic mean. In other words, analyses with results that were far from the mean were no more or less likely to have dissimilar variable sets, use random effects in their models, or receive poor peer reviews than those analyses that found results that were close to the mean. The existence of substantial variability among analysis outcomes raises important questions about how ecologists and evolutionary biologists should interpret published results, and how they should conduct analyses in the future. |
| format | Article |
| id | doaj-art-de5863fbbfd4411f8ee09e352a483284 |
| institution | Kabale University |
| issn | 1741-7007 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Biology |
| spelling | doaj-art-de5863fbbfd4411f8ee09e352a4832842025-02-09T12:54:23ZengBMCBMC Biology1741-70072025-02-0123113610.1186/s12915-024-02101-xSame data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biologyElliot Gould0Hannah S. Fraser1Timothy H. Parker2Shinichi Nakagawa3Simon C. Griffith4Peter A. Vesk5Fiona Fidler6Daniel G. Hamilton7Robin N. Abbey-Lee8Jessica K. Abbott9Luis A. Aguirre10Carles Alcaraz11Irith Aloni12Drew Altschul13Kunal Arekar14Jeff W. Atkins15Joe Atkinson16Christopher M. Baker17Meghan Barrett18Kristian Bell19Suleiman Kehinde Bello20Iván Beltrán21Bernd J. Berauer22Michael Grant Bertram23Peter D. Billman24Charlie K. Blake25Shannon Blake26Louis Bliard27Andrea Bonisoli-Alquati28Timothée Bonnet29Camille Nina Marion Bordes30Aneesh P. H. Bose31Thomas Botterill-James32Melissa Anna Boyd33Sarah A. Boyle34Tom Bradfer-Lawrence35Jennifer Bradham36Jack A. Brand37Martin I. Brengdahl38Martin Bulla39Luc Bussière40Ettore Camerlenghi41Sara E. Campbell42Leonardo L. F. Campos43Anthony Caravaggi44Pedro Cardoso45Charles J. W. Carroll46Therese A. Catanach47Xuan Chen48Heung Ying Janet Chik49Emily Sarah Choy50Alec Philip Christie51Angela Chuang52Amanda J. Chunco53Bethany L. Clark54Andrea Contina55Garth A. Covernton56Murray P. Cox57Kimberly A. Cressman58Marco Crotti59Connor Davidson Crouch60Pietro B. D’Amelio61Alexandra Allison de Sousa62Timm Fabian Döbert63Ralph Dobler64Adam J. Dobson65Tim S. Doherty66Szymon Marian Drobniak67Alexandra Grace Duffy68Alison B. Duncan69Robert P. Dunn70Jamie Dunning71Trishna Dutta72Luke Eberhart-Hertel73Jared Alan Elmore74Mahmoud Medhat Elsherif75Holly M. English76David C. Ensminger77Ulrich Rainer Ernst78Stephen M. Ferguson79Esteban Fernandez-Juricic80Thalita Ferreira-Arruda81John Fieberg82Elizabeth A. Finch83Evan A. Fiorenza84David N. Fisher85Amélie Fontaine86Wolfgang Forstmeier87Yoan Fourcade88Graham S. Frank89Cathryn A. Freund90Eduardo Fuentes-Lillo91Sara L. Gandy92Dustin G. Gannon93Ana I. García-Cervigón94Alexis C. Garretson95Xuezhen Ge96William L. Geary97Charly Géron98Marc Gilles99Antje Girndt100Daniel Gliksman101Harrison B. Goldspiel102Dylan G. E. Gomes103Megan Kate Good104Sarah C. Goslee105J. Stephen Gosnell106Eliza M. Grames107Paolo Gratton108Nicholas M. Grebe109Skye M. Greenler110Maaike Griffioen111Daniel M. Griffith112Frances J. Griffith113Jake J. Grossman114Ali Güncan115Stef Haesen116James G. Hagan117Heather A. Hager118Jonathan Philo Harris119Natasha Dean Harrison120Sarah Syedia Hasnain121Justin Chase Havird122Andrew J. Heaton123María Laura Herrera-Chaustre124Tanner J. Howard125Bin-Yan Hsu126Fabiola Iannarilli127Esperanza C. Iranzo128Erik N. K. Iverson129Saheed Olaide Jimoh130Douglas H. Johnson131Martin Johnsson132Jesse Jorna133Tommaso Jucker134Martin Jung135Ineta Kačergytė136Oliver Kaltz137Alison Ke138Clint D. Kelly139Katharine Keogan140Friedrich Wolfgang Keppeler141Alexander K. Killion142Dongmin Kim143David P. Kochan144Peter Korsten145Shan Kothari146Jonas Kuppler147Jillian M. Kusch148Malgorzata Lagisz149Kristen Marianne Lalla150Daniel J. Larkin151Courtney L. Larson152Katherine S. Lauck153M. Elise Lauterbur154Alan Law155Don-Jean Léandri-Breton156Jonas J. Lembrechts157Kiara L’Herpiniere158Eva J. P. Lievens159Daniela Oliveira de Lima160Shane Lindsay161Martin Luquet162Ross MacLeod163Kirsty H. Macphie164Kit MagellanMagdalena M. Mair165Lisa E. Malm166Stefano Mammola167Caitlin P. Mandeville168Michael Manhart169Laura Milena Manrique-Garzon170Elina Mäntylä171Philippe Marchand172Benjamin Michael Marshall173Charles A. Martin174Dominic Andreas Martin175Jake Mitchell Martin176April Robin Martinig177Erin S. McCallum178Mark McCauley179Sabrina M. McNew180Scott J. Meiners181Thomas Merkling182Marcus Michelangeli183Maria Moiron184Bruno Moreira185Jennifer Mortensen186Benjamin Mos187Taofeek Olatunbosun Muraina188Penelope Wrenn Murphy189Luca Nelli190Petri Niemelä191Josh Nightingale192Gustav Nilsonne193Sergio Nolazco194Sabine S. Nooten195Jessie Lanterman Novotny196Agnes Birgitta Olin197Chris L. Organ198Kate L. Ostevik199Facundo Xavier Palacio200Matthieu Paquet201Darren James Parker202David J. Pascall203Valerie J. Pasquarella204John Harold Paterson205Ana Payo-Payo206Karen Marie Pedersen207Grégoire Perez208Kayla I. Perry209Patrice Pottier210Michael J. Proulx211Raphaël Proulx212Jessica L Pruett213Veronarindra Ramananjato214Finaritra Tolotra Randimbiarison215Onja H. Razafindratsima216Diana J. Rennison217Federico Riva218Sepand Riyahi219Michael James Roast220Felipe Pereira Rocha221Dominique G. Roche222Cristian Román-Palacios223Michael S. Rosenberg224Jessica Ross225Freya E. Rowland226Deusdedith Rugemalila227Avery L. Russell228Suvi Ruuskanen229Patrick Saccone230Asaf Sadeh231Stephen M. Salazar232Kris Sales233Pablo Salmón234Alfredo Sánchez-Tójar235Leticia Pereira Santos236Francesca Santostefano237Hayden T. Schilling238Marcus Schmidt239Tim Schmoll240Adam C. Schneider241Allie E. Schrock242Julia Schroeder243Nicolas Schtickzelle244Nick L. Schultz245Drew A. Scott246Michael Peter Scroggie247Julie Teresa Shapiro248Nitika Sharma249Caroline L. Shearer250Diego Simón251Michael I. SitvarinFabrício Luiz Skupien252Heather Lea Slinn253Grania Polly Smith254Jeremy A. Smith255Rahel Sollmann256Kaitlin Stack Whitney257Shannon Michael Still258Erica F. Stuber259Guy F. Sutton260Ben Swallow261Conor Claverie Taff262Elina Takola263Andrew J. Tanentzap264Rocío Tarjuelo265Richard J. Telford266Christopher J. Thawley267Hugo Thierry268Jacqueline Thomson269Svenja Tidau270Emily M. Tompkins271Claire Marie Tortorelli272Andrew Trlica273Biz R. Turnell274Lara Urban275Stijn Van de Vondel276Jessica Eva Megan van der Wal277Jens Van Eeckhoven278Francis van Oordt279K. Michelle Vanderwel280Mark C. Vanderwel281Karen J. Vanderwolf282Juliana Vélez283Diana Carolina Vergara-Florez284Brian C. Verrelli285Marcus Vinícius Vieira286Nora Villamil287Valerio Vitali288Julien Vollering289Jeffrey Walker290Xanthe J. Walker291Jonathan A. Walter292Pawel Waryszak293Ryan J. Weaver294Ronja E. M. Wedegärtner295Daniel L. Weller296Shannon Whelan297Rachel Louise White298David William Wolfson299Andrew Wood300Scott W. Yanco301Jian D. L. Yen302Casey Youngflesh303Giacomo Zilio304Cédric Zimmer305Gregory Mark Zimmerman306Rachel A. Zitomer307School of Agriculture Food and Ecosystem Sciences, University of MelbourneSchool of Historical and Philosophical Studies, University of MelbourneDepartment of Biology, Whitman CollegeSchool of Biological, Earth & Environmental Sciences, University of New South WalesSchool of Natural Sciences, Macquarie UniversitySchool of Agriculture Food and Ecosystem Sciences, University of MelbourneSchool of Historical and Philosophical Studies, University of MelbourneSchool of Public Health and Preventive Medicine, Monash UniversityLänsstyrelsen ÖstergötlandBiology Department, Lund UniversityDepartment of Biology, University of MassachusettsMarine and Continental Waters, IRTADepartment of Life Sciences, Ben Gurion University of the NegevDepartment of Psychology, The University of EdinburghCentre for Ecological Sciences, Indian Institute of Science, Indian Institute of ScienceSouthern Research Station, USDA Forest ServiceCenter for Ecological Dynamics in a Novel Biosphere (ECONOVO), Department of Biology, Aarhus UniversitySchool of Mathematics and Statistics, University of MelbourneBiology, Indiana University Purdue University IndianapolisSchool of Life and Environmental Sciences, Deakin UniversityDepartment of Arid Land Agriculture, King Abdulaziz UniversityDepartment of Biological Sciences, Macquarie UniversityDepartment of Plant Ecology, University of Hohenheim, Institute of Landscape and Plant EcologyDepartment of Wildlife, Fish, and Environmental Studies, Swedish University of Agricultural SciencesDepartment of Ecology and Evolutionary Biology, University of ConnecticutSTEM Center, Southern Illinois University EdwardsvilleUniversity of GuelphDepartment of Evolutionary Biology and Environmental Studies, University of ZurichDepartment of Biological Sciences, California State Polytechnic UniversityCentre d’Études Biologiques de Chizé, UMR 7372, Université de la Rochelle - Centre National de la Recherche ScientifiqueFaculty of Life Sciences, Bar Ilan UniversityDepartment of Wildlife, Fish, and Environmental Studies, Swedish University of Agricultural SciencesSchool of Natural Sciences, University of Tasmania, TASWhitebark InstituteDepartment of Biology, Rhodes CollegeCentre for Conservation Science, RSPBEnvironmental Studies, Wofford CollegeDepartment of Wildlife, Fish, and Environmental Studies, Swedish University of Agricultural SciencesIFM Biology, Linköping UniversityFaculty of Environmental Sciences, Czech University of Life Sciences Prague, Czech RepublicBiological and Environmental Sciences & Gothenburg Global Biodiversity Centre, University of GothenburgSchool of Biological Sciences, Monash UniversityEcology and Evolutionary Biology, University of Tennessee KnoxvilleDepartamento de Ecologia e Zoologia, Centro de Ciências Biológicas, Universidade Federal de Santa Catarina, UFSC, Campus Universitário - Córrego Grande Florianópolis – SC; CEPSchool of Biological and Forensic Sciences, University of South WalesCentre for Ecology, Evolution and Environmental Changes (cE3c) &, CHANGE - Global Change and Sustainability Institute, Faculdade de Ciências, Universidade de LisboaForest and Rangeland Stewardship, Colorado State UniversityDepartment of Ornithology, Academy of Natural Sciences of Drexel UniversitySalisbury UniversityGroningen Institute for Evolutionary Life Sciences, University of GroningenDepartment of Biology, McMaster UniversityDepartment of Zoology, University of CambridgeEntomology and Nematology, University of FloridaEnvironmental Studies, Elon UniversityBirdLife InternationalSchool of Integrative Biological and Chemical Sciences, The University of Texas Rio Grande Valley, One West University BoulevardDepartment of Ecology and Evolutionary Biology, University of TorontoDepartment of Statistics, University of AucklandLLC, Catbird StatsSchool of Biodiversity, One Health & Veterinary Medicine, University of GlasgowSchool of Forestry, Northern Arizona UniversityDepartment of Behavioural Neurobiology, Max Planck Institute for Biological Intelligence, Eberhard-Gwinner-StrasseSchool of Sciences: Center for Health and Cognition, Bath Spa UniversityDepartment of Biological Sciences, University of AlbertaApplied ZoologySchool of Molecular Biosciences, College of Medical Veterinary & Life Sciences, University of GlasgowSchool of Life and Environmental Sciences, The University of SydneyInstitute of Environmental Sciences, Jagiellonian UniversityBiology Department, Brigham Young UniversityInstitute of Evolutionary Sciences Montpellier, University of Montpellier, CNRS, IRDBaruch Marine Field Laboratory, University of South CarolinaDepartment of Life Sciences, Imperial College LondonEuropean Forest InstituteDepartment of Ornithology, Max Planck Institute for Biological IntelligenceForestry and Environmental Conservation, National Bobwhite and Grassland Initiative, Clemson UniversityDepartment of Psychology and Vision Science, University of BirminghamSchool of Biology and Environmental Science, University College DublinDepartment of Biological Sciences, San José State UniversityApicultural State Institute, University of HohenheimDepartment of Biology, St. Norbert CollegeDepartment of Biological Sciences, Purdue UniversityBiodiversity, Faculty of Forest Sciences and Forest Ecology, University of GöttingenDepartment of Fisheries, Wildlife and Conservation Biology, University of Minnesota-Twin CitiesCABIDepartment of Ecology and Evolutionary Biology, School of Biological Sciences, University of CaliforniaSchool of Biological Sciences, University of AberdeenDepartment of Natural Resource Sciences, McGill UniversityDepartment of Ornithology, Max Planck Institute for Biological IntelligenceInstitute of Ecology and Environmental Sciences (iEES), Univ. Paris-Est CreteilDepartment of Forest Ecosystems and Society, Oregon State UniversityWake Forest UniversityLaboratorio de Invasiones Biológicas (LIB), Instituto de Ecología y BiodiversidadInstitute for Biodiversity, Animal Health and Comparative Medicine, University of GlasgowDepartment of Forest Ecosystems and Society, College of Forestry, Oregon State UniversityBiodiversity and Conservation Area, Rey Juan Carlos UniversityGraduate School of Biomedical Sciences, Tufts UniversityDepartment of Integrative Biology, University of GuelphSchool of Life and Environmental Sciences (Burwood Campus), Deakin UniversityCNRS, University of RennesDepartment of Behavioural Ecology, Bielefeld UniversityFakultät für Biologie, Arbeitsgruppe Evolutionsbiologie, Universität BielefeldChair of Meteorology, Institute for Hydrology and Meteorology, Faculty of Environmental Sciences, Technische Universität DresdenDepartment of Wildlife, Fisheries, and Conservation Biology, University of MaineDepartment of Biological Sciences, Boise State UniversitySchool of Agriculture, Food and Ecosystem Sciences, The University of MelbournePastures Systems and Watershed Management Research Unit, USDA Agricultural Research Service, USDA-ARS PSWMRUDepartment of Natural Sciences, Baruch College, City University of New YorkDepartment of Biological Sciences, Binghamton UniversityDipartimento di Biologia, Università di Roma “Tor Vergata”Department of Anthropology, University of MichiganCollege of Forestry, Oregon State UniversityUniversity of AntwerpEarth & Environmental Sciences, Wesleyan UniversityDepartment of Psychiatry, Yale School of Medicine, Yale UniversityBiology Department and Environmental Studies Department, St. Olaf CollegeDepartment of Plant Protection, Faculty of Agriculture, Department of Plant Protection, Faculty of Agriculture, Ordu University, Ordu UniversityDepartment of Earth and Environmental Sciences, KU LeuvenDepartment of Marine Sciences, University of GothenburgDepartment of Biology, Wilfrid Laurier UniversityNatural Resource Ecology and Management, Iowa State UniversitySchool of Biological Sciences, University of Western AustraliaDepartment of Biological Sciences, Middle East Technical UniversityDept. of Integrative Biology, University of Texas at Austin,2415 Speedway #C0930, Grand Bay National Estuarine Research ReserveUniversidad de los AndesSchool of Agriculture Food and Ecosystem Sciences, University of MelbourneDepartment of Biology, University of TurkuDepartment of Fisheries, Wildlife and Conservation Biology, University of Minnesota-Twin CitiesInstituto de Ciencia Animal. Facultad de Ciencias Veterinarias, Universidad Austral de ChileDepartment of Integrative Biology, The University of Texas at AustinDepartment of Botany, University of WyomingDepartment of Fisheries, Wildlife and Conservation Biology, University of Minnesota-Twin CitiesDepartment of Animal Breeding and Genetics, Swedish University of Agricultural SciencesDepartment of Biology, Brigham Young University, Brigham Young University, Brigham Young UniversitySchool of Biological Sciences, University of BristolInternational Institute for Applied Systems Analysis (IIASA)Department of Ecology, Swedish University of Agricultural SciencesUniversité de Montpellier, ISEM, University of Montpellier, CNRS, EPHEDepartment of Wildlife, Fish, and Conservation Biology, University of CaliforniaDépartement des Sciences biologiques, Université du Québec à MontréalInstitute of Evolutionary Biology, University of EdinburghCenter for Limnology, University of Wisconsin - MadisonCenter for Biodiversity and Global Change, Yale UniversityDepartment of Ecology, Evolution, and Behavior, University of MinnesotaInstitute of Environment and Department of Biological Sciences, Florida International UniversityDepartment of Life Sciences, Aberystwyth UniversityInstitut de recherche en biologie végétale, Université de MontréalInstitute of Evolutionary Ecology and Conservation Genomics, Ulm UniversityDepartment of Biology, Memorial University of NewfoundlandEvolution & Ecology Research Centre, School of Biological, Earth & Environmental Sciences, University of New South Wales, UNSW SydneyDepartment of Natural Resource Sciences, McGill UniversityDepartment of Fisheries, Wildlife and Conservation Biology, University of Minnesota-Twin CitiesThe Nature ConservancyDepartment of Wildlife, Fish, and Conservation Biology, University of CaliforniaEcology and Evolutionary Biology, University of ArizonaBiological and Environmental Sciences, University of StirlingDepartment of Natural Resource Sciences, McGill UniversityDepartment of Biology, University of AntwerpSchool of Natural Sciences, Macquarie UniversityAquatic Ecology and Evolution Group, Limnological Institute, University of KonstanzCampus Cerro Largo, Universidade Federal da Fronteira Sul, Rua Jacob HaupenthalSchool of Psychology and Social Work, University of Hull, Cottingham RdUMR 1224, ECOBIOP, Université de Pau et des Pays de l′AdourSchool of Biological & Environmental Sciences, Liverpool John Moores UniversityInstitute of Ecology and Evolution, University of Edinburgh, The University of EdinburghStatistical Ecotoxicology, Bayreuth Center of Ecology and Environmental Research (BayCEER), University of BayreuthEcology and Environmental Science, Umeå UniversityMolecular Ecology Group (MEG), Water Research Institute (IRSA), National Research Council of Italy (CNR)Department of Natural History, Norwegian University of Science and TechnologyCenter for Advanced Biotechnology and Medicine, Rutgers University Robert Wood Johnson Medical SchoolDepartamento de Ciencias Biológicas, Universidad de los AndesDepartment of Biology, University of TurkuInstitut de recherche sur les forêts, Université du Québec en Abitibi-TémiscamingueBiological and Environmental Sciences, University of StirlingUniversité du Québec à Trois-RivièresInstitute of Plant Sciences, University of BernDepartment of Wildlife, Fish, and Environmental Studies, Swedish University of Agricultural SciencesSchool of Biological, Earth and Environmental Sciences, University of New South WalesDepartment of Wildlife, Fish, and Environmental Studies, Swedish University of Agricultural SciencesWhitney Laboratory for Marine Bioscience, University of FloridaEcology and Evolutionary Biology, University of ArizonaBiological Sciences, Eastern Illinois UniversityCentre d’Investigations Clinique Plurithématique - Institut Lorrain du Coeur et des Vaisseaux, Université de Lorraine, Inserm1433 CIC-P CHRU de NancyDepartment of Wildlife, Fish, and Environmental Studies, Swedish University of Agricultural SciencesEvolutionary biology department, Bielefeld University, Bielefeld UniversityDepartment of Ecology and global change, Centro de Investigaciones sobre Desertificación, Consejo Superior de Investigaciones Cientificas (CIDE-CSIC/UV/GV)Department of Biological Sciences, University of ArkansasSchool of the Environment, Faculty of Science, The University of Queensland, The University of QueenslandDepartment of Animal Health and Production, Oyo State College of Agriculture and TechnologyDepartment of Forest & Wildlife Ecology, University of Wisconsin-MadisonSchool of Biodiversity, One Health & Veterinary Medicine, University of GlasgowOrganismal and Evolutionary Biology Research Programme, Faculty of Biological and Environmental Sciences, University of HelsinkiSouth Iceland Research Centre, University of IcelandDepartment of Clinical Neuroscience, Karolinska InstitutetSchool of Biological Sciences, Monash UniversityAnimal Ecology and Tropical Biology, University of WürzburgHiram CollegeDepartment of Aquatic Resources, Swedish University of Agricultural SciencesDepartment of Earth Sciences, Montana State UniversityDepartment of Evolution, Ecology, and Organismal Biology, University of CaliforniaSección Ornitología, Universidad Nacional de La PlataDepartment of Ecology, Swedish University of Agricultural SciencesBangor University, Bangor UniversityMRC Biostatistics Unit, University of CambridgeHarvard Forest, Harvard UniversityDepartment of Marine Sciences, University of GothenburgDepartamento de Biodiversidad, Ecología y Evolución, Universidad Complutense de Madrid, C. de José Antonio NovaisBiology Department, Technische Universität DarmstadtUMR 1309, ASTRE, CIRADDepartment of Entomology, The Ohio State UniversityEvolution & Ecology Research Centre, School of Biological, Earth & Environmental Sciences, University of New South Wales, UNSW SydneyDepartment of Psychology, University of BathChaire de recherche en intégrité écologique, Université du Québec à Trois-RivièresMississippi Based RESTORE Act Center of Excellence, University of Southern MississippiDepartment of Integrative Biology, University of CaliforniaMention Zoologie et Biodiversité Animale, Faculté des Sciences, Université d’Antananarivo, Mention Zoologie et Biodiversié Animale, Université d’AntananarivoDepartment of Integrative Biology, Valley Life Sciences Building 3140, University of California, University of California BerkeleyDepartment of Ecology, Behavior and Evolution, University of CaliforniaInstitute for Environmental Sciences, VU AmsterdamDepartment of Evolutionary Anthropology, University of ViennaKonrad Lorenz Institute for Ethology, University of Veterinary MedicineSchool of Biological Sciences, The University of Hong KongInstitut de biologie, Université de NeuchâtelSchool of Information, University of ArizonaCenter for Biological Data Science, Virginia Commonwealth UniversityUniversity of WisconsinSchool of the Environment, Yale UniversityInstitute of the Environment, Florida International UniversityDepartment of Biology, Missouri State UniversityDepartment of Biological and Environmental Science, University of JyväskyläInstitute for Interdisciplinary Mountain Research, OeAW (Austrian Academy of Sciences), GLORIADepartment of Natural Resources, Newe Ya’ar Research Center, Agricultural Research Organization (Volcani Institute)Department of Animal Behaviour, Bielefeld UniversityOffice for National Statistics, Segensworth RdInstitute of Avian Research “Vogelwarte Helgoland”Department of Evolutionary Biology, Bielefeld University, North Rhine-WestphaliaEcology Department, Universidade Federal de GoiásCentre for Ecology and Conservation, University of Exeter, Penryn CampusNew South Wales, Department of Primary Industries FisheriesResearch Data Management, Leibniz Centre for Agricultural Landscape Research (ZALF)Department of Evolutionary Biology, Bielefeld University, North Rhine-WestphaliaBiology Department, University of Wisconsin-La CrosseDepartment of Evolutionary Anthropology, Duke UniversityDepartment of Life Sciences, Imperial College LondonEarth and Life Institute, Ecology and Biodiversity, UCLouvainFuture Regions Research Centre, Federation University AustraliaUnited States, Department of Agriculture- Agricultural Research ServiceArthur Rylah Institute for Environmental ResearchEpidemiology and Surveillance Support Unit, University of Lyon - French Agency for Food, Environmental and Occupational Health and Safety (ANSES)Center for Impact, UCLA Anderson, University of CaliforniaDepartment of Evolutionary Anthropology, Duke UniversityFacultad de Ciencias, Universidad de la RepúblicaPrograma de Pós-Graduação em Ecologia, Instituto de Biologia, Centro de Ciências da Saúde, Universidade Federal do Rio de Janeiro, Cidade UniversitáriaVive Crop ProtectionUniversity of Cambridge, Trinity LnBritish Trust for Ornithology, BTODepartment of Forest Ecosystems and Society, College of Forestry, Oregon State UniversityTechnology & Society Department, Rochester Institute of TechnologyNomad EcologyWildland Resources Department, Utah State UniversityCenter for Biological Control, Department of Zoology and Entomology, Rhodes UniversitySchool of Mathematics and Statistics and Centre for Research in Ecological and Environmental Modelling, University of St Andrews, Buchanan GardensDepartment of Ecology and Evolutionary Biology, Cornell UniversityDepartment of Computational Landscape Ecology, Helmholtz Centre for Environmental Research – UFZEcosystems and Global Change Group, School of the Environment, Trent UniversityInstituto Universitario de Investigación en Gestión Forestal Sostenible (iuFOR), Universidad de ValladolidDepartment of Biological Sciences, University of BergenDepartment of Biological Science, University of Rhode IslandDepartment of Geography, McGill UniversityDepartment of Integrative Biology, University of GuelphSchool of Biological and Marine Sciences, University of PlymouthBiology Department, Wake Forest UniversityPlant Sciences, University of CaliforniaCollege of Natural Resources, North Carolina State UniversityInstitute of Zoology, Technische Universität DresdenHelmholtz AI, Helmholtz Zentrum MuenchenDepartment of Biology, University of AntwerpFitzPatrick Institute of African Ornithology, University of Cape Town, University of Cape TownDepartment of Cell & Developmental Biology, Division of Biosciences, University College LondonDepartment of Natural Resource Sciences, McGill UniversityBiology, University of Saskatchewan, University of SaskatchewanDepartment of Biology, University of ReginaBiology, University of WaterlooDepartment of Fisheries, Wildlife and Conservation Biology, University of Minnesota-Twin CitiesDepartment of Ecology & Evolutionary Biology, Biological Science Building, University of MichiganDepartment of Biology, University of AntwerpDept. Ecologia, Instituto de Biologia, Universidade Federal do Rio de JaneiroLothian Analytical Services, Public Health ScotlandInstitute for Evolution and Biodiversity, University of MuensterDepartment of Environmental Sciences, Western Norway University of Applied SciencesDepartment of Biological Sciences, University of Southern MaineCenter for Ecosystem Science and Society, Northern Arizona UniversityCenter for Watershed Sciences, University of CaliforniaSchool of Agriculture and Environmental Science, University of Southern QueenslandDepartment of Ecology, Evolution, and Organismal Biology, Iowa State UniversityFram Project ASDepartment of Food Science & Technology, Virginia Polytechnic Institute and State UniversityDepartment of Natural Resource Sciences, McGill UniversitySchool of Applied Sciences, School of Applied Sciences, University of Brighton, University of BrightonDepartment of Fisheries, Wildlife and Conservation Biology, University of Minnesota-Twin CitiesDepartment of Biology, Biology Research and Administration Building, University of OxfordDepartment of Integrative Biology, University of ColoradoArthur Rylah Institute for Environmental ResearchEcology, Evolution, and Behavior Program, Michigan State UniversityISEM, University of Montpellier, CNRSLaboratoire d’Ethologie Expérimentale et Comparée, LEEC, Université Sorbonne Paris NordDepartment of Science and Environment, Lake Superior State UniversityDepartment of Forest Ecosystems and Society, Oregon State UniversityAbstract Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they make regarding statistical analyses. A growing array of studies has explored this analytical variability in different fields and has found substantial variability among results despite analysts having the same data and research question. Many of these studies have been in the social sciences, but one small “many analyst” study found similar variability in ecology. We expanded the scope of this prior work by implementing a large-scale empirical exploration of the variation in effect sizes and model predictions generated by the analytical decisions of different researchers in ecology and evolutionary biology. We used two unpublished datasets, one from evolutionary ecology (blue tit, Cyanistes caeruleus, to compare sibling number and nestling growth) and one from conservation ecology (Eucalyptus, to compare grass cover and tree seedling recruitment). The project leaders recruited 174 analyst teams, comprising 246 analysts, to investigate the answers to prespecified research questions. Analyses conducted by these teams yielded 141 usable effects (compatible with our meta-analyses and with all necessary information provided) for the blue tit dataset, and 85 usable effects for the Eucalyptus dataset. We found substantial heterogeneity among results for both datasets, although the patterns of variation differed between them. For the blue tit analyses, the average effect was convincingly negative, with less growth for nestlings living with more siblings, but there was near continuous variation in effect size from large negative effects to effects near zero, and even effects crossing the traditional threshold of statistical significance in the opposite direction. In contrast, the average relationship between grass cover and Eucalyptus seedling number was only slightly negative and not convincingly different from zero, and most effects ranged from weakly negative to weakly positive, with about a third of effects crossing the traditional threshold of significance in one direction or the other. However, there were also several striking outliers in the Eucalyptus dataset, with effects far from zero. For both datasets, we found substantial variation in the variable selection and random effects structures among analyses, as well as in the ratings of the analytical methods by peer reviewers, but we found no strong relationship between any of these and deviation from the meta-analytic mean. In other words, analyses with results that were far from the mean were no more or less likely to have dissimilar variable sets, use random effects in their models, or receive poor peer reviews than those analyses that found results that were close to the mean. The existence of substantial variability among analysis outcomes raises important questions about how ecologists and evolutionary biologists should interpret published results, and how they should conduct analyses in the future.https://doi.org/10.1186/s12915-024-02101-xAnalytical heterogeneityMetascienceMany-analystReplication crisisReproducibility |
| spellingShingle | Elliot Gould Hannah S. Fraser Timothy H. Parker Shinichi Nakagawa Simon C. Griffith Peter A. Vesk Fiona Fidler Daniel G. Hamilton Robin N. Abbey-Lee Jessica K. Abbott Luis A. Aguirre Carles Alcaraz Irith Aloni Drew Altschul Kunal Arekar Jeff W. Atkins Joe Atkinson Christopher M. Baker Meghan Barrett Kristian Bell Suleiman Kehinde Bello Iván Beltrán Bernd J. Berauer Michael Grant Bertram Peter D. Billman Charlie K. Blake Shannon Blake Louis Bliard Andrea Bonisoli-Alquati Timothée Bonnet Camille Nina Marion Bordes Aneesh P. H. Bose Thomas Botterill-James Melissa Anna Boyd Sarah A. Boyle Tom Bradfer-Lawrence Jennifer Bradham Jack A. Brand Martin I. Brengdahl Martin Bulla Luc Bussière Ettore Camerlenghi Sara E. Campbell Leonardo L. F. Campos Anthony Caravaggi Pedro Cardoso Charles J. W. Carroll Therese A. Catanach Xuan Chen Heung Ying Janet Chik Emily Sarah Choy Alec Philip Christie Angela Chuang Amanda J. Chunco Bethany L. Clark Andrea Contina Garth A. Covernton Murray P. Cox Kimberly A. Cressman Marco Crotti Connor Davidson Crouch Pietro B. D’Amelio Alexandra Allison de Sousa Timm Fabian Döbert Ralph Dobler Adam J. Dobson Tim S. Doherty Szymon Marian Drobniak Alexandra Grace Duffy Alison B. Duncan Robert P. Dunn Jamie Dunning Trishna Dutta Luke Eberhart-Hertel Jared Alan Elmore Mahmoud Medhat Elsherif Holly M. English David C. Ensminger Ulrich Rainer Ernst Stephen M. Ferguson Esteban Fernandez-Juricic Thalita Ferreira-Arruda John Fieberg Elizabeth A. Finch Evan A. Fiorenza David N. Fisher Amélie Fontaine Wolfgang Forstmeier Yoan Fourcade Graham S. Frank Cathryn A. Freund Eduardo Fuentes-Lillo Sara L. Gandy Dustin G. Gannon Ana I. García-Cervigón Alexis C. Garretson Xuezhen Ge William L. Geary Charly Géron Marc Gilles Antje Girndt Daniel Gliksman Harrison B. Goldspiel Dylan G. E. Gomes Megan Kate Good Sarah C. Goslee J. Stephen Gosnell Eliza M. Grames Paolo Gratton Nicholas M. Grebe Skye M. Greenler Maaike Griffioen Daniel M. Griffith Frances J. Griffith Jake J. Grossman Ali Güncan Stef Haesen James G. Hagan Heather A. Hager Jonathan Philo Harris Natasha Dean Harrison Sarah Syedia Hasnain Justin Chase Havird Andrew J. Heaton María Laura Herrera-Chaustre Tanner J. Howard Bin-Yan Hsu Fabiola Iannarilli Esperanza C. Iranzo Erik N. K. Iverson Saheed Olaide Jimoh Douglas H. Johnson Martin Johnsson Jesse Jorna Tommaso Jucker Martin Jung Ineta Kačergytė Oliver Kaltz Alison Ke Clint D. Kelly Katharine Keogan Friedrich Wolfgang Keppeler Alexander K. Killion Dongmin Kim David P. Kochan Peter Korsten Shan Kothari Jonas Kuppler Jillian M. Kusch Malgorzata Lagisz Kristen Marianne Lalla Daniel J. Larkin Courtney L. Larson Katherine S. Lauck M. Elise Lauterbur Alan Law Don-Jean Léandri-Breton Jonas J. Lembrechts Kiara L’Herpiniere Eva J. P. Lievens Daniela Oliveira de Lima Shane Lindsay Martin Luquet Ross MacLeod Kirsty H. Macphie Kit Magellan Magdalena M. Mair Lisa E. Malm Stefano Mammola Caitlin P. Mandeville Michael Manhart Laura Milena Manrique-Garzon Elina Mäntylä Philippe Marchand Benjamin Michael Marshall Charles A. Martin Dominic Andreas Martin Jake Mitchell Martin April Robin Martinig Erin S. McCallum Mark McCauley Sabrina M. McNew Scott J. Meiners Thomas Merkling Marcus Michelangeli Maria Moiron Bruno Moreira Jennifer Mortensen Benjamin Mos Taofeek Olatunbosun Muraina Penelope Wrenn Murphy Luca Nelli Petri Niemelä Josh Nightingale Gustav Nilsonne Sergio Nolazco Sabine S. Nooten Jessie Lanterman Novotny Agnes Birgitta Olin Chris L. Organ Kate L. Ostevik Facundo Xavier Palacio Matthieu Paquet Darren James Parker David J. Pascall Valerie J. Pasquarella John Harold Paterson Ana Payo-Payo Karen Marie Pedersen Grégoire Perez Kayla I. Perry Patrice Pottier Michael J. Proulx Raphaël Proulx Jessica L Pruett Veronarindra Ramananjato Finaritra Tolotra Randimbiarison Onja H. Razafindratsima Diana J. Rennison Federico Riva Sepand Riyahi Michael James Roast Felipe Pereira Rocha Dominique G. Roche Cristian Román-Palacios Michael S. Rosenberg Jessica Ross Freya E. Rowland Deusdedith Rugemalila Avery L. Russell Suvi Ruuskanen Patrick Saccone Asaf Sadeh Stephen M. Salazar Kris Sales Pablo Salmón Alfredo Sánchez-Tójar Leticia Pereira Santos Francesca Santostefano Hayden T. Schilling Marcus Schmidt Tim Schmoll Adam C. Schneider Allie E. Schrock Julia Schroeder Nicolas Schtickzelle Nick L. Schultz Drew A. Scott Michael Peter Scroggie Julie Teresa Shapiro Nitika Sharma Caroline L. Shearer Diego Simón Michael I. Sitvarin Fabrício Luiz Skupien Heather Lea Slinn Grania Polly Smith Jeremy A. Smith Rahel Sollmann Kaitlin Stack Whitney Shannon Michael Still Erica F. Stuber Guy F. Sutton Ben Swallow Conor Claverie Taff Elina Takola Andrew J. Tanentzap Rocío Tarjuelo Richard J. Telford Christopher J. Thawley Hugo Thierry Jacqueline Thomson Svenja Tidau Emily M. Tompkins Claire Marie Tortorelli Andrew Trlica Biz R. Turnell Lara Urban Stijn Van de Vondel Jessica Eva Megan van der Wal Jens Van Eeckhoven Francis van Oordt K. Michelle Vanderwel Mark C. Vanderwel Karen J. Vanderwolf Juliana Vélez Diana Carolina Vergara-Florez Brian C. Verrelli Marcus Vinícius Vieira Nora Villamil Valerio Vitali Julien Vollering Jeffrey Walker Xanthe J. Walker Jonathan A. Walter Pawel Waryszak Ryan J. Weaver Ronja E. M. Wedegärtner Daniel L. Weller Shannon Whelan Rachel Louise White David William Wolfson Andrew Wood Scott W. Yanco Jian D. L. Yen Casey Youngflesh Giacomo Zilio Cédric Zimmer Gregory Mark Zimmerman Rachel A. Zitomer Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology BMC Biology Analytical heterogeneity Metascience Many-analyst Replication crisis Reproducibility |
| title | Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology |
| title_full | Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology |
| title_fullStr | Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology |
| title_full_unstemmed | Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology |
| title_short | Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology |
| title_sort | same data different analysts variation in effect sizes due to analytical decisions in ecology and evolutionary biology |
| topic | Analytical heterogeneity Metascience Many-analyst Replication crisis Reproducibility |
| url | https://doi.org/10.1186/s12915-024-02101-x |
| work_keys_str_mv | AT elliotgould samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT hannahsfraser samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT timothyhparker samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT shinichinakagawa samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT simoncgriffith samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT peteravesk samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT fionafidler samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT danielghamilton samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT robinnabbeylee samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jessicakabbott samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT luisaaguirre samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT carlesalcaraz samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT irithaloni samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT drewaltschul samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kunalarekar samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jeffwatkins samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT joeatkinson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT christophermbaker samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT meghanbarrett samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kristianbell samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT suleimankehindebello samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ivanbeltran samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT berndjberauer samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT michaelgrantbertram samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT peterdbillman samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT charliekblake samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT shannonblake samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT louisbliard samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT andreabonisolialquati samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT timotheebonnet samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT camilleninamarionbordes samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT aneeshphbose samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT thomasbotterilljames samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT melissaannaboyd samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT sarahaboyle samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT tombradferlawrence samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jenniferbradham samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jackabrand samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT martinibrengdahl samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT martinbulla samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT lucbussiere samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ettorecamerlenghi samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT saraecampbell samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT leonardolfcampos samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT anthonycaravaggi samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT pedrocardoso samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT charlesjwcarroll samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT thereseacatanach samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT xuanchen samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT heungyingjanetchik samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT emilysarahchoy samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alecphilipchristie samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT angelachuang samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT amandajchunco samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT bethanylclark samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT andreacontina samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT garthacovernton samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT murraypcox samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kimberlyacressman samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT marcocrotti samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT connordavidsoncrouch samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT pietrobdamelio samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alexandraallisondesousa samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT timmfabiandobert samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ralphdobler samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT adamjdobson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT timsdoherty samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT szymonmariandrobniak samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alexandragraceduffy samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alisonbduncan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT robertpdunn samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jamiedunning samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT trishnadutta samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT lukeeberharthertel samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jaredalanelmore samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT mahmoudmedhatelsherif samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT hollymenglish samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT davidcensminger samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ulrichrainerernst samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT stephenmferguson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT estebanfernandezjuricic samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT thalitaferreiraarruda samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT johnfieberg samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT elizabethafinch samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT evanafiorenza samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT davidnfisher samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ameliefontaine samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT wolfgangforstmeier samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT yoanfourcade samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT grahamsfrank samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT cathrynafreund samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT eduardofuenteslillo samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT saralgandy samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT dustinggannon samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT anaigarciacervigon samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alexiscgarretson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT xuezhenge samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT williamlgeary samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT charlygeron samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT marcgilles samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT antjegirndt samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT danielgliksman samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT harrisonbgoldspiel samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT dylangegomes samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT megankategood samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT sarahcgoslee samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jstephengosnell samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT elizamgrames samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT paologratton samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT nicholasmgrebe samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT skyemgreenler samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT maaikegriffioen samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT danielmgriffith samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT francesjgriffith samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jakejgrossman samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT aliguncan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT stefhaesen samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jamesghagan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT heatherahager samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jonathanphiloharris samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT natashadeanharrison samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT sarahsyediahasnain samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT justinchasehavird samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT andrewjheaton samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT marialauraherrerachaustre samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT tannerjhoward samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT binyanhsu samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT fabiolaiannarilli samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT esperanzaciranzo samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT eriknkiverson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT saheedolaidejimoh samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT douglashjohnson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT martinjohnsson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jessejorna samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT tommasojucker samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT martinjung samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT inetakacergyte samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT oliverkaltz samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alisonke samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT clintdkelly samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT katharinekeogan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT friedrichwolfgangkeppeler samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alexanderkkillion samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT dongminkim samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT davidpkochan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT peterkorsten samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT shankothari samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jonaskuppler samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jillianmkusch samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT malgorzatalagisz samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kristenmariannelalla samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT danieljlarkin samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT courtneyllarson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT katherineslauck samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT meliselauterbur samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alanlaw samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT donjeanleandribreton samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jonasjlembrechts samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kiaralherpiniere samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT evajplievens samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT danielaoliveiradelima samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT shanelindsay samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT martinluquet samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT rossmacleod samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kirstyhmacphie samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kitmagellan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT magdalenammair samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT lisaemalm samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT stefanomammola samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT caitlinpmandeville samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT michaelmanhart samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT lauramilenamanriquegarzon samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT elinamantyla samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT philippemarchand samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT benjaminmichaelmarshall samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT charlesamartin samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT dominicandreasmartin samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jakemitchellmartin samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT aprilrobinmartinig samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT erinsmccallum samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT markmccauley samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT sabrinammcnew samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT scottjmeiners samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT thomasmerkling samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT marcusmichelangeli samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT mariamoiron samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT brunomoreira samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jennifermortensen samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT benjaminmos samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT taofeekolatunbosunmuraina samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT penelopewrennmurphy samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT lucanelli samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT petriniemela samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT joshnightingale samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT gustavnilsonne samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT sergionolazco samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT sabinesnooten samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jessielantermannovotny samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT agnesbirgittaolin samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT chrislorgan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT katelostevik samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT facundoxavierpalacio samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT matthieupaquet samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT darrenjamesparker samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT davidjpascall samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT valeriejpasquarella samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT johnharoldpaterson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT anapayopayo samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT karenmariepedersen samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT gregoireperez samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kaylaiperry samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT patricepottier samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT michaeljproulx samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT raphaelproulx samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jessicalpruett samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT veronarindraramananjato samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT finaritratolotrarandimbiarison samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT onjahrazafindratsima samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT dianajrennison samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT federicoriva samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT sepandriyahi samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT michaeljamesroast samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT felipepereirarocha samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT dominiquegroche samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT cristianromanpalacios samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT michaelsrosenberg samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jessicaross samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT freyaerowland samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT deusdedithrugemalila samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT averylrussell samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT suviruuskanen samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT patricksaccone samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT asafsadeh samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT stephenmsalazar samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT krissales samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT pablosalmon samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT alfredosancheztojar samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT leticiapereirasantos samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT francescasantostefano samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT haydentschilling samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT marcusschmidt samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT timschmoll samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT adamcschneider samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT allieeschrock samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT juliaschroeder samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT nicolasschtickzelle samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT nicklschultz samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT drewascott samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT michaelpeterscroggie samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT julieteresashapiro samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT nitikasharma samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT carolinelshearer samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT diegosimon samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT michaelisitvarin samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT fabricioluizskupien samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT heatherleaslinn samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT graniapollysmith samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jeremyasmith samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT rahelsollmann samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kaitlinstackwhitney samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT shannonmichaelstill samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ericafstuber samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT guyfsutton samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT benswallow samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT conorclaverietaff samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT elinatakola samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT andrewjtanentzap samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT rociotarjuelo samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT richardjtelford samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT christopherjthawley samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT hugothierry samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jacquelinethomson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT svenjatidau samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT emilymtompkins samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT clairemarietortorelli samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT andrewtrlica samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT bizrturnell samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT laraurban samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT stijnvandevondel samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jessicaevameganvanderwal samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jensvaneeckhoven samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT francisvanoordt samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT kmichellevanderwel samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT markcvanderwel samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT karenjvanderwolf samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT julianavelez samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT dianacarolinavergaraflorez samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT briancverrelli samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT marcusviniciusvieira samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT noravillamil samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT valeriovitali samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT julienvollering samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jeffreywalker samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT xanthejwalker samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jonathanawalter samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT pawelwaryszak samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ryanjweaver samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT ronjaemwedegartner samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT daniellweller samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT shannonwhelan samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT rachellouisewhite samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT davidwilliamwolfson samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT andrewwood samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT scottwyanco samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT jiandlyen samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT caseyyoungflesh samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT giacomozilio samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT cedriczimmer samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT gregorymarkzimmerman samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology AT rachelazitomer samedatadifferentanalystsvariationineffectsizesduetoanalyticaldecisionsinecologyandevolutionarybiology |