Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor.
The reproducibility of studies involving insect species is an underexplored area in the broader discussion about poor reproducibility in science. Our study addresses this gap by conducting a systematic multi-laboratory investigation into the reproducibility of ecological studies on insect behavior....
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-04-01
|
| Series: | PLoS Biology |
| Online Access: | https://doi.org/10.1371/journal.pbio.3003019 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849711222249750528 |
|---|---|
| author | Carolin Mundinger Nora K E Schulz Pragya Singh Steven Janz Maximilian Schurig Jacob Seidemann Joachim Kurtz Caroline Müller Holger Schielzeth Vanessa T von Kortzfleisch S Helene Richter |
| author_facet | Carolin Mundinger Nora K E Schulz Pragya Singh Steven Janz Maximilian Schurig Jacob Seidemann Joachim Kurtz Caroline Müller Holger Schielzeth Vanessa T von Kortzfleisch S Helene Richter |
| author_sort | Carolin Mundinger |
| collection | DOAJ |
| description | The reproducibility of studies involving insect species is an underexplored area in the broader discussion about poor reproducibility in science. Our study addresses this gap by conducting a systematic multi-laboratory investigation into the reproducibility of ecological studies on insect behavior. We implemented a 3 × 3 experimental design, incorporating three study sites, and three independent experiments on three insect species from different orders: the turnip sawfly (Athalia rosae, Hymenoptera), the meadow grasshopper (Pseudochorthippus parallelus, Orthoptera), and the red flour beetle (Tribolium castaneum, Coleoptera). Using random-effect meta-analysis, we compared the consistency and accuracy of treatment effects on insect behavioral traits across replicate experiments. We successfully reproduced the overall statistical treatment effect in 83% of the replicate experiments, but overall effect size replication was achieved in only 66% of the replicates. Thus, though demonstrating sufficient reproducibility in some measures, this study also provides the first experimental evidence for cases of poor reproducibility in insect experiments. Our findings further show that reasons causing poor reproducibility established in rodent research also hold for other study organisms and research questions. We believe that a rethinking of current best practices is required to face reproducibility issues in insect studies but also across disciplines. Specifically, we advocate for adopting open research practices and the implementation of methodological strategies that reduce bias and problems arising from over-standardization. With respect to the latter, the introduction of systematic variation through multi-laboratory or heterogenized designs may contribute to improved reproducibility in studies involving any living organisms. |
| format | Article |
| id | doaj-art-72a51d3cc22947c5a6b0833f396ccae9 |
| institution | DOAJ |
| issn | 1544-9173 1545-7885 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS Biology |
| spelling | doaj-art-72a51d3cc22947c5a6b0833f396ccae92025-08-20T03:14:40ZengPublic Library of Science (PLoS)PLoS Biology1544-91731545-78852025-04-01234e300301910.1371/journal.pbio.3003019Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor.Carolin MundingerNora K E SchulzPragya SinghSteven JanzMaximilian SchurigJacob SeidemannJoachim KurtzCaroline MüllerHolger SchielzethVanessa T von KortzfleischS Helene RichterThe reproducibility of studies involving insect species is an underexplored area in the broader discussion about poor reproducibility in science. Our study addresses this gap by conducting a systematic multi-laboratory investigation into the reproducibility of ecological studies on insect behavior. We implemented a 3 × 3 experimental design, incorporating three study sites, and three independent experiments on three insect species from different orders: the turnip sawfly (Athalia rosae, Hymenoptera), the meadow grasshopper (Pseudochorthippus parallelus, Orthoptera), and the red flour beetle (Tribolium castaneum, Coleoptera). Using random-effect meta-analysis, we compared the consistency and accuracy of treatment effects on insect behavioral traits across replicate experiments. We successfully reproduced the overall statistical treatment effect in 83% of the replicate experiments, but overall effect size replication was achieved in only 66% of the replicates. Thus, though demonstrating sufficient reproducibility in some measures, this study also provides the first experimental evidence for cases of poor reproducibility in insect experiments. Our findings further show that reasons causing poor reproducibility established in rodent research also hold for other study organisms and research questions. We believe that a rethinking of current best practices is required to face reproducibility issues in insect studies but also across disciplines. Specifically, we advocate for adopting open research practices and the implementation of methodological strategies that reduce bias and problems arising from over-standardization. With respect to the latter, the introduction of systematic variation through multi-laboratory or heterogenized designs may contribute to improved reproducibility in studies involving any living organisms.https://doi.org/10.1371/journal.pbio.3003019 |
| spellingShingle | Carolin Mundinger Nora K E Schulz Pragya Singh Steven Janz Maximilian Schurig Jacob Seidemann Joachim Kurtz Caroline Müller Holger Schielzeth Vanessa T von Kortzfleisch S Helene Richter Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor. PLoS Biology |
| title | Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor. |
| title_full | Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor. |
| title_fullStr | Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor. |
| title_full_unstemmed | Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor. |
| title_short | Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor. |
| title_sort | testing the reproducibility of ecological studies on insect behavior in a multi laboratory setting identifies opportunities for improving experimental rigor |
| url | https://doi.org/10.1371/journal.pbio.3003019 |
| work_keys_str_mv | AT carolinmundinger testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT norakeschulz testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT pragyasingh testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT stevenjanz testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT maximilianschurig testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT jacobseidemann testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT joachimkurtz testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT carolinemuller testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT holgerschielzeth testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT vanessatvonkortzfleisch testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor AT shelenerichter testingthereproducibilityofecologicalstudiesoninsectbehaviorinamultilaboratorysettingidentifiesopportunitiesforimprovingexperimentalrigor |