Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders
Abstract Background Accurately determining the sample size (“N”) of a dataset is a key consideration for experimental design. Misidentification of sample size can lead to pseudoreplication, a process of artificially inflating the number of experimental replicates which systematically underestimates...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-05-01
|
| Series: | Molecular Autism |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13229-025-00663-3 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849705273682296832 |
|---|---|
| author | Constantinos Eleftheriou Sarah Giachetti Raven Hickson Laura Kamnioti-Dumont Robert Templaar Alina Aaltonen Eleni Tsoukala Nawon Kim Lysandra Fryer-Petridis Chloe Henley Ceren Erdem Emma Wilson Beatriz Maio Jingjing Ye Jessica C. Pierce Kath Mazur Lucia Landa-Navarro Nina G. Petrović Sarah Bendova Hanan Woods Manuela Rizzi Vanesa Salazar-Sanchez Natasha Anstey Antonios Asiminas Shinjini Basu Sam A. Booker Anjanette Harris Sam Heyes Adam Jackson Alex Crocker-Buque Aoife C. McMahon Sally M. Till Lasani S. Wijetunge David JA Wyllie Catherine M. Abbott Timothy O’Leary Peter C. Kind |
| author_facet | Constantinos Eleftheriou Sarah Giachetti Raven Hickson Laura Kamnioti-Dumont Robert Templaar Alina Aaltonen Eleni Tsoukala Nawon Kim Lysandra Fryer-Petridis Chloe Henley Ceren Erdem Emma Wilson Beatriz Maio Jingjing Ye Jessica C. Pierce Kath Mazur Lucia Landa-Navarro Nina G. Petrović Sarah Bendova Hanan Woods Manuela Rizzi Vanesa Salazar-Sanchez Natasha Anstey Antonios Asiminas Shinjini Basu Sam A. Booker Anjanette Harris Sam Heyes Adam Jackson Alex Crocker-Buque Aoife C. McMahon Sally M. Till Lasani S. Wijetunge David JA Wyllie Catherine M. Abbott Timothy O’Leary Peter C. Kind |
| author_sort | Constantinos Eleftheriou |
| collection | DOAJ |
| description | Abstract Background Accurately determining the sample size (“N”) of a dataset is a key consideration for experimental design. Misidentification of sample size can lead to pseudoreplication, a process of artificially inflating the number of experimental replicates which systematically underestimates variability, overestimates effect sizes and invalidates statistical tests performed on the data. While many journals have adopted stringent requirements with regard to statistical reporting over the last decade, it remains unknown whether such efforts have had a meaningful impact on statistical rigour. Methods Here, we evaluated the prevalence of this type of statistical error among neuroscience studies involving animal models of Fragile-X Syndrome (FXS) and those using animal models of neurological disorders at large published between 2001 and 2024. Results We found that pseudoreplication was present in the majority of publication, increasing over time despite marked improvements in statistical reporting over the last decade. This trend generalised beyond the FXS literature to rodent studies of neurological disorders at large between 2012 and 2024, suggesting that pseudoreplication remains a widespread issue in the literature. Limitations The scope of this study was limited to rodent-model studies of neurological disorders which had the potential for being pseudoreplicated, by allowing repeat observations from individual animals. We did not consider reviews or articles whose experimental design could not allow for pseudoreplication, for example studies which reported only behavioural results, or studies which did not use inferential statistics. Conclusions These observations identify an urgent need for better standards in experimental design and increased vigilance for this type of error during peer review. While reporting standards have significantly improved over the past two decades, this alone has not been enough to curb the prevalence of pseudoreplication. We offer suggestions for how this can be remedied as well as quantifying the severity of this particular type of statistical error. Although the examined literature concerns a specific neuroscience-related area of research, the implications of pseudoreplication apply to all fields of empirical research. |
| format | Article |
| id | doaj-art-cc6766ffc52e49bf883ad5b77e1ced41 |
| institution | DOAJ |
| issn | 2040-2392 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | BMC |
| record_format | Article |
| series | Molecular Autism |
| spelling | doaj-art-cc6766ffc52e49bf883ad5b77e1ced412025-08-20T03:16:31ZengBMCMolecular Autism2040-23922025-05-0116111010.1186/s13229-025-00663-3Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disordersConstantinos Eleftheriou0Sarah Giachetti1Raven Hickson2Laura Kamnioti-Dumont3Robert Templaar4Alina Aaltonen5Eleni Tsoukala6Nawon Kim7Lysandra Fryer-Petridis8Chloe Henley9Ceren Erdem10Emma Wilson11Beatriz Maio12Jingjing Ye13Jessica C. Pierce14Kath Mazur15Lucia Landa-Navarro16Nina G. Petrović17Sarah Bendova18Hanan Woods19Manuela Rizzi20Vanesa Salazar-Sanchez21Natasha Anstey22Antonios Asiminas23Shinjini Basu24Sam A. Booker25Anjanette Harris26Sam Heyes27Adam Jackson28Alex Crocker-Buque29Aoife C. McMahon30Sally M. Till31Lasani S. Wijetunge32David JA Wyllie33Catherine M. Abbott34Timothy O’Leary35Peter C. Kind36Simons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghCentre for Translational Neuromedicine, University of CopenhagenCentre for Discovery Brain Sciences, Deanery of Biomedical Sciences, Edinburgh Medical School, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghCentre for Discovery Brain Sciences, Deanery of Biomedical Sciences, Edinburgh Medical School, University of EdinburghCentre for Discovery Brain Sciences, Deanery of Biomedical Sciences, Edinburgh Medical School, University of EdinburghEuropean Molecular Biology Laboratory, European Bioinformatics InstituteSimons Initiative for the Developing Brain, University of EdinburghCentre for Discovery Brain Sciences, Deanery of Biomedical Sciences, Edinburgh Medical School, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghSimons Initiative for the Developing Brain, University of EdinburghDepartment of Engineering, University of CambridgeSimons Initiative for the Developing Brain, University of EdinburghAbstract Background Accurately determining the sample size (“N”) of a dataset is a key consideration for experimental design. Misidentification of sample size can lead to pseudoreplication, a process of artificially inflating the number of experimental replicates which systematically underestimates variability, overestimates effect sizes and invalidates statistical tests performed on the data. While many journals have adopted stringent requirements with regard to statistical reporting over the last decade, it remains unknown whether such efforts have had a meaningful impact on statistical rigour. Methods Here, we evaluated the prevalence of this type of statistical error among neuroscience studies involving animal models of Fragile-X Syndrome (FXS) and those using animal models of neurological disorders at large published between 2001 and 2024. Results We found that pseudoreplication was present in the majority of publication, increasing over time despite marked improvements in statistical reporting over the last decade. This trend generalised beyond the FXS literature to rodent studies of neurological disorders at large between 2012 and 2024, suggesting that pseudoreplication remains a widespread issue in the literature. Limitations The scope of this study was limited to rodent-model studies of neurological disorders which had the potential for being pseudoreplicated, by allowing repeat observations from individual animals. We did not consider reviews or articles whose experimental design could not allow for pseudoreplication, for example studies which reported only behavioural results, or studies which did not use inferential statistics. Conclusions These observations identify an urgent need for better standards in experimental design and increased vigilance for this type of error during peer review. While reporting standards have significantly improved over the past two decades, this alone has not been enough to curb the prevalence of pseudoreplication. We offer suggestions for how this can be remedied as well as quantifying the severity of this particular type of statistical error. Although the examined literature concerns a specific neuroscience-related area of research, the implications of pseudoreplication apply to all fields of empirical research.https://doi.org/10.1186/s13229-025-00663-3AutismPseudoreplicationStatisticsAnimal modelsFragile X |
| spellingShingle | Constantinos Eleftheriou Sarah Giachetti Raven Hickson Laura Kamnioti-Dumont Robert Templaar Alina Aaltonen Eleni Tsoukala Nawon Kim Lysandra Fryer-Petridis Chloe Henley Ceren Erdem Emma Wilson Beatriz Maio Jingjing Ye Jessica C. Pierce Kath Mazur Lucia Landa-Navarro Nina G. Petrović Sarah Bendova Hanan Woods Manuela Rizzi Vanesa Salazar-Sanchez Natasha Anstey Antonios Asiminas Shinjini Basu Sam A. Booker Anjanette Harris Sam Heyes Adam Jackson Alex Crocker-Buque Aoife C. McMahon Sally M. Till Lasani S. Wijetunge David JA Wyllie Catherine M. Abbott Timothy O’Leary Peter C. Kind Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders Molecular Autism Autism Pseudoreplication Statistics Animal models Fragile X |
| title | Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders |
| title_full | Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders |
| title_fullStr | Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders |
| title_full_unstemmed | Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders |
| title_short | Better statistical reporting does not lead to statistical rigour: lessons from two decades of pseudoreplication in mouse-model studies of neurological disorders |
| title_sort | better statistical reporting does not lead to statistical rigour lessons from two decades of pseudoreplication in mouse model studies of neurological disorders |
| topic | Autism Pseudoreplication Statistics Animal models Fragile X |
| url | https://doi.org/10.1186/s13229-025-00663-3 |
| work_keys_str_mv | AT constantinoseleftheriou betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT sarahgiachetti betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT ravenhickson betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT laurakamniotidumont betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT roberttemplaar betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT alinaaaltonen betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT elenitsoukala betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT nawonkim betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT lysandrafryerpetridis betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT chloehenley betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT cerenerdem betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT emmawilson betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT beatrizmaio betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT jingjingye betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT jessicacpierce betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT kathmazur betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT lucialandanavarro betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT ninagpetrovic betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT sarahbendova betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT hananwoods betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT manuelarizzi betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT vanesasalazarsanchez betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT natashaanstey betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT antoniosasiminas betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT shinjinibasu betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT samabooker betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT anjanetteharris betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT samheyes betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT adamjackson betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT alexcrockerbuque betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT aoifecmcmahon betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT sallymtill betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT lasaniswijetunge betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT davidjawyllie betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT catherinemabbott betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT timothyoleary betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders AT peterckind betterstatisticalreportingdoesnotleadtostatisticalrigourlessonsfromtwodecadesofpseudoreplicationinmousemodelstudiesofneurologicaldisorders |