Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis
Translation of spinal cord injury (SCI) therapeutics from pre-clinical animal studies into human studies is challenged by effect size variability, irreproducibility, and misalignment of evidence used by pre-clinical versus clinical literature. Clinical literature values reproducibility, with the hig...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Mary Ann Liebert
2024-11-01
|
| Series: | Neurotrauma Reports |
| Subjects: | |
| Online Access: | https://www.liebertpub.com/doi/10.1089/neur.2024.0038 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849321827660201984 |
|---|---|
| author | Emma G. Iorio Alireza Khanteymoori Kenneth A. Fond Anastasia V. Keller Lex Maliga Davis Jan M. Schwab Adam R. Ferguson Abel Torres-Espin Ralf Watzlawick |
| author_facet | Emma G. Iorio Alireza Khanteymoori Kenneth A. Fond Anastasia V. Keller Lex Maliga Davis Jan M. Schwab Adam R. Ferguson Abel Torres-Espin Ralf Watzlawick |
| author_sort | Emma G. Iorio |
| collection | DOAJ |
| description | Translation of spinal cord injury (SCI) therapeutics from pre-clinical animal studies into human studies is challenged by effect size variability, irreproducibility, and misalignment of evidence used by pre-clinical versus clinical literature. Clinical literature values reproducibility, with the highest grade evidence (class 1) consisting of meta-analysis demonstrating large therapeutic efficacy replicating across multiple studies. Conversely, pre-clinical literature values novelty over replication and lacks rigorous meta-analyses to assess reproducibility of effect sizes across multiple articles. Here, we applied modified clinical meta-analysis methods to pre-clinical studies, comparing effect sizes extracted from published literature to raw data on individual animals from these same studies. Literature-extracted data (LED) from numerical and graphical outcomes reported in publications were compared with individual animal data (IAD) deposited in a federally supported repository of SCI data. The animal groups from the IAD were matched with the same cohorts in the LED for a direct comparison. We applied random-effects meta-analysis to evaluate predictors of neuroconversion in LED versus IAD. We included publications with common injury models (contusive injuries) and standardized end-points (open field assessments). The extraction of data from 25 published articles yielded n = 1841 subjects, whereas IAD from these same articles included n = 2441 subjects. We observed differences in the number of experimental groups and animals per group, insufficient reporting of dropout animals, and missing information on experimental details. Meta-analysis revealed differences in effect sizes across LED versus IAD stratifications, for instance, severe injuries had the largest effect size in LED (standardized mean difference [SMD = 4.92]), but mild injuries had the largest effect size in IAD (SMD = 6.06). Publications with smaller sample sizes yielded larger effect sizes, while studies with larger sample sizes had smaller effects. The results demonstrate the feasibility of combining IAD analysis with traditional LED meta-analysis to assess effect size reproducibility in SCI. |
| format | Article |
| id | doaj-art-0f16cf9024974ca5bc5af0aa459ce257 |
| institution | Kabale University |
| issn | 2689-288X |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Mary Ann Liebert |
| record_format | Article |
| series | Neurotrauma Reports |
| spelling | doaj-art-0f16cf9024974ca5bc5af0aa459ce2572025-08-20T03:49:37ZengMary Ann LiebertNeurotrauma Reports2689-288X2024-11-015168669810.1089/neur.2024.0038Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-AnalysisEmma G. Iorio0Alireza Khanteymoori1Kenneth A. Fond2Anastasia V. Keller3Lex Maliga Davis4Jan M. Schwab5Adam R. Ferguson6Abel Torres-Espin7Ralf Watzlawick8Department of Neurological Surgery, University of California, San Francisco, California, USA.Department of Neurosurgery, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany.Department of Neurological Surgery, University of California, San Francisco, California, USA.Department of Neurological Surgery, University of California, San Francisco, California, USA.Department of Neurological Surgery, University of California, San Francisco, California, USA.Departments of Neurology and Neurosciences, The Ohio State University, Columbus, Ohio, USA.Department of Neurological Surgery, University of California, San Francisco, California, USA.Department of Neurological Surgery, University of California, San Francisco, California, USA.Department of Neurosurgery, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany.Translation of spinal cord injury (SCI) therapeutics from pre-clinical animal studies into human studies is challenged by effect size variability, irreproducibility, and misalignment of evidence used by pre-clinical versus clinical literature. Clinical literature values reproducibility, with the highest grade evidence (class 1) consisting of meta-analysis demonstrating large therapeutic efficacy replicating across multiple studies. Conversely, pre-clinical literature values novelty over replication and lacks rigorous meta-analyses to assess reproducibility of effect sizes across multiple articles. Here, we applied modified clinical meta-analysis methods to pre-clinical studies, comparing effect sizes extracted from published literature to raw data on individual animals from these same studies. Literature-extracted data (LED) from numerical and graphical outcomes reported in publications were compared with individual animal data (IAD) deposited in a federally supported repository of SCI data. The animal groups from the IAD were matched with the same cohorts in the LED for a direct comparison. We applied random-effects meta-analysis to evaluate predictors of neuroconversion in LED versus IAD. We included publications with common injury models (contusive injuries) and standardized end-points (open field assessments). The extraction of data from 25 published articles yielded n = 1841 subjects, whereas IAD from these same articles included n = 2441 subjects. We observed differences in the number of experimental groups and animals per group, insufficient reporting of dropout animals, and missing information on experimental details. Meta-analysis revealed differences in effect sizes across LED versus IAD stratifications, for instance, severe injuries had the largest effect size in LED (standardized mean difference [SMD = 4.92]), but mild injuries had the largest effect size in IAD (SMD = 6.06). Publications with smaller sample sizes yielded larger effect sizes, while studies with larger sample sizes had smaller effects. The results demonstrate the feasibility of combining IAD analysis with traditional LED meta-analysis to assess effect size reproducibility in SCI.https://www.liebertpub.com/doi/10.1089/neur.2024.0038individual animal data (IAD)literature-extracted data (LED)meta-analysismeta-sciencespinal cord injury (SCI)systematic review |
| spellingShingle | Emma G. Iorio Alireza Khanteymoori Kenneth A. Fond Anastasia V. Keller Lex Maliga Davis Jan M. Schwab Adam R. Ferguson Abel Torres-Espin Ralf Watzlawick Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis Neurotrauma Reports individual animal data (IAD) literature-extracted data (LED) meta-analysis meta-science spinal cord injury (SCI) systematic review |
| title | Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis |
| title_full | Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis |
| title_fullStr | Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis |
| title_full_unstemmed | Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis |
| title_short | Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis |
| title_sort | effect size discrepancies in literature versus raw datasets from experimental spinal cord injury studies a climber meta analysis |
| topic | individual animal data (IAD) literature-extracted data (LED) meta-analysis meta-science spinal cord injury (SCI) systematic review |
| url | https://www.liebertpub.com/doi/10.1089/neur.2024.0038 |
| work_keys_str_mv | AT emmagiorio effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT alirezakhanteymoori effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT kennethafond effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT anastasiavkeller effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT lexmaligadavis effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT janmschwab effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT adamrferguson effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT abeltorresespin effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis AT ralfwatzlawick effectsizediscrepanciesinliteratureversusrawdatasetsfromexperimentalspinalcordinjurystudiesaclimbermetaanalysis |