Effect size and statistical power in the rodent fear conditioning literature - A systematic review.
Proposals to increase research reproducibility frequently call for focusing on effect sizes instead of p values, as well as for increasing the statistical power of experiments. However, it is unclear to what extent these two concepts are indeed taken into account in basic biomedical science. To stud...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2018-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0196258 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849682937005473792 |
|---|---|
| author | Clarissa F D Carneiro Thiago C Moulin Malcolm R Macleod Olavo B Amaral |
| author_facet | Clarissa F D Carneiro Thiago C Moulin Malcolm R Macleod Olavo B Amaral |
| author_sort | Clarissa F D Carneiro |
| collection | DOAJ |
| description | Proposals to increase research reproducibility frequently call for focusing on effect sizes instead of p values, as well as for increasing the statistical power of experiments. However, it is unclear to what extent these two concepts are indeed taken into account in basic biomedical science. To study this in a real-case scenario, we performed a systematic review of effect sizes and statistical power in studies on learning of rodent fear conditioning, a widely used behavioral task to evaluate memory. Our search criteria yielded 410 experiments comparing control and treated groups in 122 articles. Interventions had a mean effect size of 29.5%, and amnesia caused by memory-impairing interventions was nearly always partial. Mean statistical power to detect the average effect size observed in well-powered experiments with significant differences (37.2%) was 65%, and was lower among studies with non-significant results. Only one article reported a sample size calculation, and our estimated sample size to achieve 80% power considering typical effect sizes and variances (15 animals per group) was reached in only 12.2% of experiments. Actual effect sizes correlated with effect size inferences made by readers on the basis of textual descriptions of results only when findings were non-significant, and neither effect size nor power correlated with study quality indicators, number of citations or impact factor of the publishing journal. In summary, effect sizes and statistical power have a wide distribution in the rodent fear conditioning literature, but do not seem to have a large influence on how results are described or cited. Failure to take these concepts into consideration might limit attempts to improve reproducibility in this field of science. |
| format | Article |
| id | doaj-art-443d49b88fde49779943388cc8e27fef |
| institution | DOAJ |
| issn | 1932-6203 |
| language | English |
| publishDate | 2018-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-443d49b88fde49779943388cc8e27fef2025-08-20T03:24:02ZengPublic Library of Science (PLoS)PLoS ONE1932-62032018-01-01134e019625810.1371/journal.pone.0196258Effect size and statistical power in the rodent fear conditioning literature - A systematic review.Clarissa F D CarneiroThiago C MoulinMalcolm R MacleodOlavo B AmaralProposals to increase research reproducibility frequently call for focusing on effect sizes instead of p values, as well as for increasing the statistical power of experiments. However, it is unclear to what extent these two concepts are indeed taken into account in basic biomedical science. To study this in a real-case scenario, we performed a systematic review of effect sizes and statistical power in studies on learning of rodent fear conditioning, a widely used behavioral task to evaluate memory. Our search criteria yielded 410 experiments comparing control and treated groups in 122 articles. Interventions had a mean effect size of 29.5%, and amnesia caused by memory-impairing interventions was nearly always partial. Mean statistical power to detect the average effect size observed in well-powered experiments with significant differences (37.2%) was 65%, and was lower among studies with non-significant results. Only one article reported a sample size calculation, and our estimated sample size to achieve 80% power considering typical effect sizes and variances (15 animals per group) was reached in only 12.2% of experiments. Actual effect sizes correlated with effect size inferences made by readers on the basis of textual descriptions of results only when findings were non-significant, and neither effect size nor power correlated with study quality indicators, number of citations or impact factor of the publishing journal. In summary, effect sizes and statistical power have a wide distribution in the rodent fear conditioning literature, but do not seem to have a large influence on how results are described or cited. Failure to take these concepts into consideration might limit attempts to improve reproducibility in this field of science.https://doi.org/10.1371/journal.pone.0196258 |
| spellingShingle | Clarissa F D Carneiro Thiago C Moulin Malcolm R Macleod Olavo B Amaral Effect size and statistical power in the rodent fear conditioning literature - A systematic review. PLoS ONE |
| title | Effect size and statistical power in the rodent fear conditioning literature - A systematic review. |
| title_full | Effect size and statistical power in the rodent fear conditioning literature - A systematic review. |
| title_fullStr | Effect size and statistical power in the rodent fear conditioning literature - A systematic review. |
| title_full_unstemmed | Effect size and statistical power in the rodent fear conditioning literature - A systematic review. |
| title_short | Effect size and statistical power in the rodent fear conditioning literature - A systematic review. |
| title_sort | effect size and statistical power in the rodent fear conditioning literature a systematic review |
| url | https://doi.org/10.1371/journal.pone.0196258 |
| work_keys_str_mv | AT clarissafdcarneiro effectsizeandstatisticalpowerintherodentfearconditioningliteratureasystematicreview AT thiagocmoulin effectsizeandstatisticalpowerintherodentfearconditioningliteratureasystematicreview AT malcolmrmacleod effectsizeandstatisticalpowerintherodentfearconditioningliteratureasystematicreview AT olavobamaral effectsizeandstatisticalpowerintherodentfearconditioningliteratureasystematicreview |