Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses
Nonsignificant findings in psychological research are frequently misinterpreted as reflecting the effect’s absence. However, this issue’s exact prevalence remains unclear, as does whether this issue is getting better or worse. In this pre-registered study, we sought to answer these questions by exam...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The Royal Society
2025-03-01
|
| Series: | Royal Society Open Science |
| Subjects: | |
| Online Access: | https://royalsocietypublishing.org/doi/10.1098/rsos.242167 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850038161358454784 |
|---|---|
| author | Stephen Lee Murphy Raphael Merz Linda-Elisabeth Reimann Aurelio Fernández |
| author_facet | Stephen Lee Murphy Raphael Merz Linda-Elisabeth Reimann Aurelio Fernández |
| author_sort | Stephen Lee Murphy |
| collection | DOAJ |
| description | Nonsignificant findings in psychological research are frequently misinterpreted as reflecting the effect’s absence. However, this issue’s exact prevalence remains unclear, as does whether this issue is getting better or worse. In this pre-registered study, we sought to answer these questions by examining the discussion sections of 599 articles published across 10 psychology journals and three time points (2009, 2015 and 2021), and coding whether a nonsignificant finding was interpreted in such a way as to suggest the effect does not exist. Our models indicate that between 76% and 85% of psychology articles published between 2009 and 2021 that discussed a nonsignificant finding misinterpreted nonsignificance as reflecting no effect. It is likely between 54% and 62% of articles over this time period claimed explicitly that this meant no effect on the population of interest. Our findings also indicate that only between 4% and 8% of articles explicitly discussed the possibility that the nonsignificant effect may exist but could not be found. Differences in prevalence rates over time were nonsignificant. Collectively, our findings indicate this interpretative error is a major problem in psychology. We call on stakeholders with an interest in improving psychological science to prioritize tackling it. |
| format | Article |
| id | doaj-art-c1613d83cba2421898ceff521b59a6ee |
| institution | DOAJ |
| issn | 2054-5703 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | The Royal Society |
| record_format | Article |
| series | Royal Society Open Science |
| spelling | doaj-art-c1613d83cba2421898ceff521b59a6ee2025-08-20T02:56:39ZengThe Royal SocietyRoyal Society Open Science2054-57032025-03-0112310.1098/rsos.242167Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analysesStephen Lee Murphy0Raphael Merz1Linda-Elisabeth Reimann2Aurelio Fernández3Ghent University, Ghent, Flanders, BelgiumRuhr University Bochum, Bochum, North Rhine-Westphalia, GermanyUniversity of Münster, Münster, North Rhine-Westphalia, GermanyGhent University, Ghent, Flanders, BelgiumNonsignificant findings in psychological research are frequently misinterpreted as reflecting the effect’s absence. However, this issue’s exact prevalence remains unclear, as does whether this issue is getting better or worse. In this pre-registered study, we sought to answer these questions by examining the discussion sections of 599 articles published across 10 psychology journals and three time points (2009, 2015 and 2021), and coding whether a nonsignificant finding was interpreted in such a way as to suggest the effect does not exist. Our models indicate that between 76% and 85% of psychology articles published between 2009 and 2021 that discussed a nonsignificant finding misinterpreted nonsignificance as reflecting no effect. It is likely between 54% and 62% of articles over this time period claimed explicitly that this meant no effect on the population of interest. Our findings also indicate that only between 4% and 8% of articles explicitly discussed the possibility that the nonsignificant effect may exist but could not be found. Differences in prevalence rates over time were nonsignificant. Collectively, our findings indicate this interpretative error is a major problem in psychology. We call on stakeholders with an interest in improving psychological science to prioritize tackling it.https://royalsocietypublishing.org/doi/10.1098/rsos.242167non-significancemetasciencemetaresearchmisinterpretationstatistics |
| spellingShingle | Stephen Lee Murphy Raphael Merz Linda-Elisabeth Reimann Aurelio Fernández Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses Royal Society Open Science non-significance metascience metaresearch misinterpretation statistics |
| title | Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses |
| title_full | Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses |
| title_fullStr | Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses |
| title_full_unstemmed | Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses |
| title_short | Nonsignificance misinterpreted as an effect’s absence in psychology: prevalence and temporal analyses |
| title_sort | nonsignificance misinterpreted as an effect s absence in psychology prevalence and temporal analyses |
| topic | non-significance metascience metaresearch misinterpretation statistics |
| url | https://royalsocietypublishing.org/doi/10.1098/rsos.242167 |
| work_keys_str_mv | AT stephenleemurphy nonsignificancemisinterpretedasaneffectsabsenceinpsychologyprevalenceandtemporalanalyses AT raphaelmerz nonsignificancemisinterpretedasaneffectsabsenceinpsychologyprevalenceandtemporalanalyses AT lindaelisabethreimann nonsignificancemisinterpretedasaneffectsabsenceinpsychologyprevalenceandtemporalanalyses AT aureliofernandez nonsignificancemisinterpretedasaneffectsabsenceinpsychologyprevalenceandtemporalanalyses |