Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review.
<h4>Background</h4>The pathway from evidence generation to consumption contains many steps which can lead to overstatement or misinformation. The proliferation of internet-based health news may encourage selection of media and academic research articles that overstate strength of causal...
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2018-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0196346&type=printable |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850168942211891200 |
|---|---|
| author | Noah Haber Emily R Smith Ellen Moscoe Kathryn Andrews Robin Audy Winnie Bell Alana T Brennan Alexander Breskin Jeremy C Kane Mahesh Karra Elizabeth S McClure Elizabeth A Suarez CLAIMS research team |
| author_facet | Noah Haber Emily R Smith Ellen Moscoe Kathryn Andrews Robin Audy Winnie Bell Alana T Brennan Alexander Breskin Jeremy C Kane Mahesh Karra Elizabeth S McClure Elizabeth A Suarez CLAIMS research team |
| author_sort | Noah Haber |
| collection | DOAJ |
| description | <h4>Background</h4>The pathway from evidence generation to consumption contains many steps which can lead to overstatement or misinformation. The proliferation of internet-based health news may encourage selection of media and academic research articles that overstate strength of causal inference. We investigated the state of causal inference in health research as it appears at the end of the pathway, at the point of social media consumption.<h4>Methods</h4>We screened the NewsWhip Insights database for the most shared media articles on Facebook and Twitter reporting about peer-reviewed academic studies associating an exposure with a health outcome in 2015, extracting the 50 most-shared academic articles and media articles covering them. We designed and utilized a review tool to systematically assess and summarize studies' strength of causal inference, including generalizability, potential confounders, and methods used. These were then compared with the strength of causal language used to describe results in both academic and media articles. Two randomly assigned independent reviewers and one arbitrating reviewer from a pool of 21 reviewers assessed each article.<h4>Results</h4>We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015. Thirty-four percent of academic studies and 48% of media articles used language that reviewers considered too strong for their strength of causal inference. Seventy percent of academic studies were considered low or very low strength of inference, with only 6% considered high or very high strength of causal inference. The most severe issues with academic studies' causal inference were reported to be omitted confounding variables and generalizability. Fifty-eight percent of media articles were found to have inaccurately reported the question, results, intervention, or population of the academic study.<h4>Conclusions</h4>We find a large disparity between the strength of language as presented to the research consumer and the underlying strength of causal inference among the studies most widely shared on social media. However, because this sample was designed to be representative of the articles selected and shared on social media, it is unlikely to be representative of all academic and media work. More research is needed to determine how academic institutions, media organizations, and social network sharing patterns impact causal inference and language as received by the research consumer. |
| format | Article |
| id | doaj-art-9c056c1924bd4bec94e33601f834b6ad |
| institution | OA Journals |
| issn | 1932-6203 |
| language | English |
| publishDate | 2018-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-9c056c1924bd4bec94e33601f834b6ad2025-08-20T02:20:51ZengPublic Library of Science (PLoS)PLoS ONE1932-62032018-01-01135e019634610.1371/journal.pone.0196346Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review.Noah HaberEmily R SmithEllen MoscoeKathryn AndrewsRobin AudyWinnie BellAlana T BrennanAlexander BreskinJeremy C KaneMahesh KarraElizabeth S McClureElizabeth A SuarezCLAIMS research team<h4>Background</h4>The pathway from evidence generation to consumption contains many steps which can lead to overstatement or misinformation. The proliferation of internet-based health news may encourage selection of media and academic research articles that overstate strength of causal inference. We investigated the state of causal inference in health research as it appears at the end of the pathway, at the point of social media consumption.<h4>Methods</h4>We screened the NewsWhip Insights database for the most shared media articles on Facebook and Twitter reporting about peer-reviewed academic studies associating an exposure with a health outcome in 2015, extracting the 50 most-shared academic articles and media articles covering them. We designed and utilized a review tool to systematically assess and summarize studies' strength of causal inference, including generalizability, potential confounders, and methods used. These were then compared with the strength of causal language used to describe results in both academic and media articles. Two randomly assigned independent reviewers and one arbitrating reviewer from a pool of 21 reviewers assessed each article.<h4>Results</h4>We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015. Thirty-four percent of academic studies and 48% of media articles used language that reviewers considered too strong for their strength of causal inference. Seventy percent of academic studies were considered low or very low strength of inference, with only 6% considered high or very high strength of causal inference. The most severe issues with academic studies' causal inference were reported to be omitted confounding variables and generalizability. Fifty-eight percent of media articles were found to have inaccurately reported the question, results, intervention, or population of the academic study.<h4>Conclusions</h4>We find a large disparity between the strength of language as presented to the research consumer and the underlying strength of causal inference among the studies most widely shared on social media. However, because this sample was designed to be representative of the articles selected and shared on social media, it is unlikely to be representative of all academic and media work. More research is needed to determine how academic institutions, media organizations, and social network sharing patterns impact causal inference and language as received by the research consumer.https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0196346&type=printable |
| spellingShingle | Noah Haber Emily R Smith Ellen Moscoe Kathryn Andrews Robin Audy Winnie Bell Alana T Brennan Alexander Breskin Jeremy C Kane Mahesh Karra Elizabeth S McClure Elizabeth A Suarez CLAIMS research team Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review. PLoS ONE |
| title | Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review. |
| title_full | Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review. |
| title_fullStr | Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review. |
| title_full_unstemmed | Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review. |
| title_short | Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review. |
| title_sort | causal language and strength of inference in academic and media articles shared in social media claims a systematic review |
| url | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0196346&type=printable |
| work_keys_str_mv | AT noahhaber causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT emilyrsmith causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT ellenmoscoe causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT kathrynandrews causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT robinaudy causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT winniebell causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT alanatbrennan causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT alexanderbreskin causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT jeremyckane causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT maheshkarra causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT elizabethsmcclure causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT elizabethasuarez causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview AT claimsresearchteam causallanguageandstrengthofinferenceinacademicandmediaarticlessharedinsocialmediaclaimsasystematicreview |