ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research
BackgroundChatGPT, a conversational artificial intelligence developed by OpenAI, has rapidly become an invaluable tool for researchers. With the recent integration of Python code interpretation into the ChatGPT environment, there has been a significant increase in the potenti...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-02-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e63550 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825201923719954432 |
|---|---|
| author | Michael R Ruta Tony Gaidici Chase Irwin Jonathan Lifshitz |
| author_facet | Michael R Ruta Tony Gaidici Chase Irwin Jonathan Lifshitz |
| author_sort | Michael R Ruta |
| collection | DOAJ |
| description |
BackgroundChatGPT, a conversational artificial intelligence developed by OpenAI, has rapidly become an invaluable tool for researchers. With the recent integration of Python code interpretation into the ChatGPT environment, there has been a significant increase in the potential utility of ChatGPT as a research tool, particularly in terms of data analysis applications.
ObjectiveThis study aimed to assess ChatGPT as a data analysis tool and provide researchers with a framework for applying ChatGPT to data management tasks, descriptive statistics, and inferential statistics.
MethodsA subset of the National Inpatient Sample was extracted. Data analysis trials were divided into data processing, categorization, and tabulation, as well as descriptive and inferential statistics. For data processing, categorization, and tabulation assessments, ChatGPT was prompted to reclassify variables, subset variables, and present data, respectively. Descriptive statistics assessments included mean, SD, median, and IQR calculations. Inferential statistics assessments were conducted at varying levels of prompt specificity (“Basic,” “Intermediate,” and “Advanced”). Specific tests included chi-square, Pearson correlation, independent 2-sample t test, 1-way ANOVA, Fisher exact, Spearman correlation, Mann-Whitney U test, and Kruskal-Wallis H test. Outcomes from consecutive prompt-based trials were assessed against expected statistical values calculated in Python (Python Software Foundation), SAS (SAS Institute), and RStudio (Posit PBC).
ResultsChatGPT accurately performed data processing, categorization, and tabulation across all trials. For descriptive statistics, it provided accurate means, SDs, medians, and IQRs across all trials. Inferential statistics accuracy against expected statistical values varied with prompt specificity: 32.5% accuracy for “Basic” prompts, 81.3% for “Intermediate” prompts, and 92.5% for “Advanced” prompts.
ConclusionsChatGPT shows promise as a tool for exploratory data analysis, particularly for researchers with some statistical knowledge and limited programming expertise. However, its application requires careful prompt construction and human oversight to ensure accuracy. As a supplementary tool, ChatGPT can enhance data analysis efficiency and broaden research accessibility. |
| format | Article |
| id | doaj-art-8b5e23897c5741f19380157b76df5d8d |
| institution | Kabale University |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-8b5e23897c5741f19380157b76df5d8d2025-02-07T17:30:31ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-02-0127e6355010.2196/63550ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare ResearchMichael R Rutahttps://orcid.org/0009-0003-1246-3360Tony Gaidicihttps://orcid.org/0009-0001-8175-0713Chase Irwinhttps://orcid.org/0000-0002-3165-0693Jonathan Lifshitzhttps://orcid.org/0000-0002-4398-6493 BackgroundChatGPT, a conversational artificial intelligence developed by OpenAI, has rapidly become an invaluable tool for researchers. With the recent integration of Python code interpretation into the ChatGPT environment, there has been a significant increase in the potential utility of ChatGPT as a research tool, particularly in terms of data analysis applications. ObjectiveThis study aimed to assess ChatGPT as a data analysis tool and provide researchers with a framework for applying ChatGPT to data management tasks, descriptive statistics, and inferential statistics. MethodsA subset of the National Inpatient Sample was extracted. Data analysis trials were divided into data processing, categorization, and tabulation, as well as descriptive and inferential statistics. For data processing, categorization, and tabulation assessments, ChatGPT was prompted to reclassify variables, subset variables, and present data, respectively. Descriptive statistics assessments included mean, SD, median, and IQR calculations. Inferential statistics assessments were conducted at varying levels of prompt specificity (“Basic,” “Intermediate,” and “Advanced”). Specific tests included chi-square, Pearson correlation, independent 2-sample t test, 1-way ANOVA, Fisher exact, Spearman correlation, Mann-Whitney U test, and Kruskal-Wallis H test. Outcomes from consecutive prompt-based trials were assessed against expected statistical values calculated in Python (Python Software Foundation), SAS (SAS Institute), and RStudio (Posit PBC). ResultsChatGPT accurately performed data processing, categorization, and tabulation across all trials. For descriptive statistics, it provided accurate means, SDs, medians, and IQRs across all trials. Inferential statistics accuracy against expected statistical values varied with prompt specificity: 32.5% accuracy for “Basic” prompts, 81.3% for “Intermediate” prompts, and 92.5% for “Advanced” prompts. ConclusionsChatGPT shows promise as a tool for exploratory data analysis, particularly for researchers with some statistical knowledge and limited programming expertise. However, its application requires careful prompt construction and human oversight to ensure accuracy. As a supplementary tool, ChatGPT can enhance data analysis efficiency and broaden research accessibility.https://www.jmir.org/2025/1/e63550 |
| spellingShingle | Michael R Ruta Tony Gaidici Chase Irwin Jonathan Lifshitz ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research Journal of Medical Internet Research |
| title | ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research |
| title_full | ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research |
| title_fullStr | ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research |
| title_full_unstemmed | ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research |
| title_short | ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research |
| title_sort | chatgpt for univariate statistics validation of ai assisted data analysis in healthcare research |
| url | https://www.jmir.org/2025/1/e63550 |
| work_keys_str_mv | AT michaelrruta chatgptforunivariatestatisticsvalidationofaiassisteddataanalysisinhealthcareresearch AT tonygaidici chatgptforunivariatestatisticsvalidationofaiassisteddataanalysisinhealthcareresearch AT chaseirwin chatgptforunivariatestatisticsvalidationofaiassisteddataanalysisinhealthcareresearch AT jonathanlifshitz chatgptforunivariatestatisticsvalidationofaiassisteddataanalysisinhealthcareresearch |