Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case study
BackgroundThe use of ChatGPT in healthcare is still in its early stages; however, it has the potential to become a cornerstone in modern healthcare systems. This study aims to assess the accuracy of output of ChatGPT compared with those of CareNotes® in providing patient instructions for three medic...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-04-01
|
| Series: | Frontiers in Artificial Intelligence |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frai.2025.1550591/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850275484351332352 |
|---|---|
| author | Norah Othman Abanmy Nadia Al-Ghreimil Jawza F. Alsabhan Heyam Al-Baity Rana Aljadeed |
| author_facet | Norah Othman Abanmy Nadia Al-Ghreimil Jawza F. Alsabhan Heyam Al-Baity Rana Aljadeed |
| author_sort | Norah Othman Abanmy |
| collection | DOAJ |
| description | BackgroundThe use of ChatGPT in healthcare is still in its early stages; however, it has the potential to become a cornerstone in modern healthcare systems. This study aims to assess the accuracy of output of ChatGPT compared with those of CareNotes® in providing patient instructions for three medications: tirzepatide, citalopram, and apixaban.MethodsAn exploratory case study was conducted using a published questionnaire to evaluate ChatGPT-generated reports against patient instructions from CareNotes®. The evaluation focused on the completeness and correctness of the reports, as well as their potential to cause harm or lead to poor medication adherence. The evaluation was conducted by four pharmacy experts and 33 PharmD interns.ResultsThe evaluators indicated that the ChatGPT reports of tirzepatide, citalopram, and apixaban were correct but lacked completeness. Additionally, ChatGPT reports have the potential to cause harm and may negatively affect medication adherence.ConclusionAlthough ChatGPT demonstrated promising results, particularly in terms of correctness, it cannot yet be considered a reliable standalone source of patient drug information. |
| format | Article |
| id | doaj-art-aee821e12503416eac83620247c754db |
| institution | OA Journals |
| issn | 2624-8212 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Artificial Intelligence |
| spelling | doaj-art-aee821e12503416eac83620247c754db2025-08-20T01:50:42ZengFrontiers Media S.A.Frontiers in Artificial Intelligence2624-82122025-04-01810.3389/frai.2025.15505911550591Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case studyNorah Othman Abanmy0Nadia Al-Ghreimil1Jawza F. Alsabhan2Heyam Al-Baity3Rana Aljadeed4Department of Clinical Pharmacy, College of Pharmacy, King Saud University, Riyadh, Saudi ArabiaInformation Technology Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi ArabiaDepartment of Clinical Pharmacy, College of Pharmacy, King Saud University, Riyadh, Saudi ArabiaInformation Technology Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi ArabiaDepartment of Clinical Pharmacy, College of Pharmacy, King Saud University, Riyadh, Saudi ArabiaBackgroundThe use of ChatGPT in healthcare is still in its early stages; however, it has the potential to become a cornerstone in modern healthcare systems. This study aims to assess the accuracy of output of ChatGPT compared with those of CareNotes® in providing patient instructions for three medications: tirzepatide, citalopram, and apixaban.MethodsAn exploratory case study was conducted using a published questionnaire to evaluate ChatGPT-generated reports against patient instructions from CareNotes®. The evaluation focused on the completeness and correctness of the reports, as well as their potential to cause harm or lead to poor medication adherence. The evaluation was conducted by four pharmacy experts and 33 PharmD interns.ResultsThe evaluators indicated that the ChatGPT reports of tirzepatide, citalopram, and apixaban were correct but lacked completeness. Additionally, ChatGPT reports have the potential to cause harm and may negatively affect medication adherence.ConclusionAlthough ChatGPT demonstrated promising results, particularly in terms of correctness, it cannot yet be considered a reliable standalone source of patient drug information.https://www.frontiersin.org/articles/10.3389/frai.2025.1550591/fullChatGPThealthcare technologypharmacy applicationsdrug informationartificial intelligence |
| spellingShingle | Norah Othman Abanmy Nadia Al-Ghreimil Jawza F. Alsabhan Heyam Al-Baity Rana Aljadeed Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case study Frontiers in Artificial Intelligence ChatGPT healthcare technology pharmacy applications drug information artificial intelligence |
| title | Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case study |
| title_full | Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case study |
| title_fullStr | Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case study |
| title_full_unstemmed | Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case study |
| title_short | Evaluating the accuracy of ChatGPT in delivering patient instructions for medications: an exploratory case study |
| title_sort | evaluating the accuracy of chatgpt in delivering patient instructions for medications an exploratory case study |
| topic | ChatGPT healthcare technology pharmacy applications drug information artificial intelligence |
| url | https://www.frontiersin.org/articles/10.3389/frai.2025.1550591/full |
| work_keys_str_mv | AT norahothmanabanmy evaluatingtheaccuracyofchatgptindeliveringpatientinstructionsformedicationsanexploratorycasestudy AT nadiaalghreimil evaluatingtheaccuracyofchatgptindeliveringpatientinstructionsformedicationsanexploratorycasestudy AT jawzafalsabhan evaluatingtheaccuracyofchatgptindeliveringpatientinstructionsformedicationsanexploratorycasestudy AT heyamalbaity evaluatingtheaccuracyofchatgptindeliveringpatientinstructionsformedicationsanexploratorycasestudy AT ranaaljadeed evaluatingtheaccuracyofchatgptindeliveringpatientinstructionsformedicationsanexploratorycasestudy |