An academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseases
Abstract Generative Pre-trained Transformer (ChatGPT) is a web-based artificial intelligence assistant with the potential to provide information, answer questions, and make recommendations on various topics. Rare cardiovascular diseases (rCVD) are among the health problems that require specialized k...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-11567-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849344526363131904 |
|---|---|
| author | Samet Sevinç Mustafa Candemir Betül Ayça Yamak Emrullah Kızıltunç Burak Sezenöz Orhan Batur Şahin Salih Topal Yusuf Demir Mehmet Rıdvan Yalçın Asife Şahinarslan |
| author_facet | Samet Sevinç Mustafa Candemir Betül Ayça Yamak Emrullah Kızıltunç Burak Sezenöz Orhan Batur Şahin Salih Topal Yusuf Demir Mehmet Rıdvan Yalçın Asife Şahinarslan |
| author_sort | Samet Sevinç |
| collection | DOAJ |
| description | Abstract Generative Pre-trained Transformer (ChatGPT) is a web-based artificial intelligence assistant with the potential to provide information, answer questions, and make recommendations on various topics. Rare cardiovascular diseases (rCVD) are among the health problems that require specialized knowledge and attention, and web databases provide relatively limited information. In this study, we investigated the accuracy and reliability of ChatGPT’s answers to patients’ possible questions about rCVD. ChatGPT was asked forty questions about rCVD. Based on current guidelines and information, academicians who are experts in their fields evaluated ChatGPT’s answers to these questions. The success of ChatGPT, which has been repeatedly evaluated in classical diseases, was lower in rCVD. The responses to various questions exhibited significant similarity, with some answers including redundant information. In addition, ChatGPT did not give the desired answers to some questions. However, although some answers were longer than necessary, there was very little incorrect information in the answers. Although ChatGPT is competent in obtaining information about rCVD, physicians should clarify the answers given by ChatGPT to patients. Therefore, ChatGPT should be used as an auxiliary information acquisition tool rather than as a primary resource for patients with rCVD. |
| format | Article |
| id | doaj-art-e5c0eaeea7204bb7a8b1368644a4805f |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-e5c0eaeea7204bb7a8b1368644a4805f2025-08-20T03:42:38ZengNature PortfolioScientific Reports2045-23222025-07-011511710.1038/s41598-025-11567-wAn academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseasesSamet Sevinç0Mustafa Candemir1Betül Ayça Yamak2Emrullah Kızıltunç3Burak Sezenöz4Orhan Batur Şahin5Salih Topal6Yusuf Demir7Mehmet Rıdvan Yalçın8Asife Şahinarslan9Department of Cardiology, İstanbul Mehmet Akif Ersoy Thoracic and Cardiovascular Surgery Training and Research HospitalFaculty of Medicine, Department of Cardiology, Gazi UniversityDepartment of Cardiology, Hopa State HospitalFaculty of Medicine, Department of Cardiology, Gazi UniversityFaculty of Medicine, Department of Cardiology, Gazi UniversityFaculty of Medicine, Department of Cardiology, Gazi UniversityFaculty of Medicine, Department of Cardiology, Gazi UniversityÇiğli Training and Research Hospital, Department of Cardiology, Bakırçay UniversityFaculty of Medicine, Department of Cardiology, Gazi UniversityFaculty of Medicine, Department of Cardiology, Gazi UniversityAbstract Generative Pre-trained Transformer (ChatGPT) is a web-based artificial intelligence assistant with the potential to provide information, answer questions, and make recommendations on various topics. Rare cardiovascular diseases (rCVD) are among the health problems that require specialized knowledge and attention, and web databases provide relatively limited information. In this study, we investigated the accuracy and reliability of ChatGPT’s answers to patients’ possible questions about rCVD. ChatGPT was asked forty questions about rCVD. Based on current guidelines and information, academicians who are experts in their fields evaluated ChatGPT’s answers to these questions. The success of ChatGPT, which has been repeatedly evaluated in classical diseases, was lower in rCVD. The responses to various questions exhibited significant similarity, with some answers including redundant information. In addition, ChatGPT did not give the desired answers to some questions. However, although some answers were longer than necessary, there was very little incorrect information in the answers. Although ChatGPT is competent in obtaining information about rCVD, physicians should clarify the answers given by ChatGPT to patients. Therefore, ChatGPT should be used as an auxiliary information acquisition tool rather than as a primary resource for patients with rCVD.https://doi.org/10.1038/s41598-025-11567-wAccuracyArtificial intelligenceChatGPTRare cardiovascular diseaseReliability |
| spellingShingle | Samet Sevinç Mustafa Candemir Betül Ayça Yamak Emrullah Kızıltunç Burak Sezenöz Orhan Batur Şahin Salih Topal Yusuf Demir Mehmet Rıdvan Yalçın Asife Şahinarslan An academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseases Scientific Reports Accuracy Artificial intelligence ChatGPT Rare cardiovascular disease Reliability |
| title | An academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseases |
| title_full | An academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseases |
| title_fullStr | An academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseases |
| title_full_unstemmed | An academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseases |
| title_short | An academic evaluation of ChatGpt’s ability and accuracy in creating patient education resources for rare cardiovascular diseases |
| title_sort | academic evaluation of chatgpt s ability and accuracy in creating patient education resources for rare cardiovascular diseases |
| topic | Accuracy Artificial intelligence ChatGPT Rare cardiovascular disease Reliability |
| url | https://doi.org/10.1038/s41598-025-11567-w |
| work_keys_str_mv | AT sametsevinc anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT mustafacandemir anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT betulaycayamak anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT emrullahkızıltunc anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT buraksezenoz anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT orhanbatursahin anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT salihtopal anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT yusufdemir anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT mehmetrıdvanyalcın anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT asifesahinarslan anacademicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT sametsevinc academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT mustafacandemir academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT betulaycayamak academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT emrullahkızıltunc academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT buraksezenoz academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT orhanbatursahin academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT salihtopal academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT yusufdemir academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT mehmetrıdvanyalcın academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases AT asifesahinarslan academicevaluationofchatgptsabilityandaccuracyincreatingpatienteducationresourcesforrarecardiovasculardiseases |