Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultation
Introduction. ChatGPT has shown promise in medical consultation. This paper presents the preliminary findings of evaluating the capability of ChatGPT in responding to real-world patient questions from a patient point of view, using physician responses as a benchmark. Method. 24 patient questions a...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of Borås
2024-06-01
|
| Series: | Information Research: An International Electronic Journal |
| Subjects: | |
| Online Access: | https://informationr.net/infres/article/view/837 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832544571234451456 |
|---|---|
| author | Siqi Luo Hongyi Qin Hanlin Li Cui Huang |
| author_facet | Siqi Luo Hongyi Qin Hanlin Li Cui Huang |
| author_sort | Siqi Luo |
| collection | DOAJ |
| description | Introduction. ChatGPT has shown promise in medical consultation. This paper presents the preliminary findings of evaluating the capability of ChatGPT in responding to real-world patient questions from a patient point of view, using physician responses as a benchmark.
Method. 24 patient questions and physician responses were collected from a Chinese professional medical consultation platform. The corresponding ChatGPT responses were also collected. Five evaluators without medical background were given the patient questions and responses from both sources in random order. Evaluations were made in terms of the quality and presented empathy of collected responses.
Analysis. Evaluation scores were analysed using descriptive statistical method.
Results. Preliminary findings demonstrated that ChatGPT could be considered as a dependable source for acquiring useful health information. However, it was not able to present the feeling of empathy to patients compared with human physicians.
Conclusion. We recommend that physicians consider utilizing ChatGPT as a supplementary information source when addressing general medical consultations to improve the experience of seeking medical information for patients. |
| format | Article |
| id | doaj-art-b9759743b4244ff18ff8b24f1b08933d |
| institution | Kabale University |
| issn | 1368-1613 |
| language | English |
| publishDate | 2024-06-01 |
| publisher | University of Borås |
| record_format | Article |
| series | Information Research: An International Electronic Journal |
| spelling | doaj-art-b9759743b4244ff18ff8b24f1b08933d2025-02-03T10:10:34ZengUniversity of BoråsInformation Research: An International Electronic Journal1368-16132024-06-0129241942610.47989/ir292837834Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultationSiqi Luo0Hongyi Qin1Hanlin Li2Cui Huang3Zhejiang UniversityZhejiang UniversityZhejiang UniversityZhejiang UniversityIntroduction. ChatGPT has shown promise in medical consultation. This paper presents the preliminary findings of evaluating the capability of ChatGPT in responding to real-world patient questions from a patient point of view, using physician responses as a benchmark. Method. 24 patient questions and physician responses were collected from a Chinese professional medical consultation platform. The corresponding ChatGPT responses were also collected. Five evaluators without medical background were given the patient questions and responses from both sources in random order. Evaluations were made in terms of the quality and presented empathy of collected responses. Analysis. Evaluation scores were analysed using descriptive statistical method. Results. Preliminary findings demonstrated that ChatGPT could be considered as a dependable source for acquiring useful health information. However, it was not able to present the feeling of empathy to patients compared with human physicians. Conclusion. We recommend that physicians consider utilizing ChatGPT as a supplementary information source when addressing general medical consultations to improve the experience of seeking medical information for patients.https://informationr.net/infres/article/view/837chatgpthealth informationmedical consultation |
| spellingShingle | Siqi Luo Hongyi Qin Hanlin Li Cui Huang Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultation Information Research: An International Electronic Journal chatgpt health information medical consultation |
| title | Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultation |
| title_full | Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultation |
| title_fullStr | Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultation |
| title_full_unstemmed | Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultation |
| title_short | Can ChatGPT provide health information as physicians do? Preliminary findings from a cross-sectional study of online medical consultation |
| title_sort | can chatgpt provide health information as physicians do preliminary findings from a cross sectional study of online medical consultation |
| topic | chatgpt health information medical consultation |
| url | https://informationr.net/infres/article/view/837 |
| work_keys_str_mv | AT siqiluo canchatgptprovidehealthinformationasphysiciansdopreliminaryfindingsfromacrosssectionalstudyofonlinemedicalconsultation AT hongyiqin canchatgptprovidehealthinformationasphysiciansdopreliminaryfindingsfromacrosssectionalstudyofonlinemedicalconsultation AT hanlinli canchatgptprovidehealthinformationasphysiciansdopreliminaryfindingsfromacrosssectionalstudyofonlinemedicalconsultation AT cuihuang canchatgptprovidehealthinformationasphysiciansdopreliminaryfindingsfromacrosssectionalstudyofonlinemedicalconsultation |