Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists
Background and objective: Data on interaction of patients with artificial intelligence (AI) are limited, primarily derived from small-scale studies, cross-sectional surveys, and qualitative reviews. Most patients have not yet encountered AI in their clinical experience. This study explored patients’...
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | European Urology Open Science |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666168324010942 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850056214468100096 |
|---|---|
| author | Nicolas Carl Lisa Nguyen Sarah Haggenmüller Martin Joachim Hetz Jana Theres Winterstein Friedrich Otto Hartung Britta Gruene Jakob Nikolas Kather Tim Holland-Letz Maurice Stephan Michel Frederik Wessels Titus Josef Brinker |
| author_facet | Nicolas Carl Lisa Nguyen Sarah Haggenmüller Martin Joachim Hetz Jana Theres Winterstein Friedrich Otto Hartung Britta Gruene Jakob Nikolas Kather Tim Holland-Letz Maurice Stephan Michel Frederik Wessels Titus Josef Brinker |
| author_sort | Nicolas Carl |
| collection | DOAJ |
| description | Background and objective: Data on interaction of patients with artificial intelligence (AI) are limited, primarily derived from small-scale studies, cross-sectional surveys, and qualitative reviews. Most patients have not yet encountered AI in their clinical experience. This study explored patients’ confidence in AI, specifically large language models, after a direct interaction with a chatbot in a clinical setting. Through hands-on experience, the study sought to reduce potential biases due to an anticipated lack of AI experience in a real-world urological patient sample. Methods: A total of 300 patients scheduled for counseling were enrolled from February to July 2024. Participants voluntarily conversed about their medical questions with a GPT-4 powered chatbot, followed by a survey assessing their confidence in clinical capabilities of AI compared with their counseling urologists. Clinical capabilities included history taking, diagnostics, treatment recommendation, anxiety reduction, and time allocation. Key findings and limitations: Of the 292 patients who completed the study, AI was significantly preferred to physicians for consultation time allocation (p < 0.001). However, urologists were overwhelmingly favored for all other capabilities, especially treatment recommendations and anxiety reduction. Notably, age did not influence patients’ confidence in AI. Limitations include a potential social desirability bias. Conclusions and clinical implications: Our study demonstrates that urological patients prefer AI as a powerful complement to—rather than a replacement for—human expertise in clinical care. Patients appreciated the additional consultation time provided by AI. Interestingly, age was not associated with confidence in AI, suggesting that large language models are user-friendly tools for patients of all age groups. Patient summary: In this report, we explored how patients feel about using an artificial intelligence (AI)-powered chatbot in a medical setting. Patients interacted with the AI for medical questions and compared its skills with those of doctors through a survey. They appreciated the AI for providing more time during consultations but preferred doctors for other tasks, for example, diagnostics, recommendation of treatments, and reduction of anxieties. |
| format | Article |
| id | doaj-art-96461ba44349479bb2fe3a10c3b55b80 |
| institution | DOAJ |
| issn | 2666-1683 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | European Urology Open Science |
| spelling | doaj-art-96461ba44349479bb2fe3a10c3b55b802025-08-20T02:51:45ZengElsevierEuropean Urology Open Science2666-16832024-12-0170919810.1016/j.euros.2024.10.009Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus UrologistsNicolas Carl0Lisa Nguyen1Sarah Haggenmüller2Martin Joachim Hetz3Jana Theres Winterstein4Friedrich Otto Hartung5Britta Gruene6Jakob Nikolas Kather7Tim Holland-Letz8Maurice Stephan Michel9Frederik Wessels10Titus Josef Brinker11Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany; Department of Urology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, GermanyMedical Faculty Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, GermanyDigital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, GermanyDigital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, GermanyDigital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany; Medical Faculty Heidelberg, Ruprecht-Karls University of Heidelberg, Heidelberg, GermanyDepartment of Urology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, GermanyDepartment of Urology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, GermanyMedical Faculty Carl Gustav Carus, Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, GermanyDepartment of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, GermanyDepartment of Urology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, GermanyDepartment of Urology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, GermanyDigital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany; Corresponding author. Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany. Tel. +49 6221 425301.Background and objective: Data on interaction of patients with artificial intelligence (AI) are limited, primarily derived from small-scale studies, cross-sectional surveys, and qualitative reviews. Most patients have not yet encountered AI in their clinical experience. This study explored patients’ confidence in AI, specifically large language models, after a direct interaction with a chatbot in a clinical setting. Through hands-on experience, the study sought to reduce potential biases due to an anticipated lack of AI experience in a real-world urological patient sample. Methods: A total of 300 patients scheduled for counseling were enrolled from February to July 2024. Participants voluntarily conversed about their medical questions with a GPT-4 powered chatbot, followed by a survey assessing their confidence in clinical capabilities of AI compared with their counseling urologists. Clinical capabilities included history taking, diagnostics, treatment recommendation, anxiety reduction, and time allocation. Key findings and limitations: Of the 292 patients who completed the study, AI was significantly preferred to physicians for consultation time allocation (p < 0.001). However, urologists were overwhelmingly favored for all other capabilities, especially treatment recommendations and anxiety reduction. Notably, age did not influence patients’ confidence in AI. Limitations include a potential social desirability bias. Conclusions and clinical implications: Our study demonstrates that urological patients prefer AI as a powerful complement to—rather than a replacement for—human expertise in clinical care. Patients appreciated the additional consultation time provided by AI. Interestingly, age was not associated with confidence in AI, suggesting that large language models are user-friendly tools for patients of all age groups. Patient summary: In this report, we explored how patients feel about using an artificial intelligence (AI)-powered chatbot in a medical setting. Patients interacted with the AI for medical questions and compared its skills with those of doctors through a survey. They appreciated the AI for providing more time during consultations but preferred doctors for other tasks, for example, diagnostics, recommendation of treatments, and reduction of anxieties.http://www.sciencedirect.com/science/article/pii/S2666168324010942Clinical trialGenerative artificial intelligenceImplementation scienceLarge language modelsPatient interaction |
| spellingShingle | Nicolas Carl Lisa Nguyen Sarah Haggenmüller Martin Joachim Hetz Jana Theres Winterstein Friedrich Otto Hartung Britta Gruene Jakob Nikolas Kather Tim Holland-Letz Maurice Stephan Michel Frederik Wessels Titus Josef Brinker Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists European Urology Open Science Clinical trial Generative artificial intelligence Implementation science Large language models Patient interaction |
| title | Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists |
| title_full | Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists |
| title_fullStr | Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists |
| title_full_unstemmed | Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists |
| title_short | Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists |
| title_sort | comparing patient s confidence in clinical capabilities in urology large language models versus urologists |
| topic | Clinical trial Generative artificial intelligence Implementation science Large language models Patient interaction |
| url | http://www.sciencedirect.com/science/article/pii/S2666168324010942 |
| work_keys_str_mv | AT nicolascarl comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT lisanguyen comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT sarahhaggenmuller comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT martinjoachimhetz comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT janathereswinterstein comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT friedrichottohartung comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT brittagruene comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT jakobnikolaskather comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT timhollandletz comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT mauricestephanmichel comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT frederikwessels comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists AT titusjosefbrinker comparingpatientsconfidenceinclinicalcapabilitiesinurologylargelanguagemodelsversusurologists |