Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model Assessment
<b>Background:</b> While long-term opioid therapy is a widely utilized strategy for managing chronic pain, many patients have understandable questions and concerns regarding its safety, efficacy, and potential for dependency and addiction. Providing clear, accurate, and reliable informat...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Biomedicines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-9059/13/3/636 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850090306819588096 |
|---|---|
| author | Giuliano Lo Bianco Christopher L. Robinson Francesco Paolo D’Angelo Marco Cascella Silvia Natoli Emanuele Sinagra Sebastiano Mercadante Filippo Drago |
| author_facet | Giuliano Lo Bianco Christopher L. Robinson Francesco Paolo D’Angelo Marco Cascella Silvia Natoli Emanuele Sinagra Sebastiano Mercadante Filippo Drago |
| author_sort | Giuliano Lo Bianco |
| collection | DOAJ |
| description | <b>Background:</b> While long-term opioid therapy is a widely utilized strategy for managing chronic pain, many patients have understandable questions and concerns regarding its safety, efficacy, and potential for dependency and addiction. Providing clear, accurate, and reliable information is essential for fostering patient understanding and acceptance. Generative artificial intelligence (AI) applications offer interesting avenues for delivering patient education in healthcare. This study evaluates the reliability, accuracy, and comprehensibility of ChatGPT’s responses to common patient inquiries about opioid long-term therapy. <b>Methods:</b> An expert panel selected thirteen frequently asked questions regarding long-term opioid therapy based on the authors’ clinical experience in managing chronic pain patients and a targeted review of patient education materials. Questions were prioritized based on prevalence in patient consultations, relevance to treatment decision-making, and the complexity of information typically required to address them comprehensively. We assessed comprehensibility by implementing the multimodal generative AI Copilot (Microsoft 365 Copilot Chat). Spanning three domains—pre-therapy, during therapy, and post-therapy—each question was submitted to GPT-4.0 with the prompt “<i>If you were a physician, how would you answer a patient asking…</i>”. Ten pain physicians and two non-healthcare professionals independently assessed the responses using a Likert scale to rate reliability (1–6 points), accuracy (1–3 points), and comprehensibility (1–3 points). <b>Results:</b> Overall, ChatGPT’s responses demonstrated high reliability (5.2 ± 0.6) and good comprehensibility (2.8 ± 0.2), with most answers meeting or exceeding predefined thresholds. Accuracy was moderate (2.7 ± 0.3), with lower performance on more technical topics like opioid tolerance and dependency management. <b>Conclusions:</b> While AI applications exhibit significant potential as a supplementary tool for patient education on opioid long-term therapy, limitations in addressing highly technical or context-specific queries underscore the need for ongoing refinement and domain-specific training. Integrating AI systems into clinical practice should involve collaboration between healthcare professionals and AI developers to ensure safe, personalized, and up-to-date patient education in chronic pain management. |
| format | Article |
| id | doaj-art-716a433662f64d74ad4e022a1c5e5e22 |
| institution | DOAJ |
| issn | 2227-9059 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomedicines |
| spelling | doaj-art-716a433662f64d74ad4e022a1c5e5e222025-08-20T02:42:35ZengMDPI AGBiomedicines2227-90592025-03-0113363610.3390/biomedicines13030636Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model AssessmentGiuliano Lo Bianco0Christopher L. Robinson1Francesco Paolo D’Angelo2Marco Cascella3Silvia Natoli4Emanuele Sinagra5Sebastiano Mercadante6Filippo Drago7Anesthesiology and Pain Department, Foundation G. Giglio Cefalù, 90015 Palermo, ItalyAnesthesiology, Perioperative, and Pain Medicine, Brigham and Women’s Hospital, Harvard Medical School, Harvard University, Boston, MA 02115, USADepartment of Anaesthesia, Intensive Care and Emergency, University Hospital Policlinico Paolo Giaccone, 90127 Palermo, ItalyAnesthesia and Pain Medicine, Department of Medicine, Surgery and Dentistry “Scuola Medica Salernitana”, University of Salerno, 84081 Baronissi, ItalyDepartment of Clinical-Surgical, Diagnostic and Pediatric Sciences, University of Pavia, 27100 Pavia, ItalyGastroenterology and Endoscopy Unit, Fondazione Istituto San Raffaele Giglio, 90015 Cefalù, ItalyMain Regional Center for Pain Relief and Supportive/Palliative Care, La Maddalena Cancer Center, Via San Lorenzo 312, 90146 Palermo, ItalyDepartment of Biomedical and Biotechnological Sciences, University of Catania, 95124 Catania, Italy<b>Background:</b> While long-term opioid therapy is a widely utilized strategy for managing chronic pain, many patients have understandable questions and concerns regarding its safety, efficacy, and potential for dependency and addiction. Providing clear, accurate, and reliable information is essential for fostering patient understanding and acceptance. Generative artificial intelligence (AI) applications offer interesting avenues for delivering patient education in healthcare. This study evaluates the reliability, accuracy, and comprehensibility of ChatGPT’s responses to common patient inquiries about opioid long-term therapy. <b>Methods:</b> An expert panel selected thirteen frequently asked questions regarding long-term opioid therapy based on the authors’ clinical experience in managing chronic pain patients and a targeted review of patient education materials. Questions were prioritized based on prevalence in patient consultations, relevance to treatment decision-making, and the complexity of information typically required to address them comprehensively. We assessed comprehensibility by implementing the multimodal generative AI Copilot (Microsoft 365 Copilot Chat). Spanning three domains—pre-therapy, during therapy, and post-therapy—each question was submitted to GPT-4.0 with the prompt “<i>If you were a physician, how would you answer a patient asking…</i>”. Ten pain physicians and two non-healthcare professionals independently assessed the responses using a Likert scale to rate reliability (1–6 points), accuracy (1–3 points), and comprehensibility (1–3 points). <b>Results:</b> Overall, ChatGPT’s responses demonstrated high reliability (5.2 ± 0.6) and good comprehensibility (2.8 ± 0.2), with most answers meeting or exceeding predefined thresholds. Accuracy was moderate (2.7 ± 0.3), with lower performance on more technical topics like opioid tolerance and dependency management. <b>Conclusions:</b> While AI applications exhibit significant potential as a supplementary tool for patient education on opioid long-term therapy, limitations in addressing highly technical or context-specific queries underscore the need for ongoing refinement and domain-specific training. Integrating AI systems into clinical practice should involve collaboration between healthcare professionals and AI developers to ensure safe, personalized, and up-to-date patient education in chronic pain management.https://www.mdpi.com/2227-9059/13/3/636opioid long-term therapyChatGPTartificial intelligencepatient educationchronic pain managementhealthcare communication |
| spellingShingle | Giuliano Lo Bianco Christopher L. Robinson Francesco Paolo D’Angelo Marco Cascella Silvia Natoli Emanuele Sinagra Sebastiano Mercadante Filippo Drago Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model Assessment Biomedicines opioid long-term therapy ChatGPT artificial intelligence patient education chronic pain management healthcare communication |
| title | Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model Assessment |
| title_full | Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model Assessment |
| title_fullStr | Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model Assessment |
| title_full_unstemmed | Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model Assessment |
| title_short | Effectiveness of Generative Artificial Intelligence-Driven Responses to Patient Concerns in Long-Term Opioid Therapy: Cross-Model Assessment |
| title_sort | effectiveness of generative artificial intelligence driven responses to patient concerns in long term opioid therapy cross model assessment |
| topic | opioid long-term therapy ChatGPT artificial intelligence patient education chronic pain management healthcare communication |
| url | https://www.mdpi.com/2227-9059/13/3/636 |
| work_keys_str_mv | AT giulianolobianco effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment AT christopherlrobinson effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment AT francescopaolodangelo effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment AT marcocascella effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment AT silvianatoli effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment AT emanuelesinagra effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment AT sebastianomercadante effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment AT filippodrago effectivenessofgenerativeartificialintelligencedrivenresponsestopatientconcernsinlongtermopioidtherapycrossmodelassessment |