Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back pain
Background The self-management of low back pain (LBP) through patient information interventions offers significant benefits in terms of cost, reduced work absenteeism, and overall healthcare utilization. Using a large language model (LLM), such as ChatGPT (OpenAI) or Copilot (Microsoft), could poten...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Yeungnam University College of Medicine, Yeungnam University Institute Medical Science
2025-01-01
|
| Series: | Journal of Yeungnam Medical Science |
| Subjects: | |
| Online Access: | http://www.e-jyms.org/upload/pdf/jyms-2024-01151.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823859278468874240 |
|---|---|
| author | Christophe Ah-Yan Ève Boissonnault Mathieu Boudier-Revéret Christopher Mares |
| author_facet | Christophe Ah-Yan Ève Boissonnault Mathieu Boudier-Revéret Christopher Mares |
| author_sort | Christophe Ah-Yan |
| collection | DOAJ |
| description | Background The self-management of low back pain (LBP) through patient information interventions offers significant benefits in terms of cost, reduced work absenteeism, and overall healthcare utilization. Using a large language model (LLM), such as ChatGPT (OpenAI) or Copilot (Microsoft), could potentially enhance these outcomes further. Thus, it is important to evaluate the LLMs ChatGPT and Copilot in providing medical advice for LBP and assessing the impact of clinical context on the quality of responses. Methods This was a qualitative comparative observational study. It was conducted within the Department of Physical Medicine and Rehabilitation, University of Montreal in Montreal, QC, Canada. ChatGPT and Copilot were used to answer 27 common questions related to LBP, with and without a specific clinical context. The responses were evaluated by physiatrists for validity, safety, and usefulness using a 4-point Likert scale (4, most favorable). Results Both ChatGPT and Copilot demonstrated good performance across all measures. Validity scores were 3.33 for ChatGPT and 3.18 for Copilot, safety scores were 3.19 for ChatGPT and 3.13 for Copilot, and usefulness scores were 3.60 for ChatGPT and 3.57 for Copilot. The inclusion of clinical context did not significantly change the results. Conclusion LLMs, such as ChatGPT and Copilot, can provide reliable medical advice on LBP, irrespective of the detailed clinical context, supporting their potential to aid in patient self-management. |
| format | Article |
| id | doaj-art-c1a4c00271b14095a3011b133cfc4a87 |
| institution | Kabale University |
| issn | 2799-8010 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Yeungnam University College of Medicine, Yeungnam University Institute Medical Science |
| record_format | Article |
| series | Journal of Yeungnam Medical Science |
| spelling | doaj-art-c1a4c00271b14095a3011b133cfc4a872025-02-11T06:26:37ZengYeungnam University College of Medicine, Yeungnam University Institute Medical ScienceJournal of Yeungnam Medical Science2799-80102025-01-014210.12701/jyms.2024.011512872Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back painChristophe Ah-Yan0Ève Boissonnault1Mathieu Boudier-Revéret2Christopher Mares3 Department of Physical Medicine and Rehabilitation, University of Montreal, Montreal, QC, Canada Department of Physical Medicine and Rehabilitation, Centre Hospitalier de l’Université de Montréal, Montreal, QC, Canada Department of Physical Medicine and Rehabilitation, Centre Hospitalier de l’Université de Montréal, Montreal, QC, Canada Department of Physical Medicine and Rehabilitation, Centre Hospitalier de l’Université de Montréal, Montreal, QC, CanadaBackground The self-management of low back pain (LBP) through patient information interventions offers significant benefits in terms of cost, reduced work absenteeism, and overall healthcare utilization. Using a large language model (LLM), such as ChatGPT (OpenAI) or Copilot (Microsoft), could potentially enhance these outcomes further. Thus, it is important to evaluate the LLMs ChatGPT and Copilot in providing medical advice for LBP and assessing the impact of clinical context on the quality of responses. Methods This was a qualitative comparative observational study. It was conducted within the Department of Physical Medicine and Rehabilitation, University of Montreal in Montreal, QC, Canada. ChatGPT and Copilot were used to answer 27 common questions related to LBP, with and without a specific clinical context. The responses were evaluated by physiatrists for validity, safety, and usefulness using a 4-point Likert scale (4, most favorable). Results Both ChatGPT and Copilot demonstrated good performance across all measures. Validity scores were 3.33 for ChatGPT and 3.18 for Copilot, safety scores were 3.19 for ChatGPT and 3.13 for Copilot, and usefulness scores were 3.60 for ChatGPT and 3.57 for Copilot. The inclusion of clinical context did not significantly change the results. Conclusion LLMs, such as ChatGPT and Copilot, can provide reliable medical advice on LBP, irrespective of the detailed clinical context, supporting their potential to aid in patient self-management.http://www.e-jyms.org/upload/pdf/jyms-2024-01151.pdfartificial intelligencelarge language modelslow back painpatient carephysical and rehabilitation medicine |
| spellingShingle | Christophe Ah-Yan Ève Boissonnault Mathieu Boudier-Revéret Christopher Mares Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back pain Journal of Yeungnam Medical Science artificial intelligence large language models low back pain patient care physical and rehabilitation medicine |
| title | Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back pain |
| title_full | Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back pain |
| title_fullStr | Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back pain |
| title_full_unstemmed | Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back pain |
| title_short | Impact of artificial intelligence in managing musculoskeletal pathologies in physiatry: a qualitative observational study evaluating the potential use of ChatGPT versus Copilot for patient information and clinical advice on low back pain |
| title_sort | impact of artificial intelligence in managing musculoskeletal pathologies in physiatry a qualitative observational study evaluating the potential use of chatgpt versus copilot for patient information and clinical advice on low back pain |
| topic | artificial intelligence large language models low back pain patient care physical and rehabilitation medicine |
| url | http://www.e-jyms.org/upload/pdf/jyms-2024-01151.pdf |
| work_keys_str_mv | AT christopheahyan impactofartificialintelligenceinmanagingmusculoskeletalpathologiesinphysiatryaqualitativeobservationalstudyevaluatingthepotentialuseofchatgptversuscopilotforpatientinformationandclinicaladviceonlowbackpain AT eveboissonnault impactofartificialintelligenceinmanagingmusculoskeletalpathologiesinphysiatryaqualitativeobservationalstudyevaluatingthepotentialuseofchatgptversuscopilotforpatientinformationandclinicaladviceonlowbackpain AT mathieuboudierreveret impactofartificialintelligenceinmanagingmusculoskeletalpathologiesinphysiatryaqualitativeobservationalstudyevaluatingthepotentialuseofchatgptversuscopilotforpatientinformationandclinicaladviceonlowbackpain AT christophermares impactofartificialintelligenceinmanagingmusculoskeletalpathologiesinphysiatryaqualitativeobservationalstudyevaluatingthepotentialuseofchatgptversuscopilotforpatientinformationandclinicaladviceonlowbackpain |