Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China
Objectives To evaluate the potential of large language models (LLMs) in health education for patients with ankylosing spondylitis (AS)/spondyloarthritis (SpA), focusing on the accuracy of information transmission, patient acceptance and performance differences between different models.Design Cross-s...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2025-03-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/15/3/e097528.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850276923345731584 |
|---|---|
| author | Yong Ren Yuanqing Li Jieruo Gu Qing Lv Wenqi Xia Jingyu Zhang Huifen Liu Ya Wen Liling Xu Yuling Chen Yue-ning Kang Shuang-yan Cao Fanxuan Meng Ruyi Liao Xiaomin Li Jiayun Wu Shenghui Wen |
| author_facet | Yong Ren Yuanqing Li Jieruo Gu Qing Lv Wenqi Xia Jingyu Zhang Huifen Liu Ya Wen Liling Xu Yuling Chen Yue-ning Kang Shuang-yan Cao Fanxuan Meng Ruyi Liao Xiaomin Li Jiayun Wu Shenghui Wen |
| author_sort | Yong Ren |
| collection | DOAJ |
| description | Objectives To evaluate the potential of large language models (LLMs) in health education for patients with ankylosing spondylitis (AS)/spondyloarthritis (SpA), focusing on the accuracy of information transmission, patient acceptance and performance differences between different models.Design Cross-sectional, single-blind study.Setting Multiple centres in China.Participants 182 volunteers, including 4 rheumatologists and 178 patients with AS/SpA.Primary and secondary outcome measures Scientificity, precision and accessibility of the content of the answers provided by LLMs; patient acceptance of the answers.Results LLMs performed well in terms of scientificity, precision and accessibility, with ChatGPT-4o and Kimi models outperforming traditional guidelines. Most patients with AS/SpA showed a higher level of understanding and acceptance of the responses from LLMs.Conclusions LLMs have significant potential in medical knowledge transmission and patient education, making them promising tools for future medical practice. |
| format | Article |
| id | doaj-art-c3f8a15833b948a093bf8cf7eef528e4 |
| institution | OA Journals |
| issn | 2044-6055 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-c3f8a15833b948a093bf8cf7eef528e42025-08-20T01:50:02ZengBMJ Publishing GroupBMJ Open2044-60552025-03-0115310.1136/bmjopen-2024-097528Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in ChinaYong Ren0Yuanqing Li1Jieruo Gu2Qing Lv3Wenqi Xia4Jingyu Zhang5Huifen Liu6Ya Wen7Liling Xu8Yuling Chen9Yue-ning Kang10Shuang-yan Cao11Fanxuan Meng12Ruyi Liao13Xiaomin Li14Jiayun Wu15Shenghui Wen162 The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China4 School of Automation Science and Engineering, South China University of Technology, Guangzhou, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, China3 Department of Rheumatology, The Seventh Affiliated Hospital of Sun Yat-sen University, Shenzhen, Guangdong, ChinaObjectives To evaluate the potential of large language models (LLMs) in health education for patients with ankylosing spondylitis (AS)/spondyloarthritis (SpA), focusing on the accuracy of information transmission, patient acceptance and performance differences between different models.Design Cross-sectional, single-blind study.Setting Multiple centres in China.Participants 182 volunteers, including 4 rheumatologists and 178 patients with AS/SpA.Primary and secondary outcome measures Scientificity, precision and accessibility of the content of the answers provided by LLMs; patient acceptance of the answers.Results LLMs performed well in terms of scientificity, precision and accessibility, with ChatGPT-4o and Kimi models outperforming traditional guidelines. Most patients with AS/SpA showed a higher level of understanding and acceptance of the responses from LLMs.Conclusions LLMs have significant potential in medical knowledge transmission and patient education, making them promising tools for future medical practice.https://bmjopen.bmj.com/content/15/3/e097528.full |
| spellingShingle | Yong Ren Yuanqing Li Jieruo Gu Qing Lv Wenqi Xia Jingyu Zhang Huifen Liu Ya Wen Liling Xu Yuling Chen Yue-ning Kang Shuang-yan Cao Fanxuan Meng Ruyi Liao Xiaomin Li Jiayun Wu Shenghui Wen Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China BMJ Open |
| title | Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China |
| title_full | Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China |
| title_fullStr | Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China |
| title_full_unstemmed | Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China |
| title_short | Evaluating the performance of large language models in health education for patients with ankylosing spondylitis/spondyloarthritis: a cross-sectional, single-blind study in China |
| title_sort | evaluating the performance of large language models in health education for patients with ankylosing spondylitis spondyloarthritis a cross sectional single blind study in china |
| url | https://bmjopen.bmj.com/content/15/3/e097528.full |
| work_keys_str_mv | AT yongren evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT yuanqingli evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT jieruogu evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT qinglv evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT wenqixia evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT jingyuzhang evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT huifenliu evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT yawen evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT lilingxu evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT yulingchen evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT yueningkang evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT shuangyancao evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT fanxuanmeng evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT ruyiliao evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT xiaominli evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT jiayunwu evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina AT shenghuiwen evaluatingtheperformanceoflargelanguagemodelsinhealtheducationforpatientswithankylosingspondylitisspondyloarthritisacrosssectionalsingleblindstudyinchina |