Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease
BackgroundDiabetic kidney disease (DKD) is a common and serious complication of diabetes mellitus and has become the most important cause of end-stage renal disease (ESRD). In light of the rising prevalence of diabetes, there is a growing imperative for the early detection and intervention of DKD. W...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-04-01
|
| Series: | Frontiers in Endocrinology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fendo.2025.1559265/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850179779073933312 |
|---|---|
| author | Miaomiao Cheng Qi Zhang Hua Liang Yanan Wang Jun Qin Lei Gong Sha Wang Luyao Li Xiaoyan Xiao |
| author_facet | Miaomiao Cheng Qi Zhang Hua Liang Yanan Wang Jun Qin Lei Gong Sha Wang Luyao Li Xiaoyan Xiao |
| author_sort | Miaomiao Cheng |
| collection | DOAJ |
| description | BackgroundDiabetic kidney disease (DKD) is a common and serious complication of diabetes mellitus and has become the most important cause of end-stage renal disease (ESRD). In light of the rising prevalence of diabetes, there is a growing imperative for the early detection and intervention of DKD. With the rapid development of artificial intelligence (AI) technologies, its potential applications in patient education are receiving increasing attention, especially large language models (LLMs). The aim of this study was to evaluate the quality of LLMs-generated patient education materials (PEMs) for early DKD and to explore its feasibility in patient education.MethodsFour LLMs (ERNIE Bot 4.0, GPT-4o, ChatGLM4, and ChatGPT-o1) were selected for this study to generate PEMs. Among them, ERNIE Bot 4.0, GPT-4o, and ChatGLM4 generated 2 versions of PEMs based on American Diabetes Association(ADA) guidelines and without ADA guidelines, respectively. ChatGPT-o1 only generated a PEM without ADA guidelines. An experienced physician wrote a PEM based on ADA guidelines. All materials were assessed using a Likert scale which covered the dimensions of accuracy, completeness, safety, and patient comprehensibility. A total of 7 medical experts (including nephrologists and endocrinologists) and 50 diabetic patients were invited to evaluate the study. We recorded basic information on the patient evaluators.ResultsExperts evaluated PEMs from ERNIE Bot 4.0, GPT-4o, ChatGLM4, and ChatGPT-o1, plus physician-sourced PEM. Results showed ERNIE Bot 4.0’s non-guideline PEM and physician-sourced PEM were the top two. Patient assessments of the 2 top-scoring PEMs found that the ERNIE Bot 4.0’s non-guideline PEM performed as well as, if not slightly better than, the physician-sourced PEM in terms of patient comprehensibility, completeness, and safety. In addition, the non-guideline-based PEM was preferred for patients with a history of diabetes longer than 5 years and for patients with proteinuria. Surprisingly, GPT-4o and ChatGLM4’s non-guideline PEMs outperformed guideline-based ones.ConclusionThe LLMs-sourced PEMs, especially the ERNIE Bot 4.0’s non-guideline PEM for early DKD, performed comparably to the physician-sourced PEM in terms of accuracy, completeness, safety, and patient comprehensibility, and exerted a high degree of feasibility. AI may show the potential for broader applications in patient education in the near future. |
| format | Article |
| id | doaj-art-9ea27d6f6b0849fcaa3d98943f7684f1 |
| institution | OA Journals |
| issn | 1664-2392 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Endocrinology |
| spelling | doaj-art-9ea27d6f6b0849fcaa3d98943f7684f12025-08-20T02:18:24ZengFrontiers Media S.A.Frontiers in Endocrinology1664-23922025-04-011610.3389/fendo.2025.15592651559265Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney diseaseMiaomiao Cheng0Qi Zhang1Hua Liang2Yanan Wang3Jun Qin4Lei Gong5Sha Wang6Luyao Li7Xiaoyan Xiao8Qilu Hospital of Shandong University, Department of Nephrology, Jinan, Shandong, ChinaHealthcare Big Data Research Institute, Cheeloo College of Medicine, Shandong University, Jinan, Shandong, ChinaQilu Hospital of Shandong University, Department of Nephrology, Jinan, Shandong, ChinaQilu Hospital of Shandong University, Department of Nephrology, Jinan, Shandong, ChinaQilu Hospital of Shandong University, Department of Endocrinology, Jinan, Shandong, ChinaQilu Hospital of Shandong University, Department of Endocrinology, Jinan, Shandong, ChinaQilu Hospital of Shandong University, Department of Nephrology, Jinan, Shandong, ChinaQilu Hospital of Shandong University, Department of Nephrology, Jinan, Shandong, ChinaQilu Hospital of Shandong University, Department of Nephrology, Jinan, Shandong, ChinaBackgroundDiabetic kidney disease (DKD) is a common and serious complication of diabetes mellitus and has become the most important cause of end-stage renal disease (ESRD). In light of the rising prevalence of diabetes, there is a growing imperative for the early detection and intervention of DKD. With the rapid development of artificial intelligence (AI) technologies, its potential applications in patient education are receiving increasing attention, especially large language models (LLMs). The aim of this study was to evaluate the quality of LLMs-generated patient education materials (PEMs) for early DKD and to explore its feasibility in patient education.MethodsFour LLMs (ERNIE Bot 4.0, GPT-4o, ChatGLM4, and ChatGPT-o1) were selected for this study to generate PEMs. Among them, ERNIE Bot 4.0, GPT-4o, and ChatGLM4 generated 2 versions of PEMs based on American Diabetes Association(ADA) guidelines and without ADA guidelines, respectively. ChatGPT-o1 only generated a PEM without ADA guidelines. An experienced physician wrote a PEM based on ADA guidelines. All materials were assessed using a Likert scale which covered the dimensions of accuracy, completeness, safety, and patient comprehensibility. A total of 7 medical experts (including nephrologists and endocrinologists) and 50 diabetic patients were invited to evaluate the study. We recorded basic information on the patient evaluators.ResultsExperts evaluated PEMs from ERNIE Bot 4.0, GPT-4o, ChatGLM4, and ChatGPT-o1, plus physician-sourced PEM. Results showed ERNIE Bot 4.0’s non-guideline PEM and physician-sourced PEM were the top two. Patient assessments of the 2 top-scoring PEMs found that the ERNIE Bot 4.0’s non-guideline PEM performed as well as, if not slightly better than, the physician-sourced PEM in terms of patient comprehensibility, completeness, and safety. In addition, the non-guideline-based PEM was preferred for patients with a history of diabetes longer than 5 years and for patients with proteinuria. Surprisingly, GPT-4o and ChatGLM4’s non-guideline PEMs outperformed guideline-based ones.ConclusionThe LLMs-sourced PEMs, especially the ERNIE Bot 4.0’s non-guideline PEM for early DKD, performed comparably to the physician-sourced PEM in terms of accuracy, completeness, safety, and patient comprehensibility, and exerted a high degree of feasibility. AI may show the potential for broader applications in patient education in the near future.https://www.frontiersin.org/articles/10.3389/fendo.2025.1559265/fulldiabetesdiabetic kidney diseaseartificial intelligencelarge language modelspatient education |
| spellingShingle | Miaomiao Cheng Qi Zhang Hua Liang Yanan Wang Jun Qin Lei Gong Sha Wang Luyao Li Xiaoyan Xiao Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease Frontiers in Endocrinology diabetes diabetic kidney disease artificial intelligence large language models patient education |
| title | Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease |
| title_full | Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease |
| title_fullStr | Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease |
| title_full_unstemmed | Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease |
| title_short | Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease |
| title_sort | comparison of artificial intelligence generated and physician generated patient education materials on early diabetic kidney disease |
| topic | diabetes diabetic kidney disease artificial intelligence large language models patient education |
| url | https://www.frontiersin.org/articles/10.3389/fendo.2025.1559265/full |
| work_keys_str_mv | AT miaomiaocheng comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT qizhang comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT hualiang comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT yananwang comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT junqin comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT leigong comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT shawang comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT luyaoli comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease AT xiaoyanxiao comparisonofartificialintelligencegeneratedandphysiciangeneratedpatienteducationmaterialsonearlydiabetickidneydisease |