Comparing orthodontic pre-treatment information provided by large language models
Abstract This study collected and screened the 50 most common pre-treatment consultation questions from adult orthodontic patients through clinical practice. Responses to these questions were generated using three large language models: Ernie Bot, ChatGPT, and Gemini. The responses were evaluated ac...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-05-01
|
| Series: | BMC Oral Health |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12903-025-06246-1 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850231135090507776 |
|---|---|
| author | Jingcheng Chen Xiangyu Ge Chenyang Yuan Yanan Chen Xiangyu Li Xi Zhang Shixiang Chen WeiYing Zheng Chunqin Miao |
| author_facet | Jingcheng Chen Xiangyu Ge Chenyang Yuan Yanan Chen Xiangyu Li Xi Zhang Shixiang Chen WeiYing Zheng Chunqin Miao |
| author_sort | Jingcheng Chen |
| collection | DOAJ |
| description | Abstract This study collected and screened the 50 most common pre-treatment consultation questions from adult orthodontic patients through clinical practice. Responses to these questions were generated using three large language models: Ernie Bot, ChatGPT, and Gemini. The responses were evaluated across six dimensions: Professional Accuracy (PA), Accuracy of Content(AC), Clarity and Comprehensibility (CC), Personalization and Relevance (PR), Information Completeness (IC), and Empathy and Patient-Centeredness (EHC). Results indicated that scores for each group in various dimensions primarily fell within the range of 3–4 points, with relatively few high-quality scores (5 points). While large language models demonstrate some capability in addressing open-ended questions, their use in medical consultation, particularly in orthodontic medicine, requires caution and further integration with professional guidance and verification. Future research and technological improvements should focus on enhancing AI(Artificial Intelligence) performance in accuracy, information completeness, and humanistic care to better meet the needs of diverse clinical scenarios. |
| format | Article |
| id | doaj-art-12ecdda893b547c5886e2f8df0239a4a |
| institution | OA Journals |
| issn | 1472-6831 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Oral Health |
| spelling | doaj-art-12ecdda893b547c5886e2f8df0239a4a2025-08-20T02:03:39ZengBMCBMC Oral Health1472-68312025-05-012511810.1186/s12903-025-06246-1Comparing orthodontic pre-treatment information provided by large language modelsJingcheng Chen0Xiangyu Ge1Chenyang Yuan2Yanan Chen3Xiangyu Li4Xi Zhang5Shixiang Chen6WeiYing Zheng7Chunqin Miao8Jiaxing Nanhu District People’s HospitalThe Second Hospital of JiaxingThe Second Hospital of JiaxingThe Second Hospital of JiaxingThe Second Hospital of JiaxingThe Second Hospital of JiaxingTongxiang Hospital of Traditional Chinese MedicineThe Second Hospital of JiaxingThe Second Hospital of JiaxingAbstract This study collected and screened the 50 most common pre-treatment consultation questions from adult orthodontic patients through clinical practice. Responses to these questions were generated using three large language models: Ernie Bot, ChatGPT, and Gemini. The responses were evaluated across six dimensions: Professional Accuracy (PA), Accuracy of Content(AC), Clarity and Comprehensibility (CC), Personalization and Relevance (PR), Information Completeness (IC), and Empathy and Patient-Centeredness (EHC). Results indicated that scores for each group in various dimensions primarily fell within the range of 3–4 points, with relatively few high-quality scores (5 points). While large language models demonstrate some capability in addressing open-ended questions, their use in medical consultation, particularly in orthodontic medicine, requires caution and further integration with professional guidance and verification. Future research and technological improvements should focus on enhancing AI(Artificial Intelligence) performance in accuracy, information completeness, and humanistic care to better meet the needs of diverse clinical scenarios.https://doi.org/10.1186/s12903-025-06246-1Large language modelsPre-treatment informationOrthodonticErnie botChatgptGemini |
| spellingShingle | Jingcheng Chen Xiangyu Ge Chenyang Yuan Yanan Chen Xiangyu Li Xi Zhang Shixiang Chen WeiYing Zheng Chunqin Miao Comparing orthodontic pre-treatment information provided by large language models BMC Oral Health Large language models Pre-treatment information Orthodontic Ernie bot Chatgpt Gemini |
| title | Comparing orthodontic pre-treatment information provided by large language models |
| title_full | Comparing orthodontic pre-treatment information provided by large language models |
| title_fullStr | Comparing orthodontic pre-treatment information provided by large language models |
| title_full_unstemmed | Comparing orthodontic pre-treatment information provided by large language models |
| title_short | Comparing orthodontic pre-treatment information provided by large language models |
| title_sort | comparing orthodontic pre treatment information provided by large language models |
| topic | Large language models Pre-treatment information Orthodontic Ernie bot Chatgpt Gemini |
| url | https://doi.org/10.1186/s12903-025-06246-1 |
| work_keys_str_mv | AT jingchengchen comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT xiangyuge comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT chenyangyuan comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT yananchen comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT xiangyuli comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT xizhang comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT shixiangchen comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT weiyingzheng comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels AT chunqinmiao comparingorthodonticpretreatmentinformationprovidedbylargelanguagemodels |