Reference decisions enhance LLM performance, amplified by source disclosure
Objective The rapid integration of large language models (LLMs) has propelled advancements in automated dialog technologies, improving the public's access to healthcare services. Drawing inspiration from the collaborative decision-making practices of medical professionals in complex cases, we i...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2025-05-01
|
| Series: | Digital Health |
| Online Access: | https://doi.org/10.1177/20552076251342078 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850269448547598336 |
|---|---|
| author | Yongxiang Zhang Zhaobin Liu Shaosen Bai Ting Xu Raymond YK Lau |
| author_facet | Yongxiang Zhang Zhaobin Liu Shaosen Bai Ting Xu Raymond YK Lau |
| author_sort | Yongxiang Zhang |
| collection | DOAJ |
| description | Objective The rapid integration of large language models (LLMs) has propelled advancements in automated dialog technologies, improving the public's access to healthcare services. Drawing inspiration from the collaborative decision-making practices of medical professionals in complex cases, we investigated whether LLMs could enhance their diagnostic accuracy through interaction. Methods An experimental study was conducted in China (September–December 2024) to investigate the impact of LLM-generated reference decisions and source disclosure on LLMs’ diagnostic performance. We used a Chinese clinical diagnostic task in a controlled comparative design, where three Chinese LLMs interpreted symptoms and conditions based on patient queries. LLMs’ outcomes were evaluated through accuracy and weighted F1 score metrics, with statistical analysis to determine significance. Results Analysis of variance on LLMs’ diagnostic accuracy scores demonstrated that incorporating LLM-generated decisions as a reference significantly improved diagnostic outcomes, with source disclosure amplifying this improvement. Conclusion Our findings underscore the potential of LLM collaboration in healthcare, offering strategies to refine response generation and decision-making across various applications. |
| format | Article |
| id | doaj-art-4eb4eb33a2cb4fb09d708c91413d5bb3 |
| institution | OA Journals |
| issn | 2055-2076 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Digital Health |
| spelling | doaj-art-4eb4eb33a2cb4fb09d708c91413d5bb32025-08-20T01:53:08ZengSAGE PublishingDigital Health2055-20762025-05-011110.1177/20552076251342078Reference decisions enhance LLM performance, amplified by source disclosureYongxiang Zhang0Zhaobin Liu1Shaosen Bai2Ting Xu3Raymond YK Lau4 Department of Information Systems, , Hong Kong SAR, China Department of Information Systems, , Hong Kong SAR, China Department of Decisions, Operations and Technology, , Hong Kong SAR, China Department of Information Systems and Management Engineering, , Shenzhen, China Department of Information Systems, , Hong Kong SAR, ChinaObjective The rapid integration of large language models (LLMs) has propelled advancements in automated dialog technologies, improving the public's access to healthcare services. Drawing inspiration from the collaborative decision-making practices of medical professionals in complex cases, we investigated whether LLMs could enhance their diagnostic accuracy through interaction. Methods An experimental study was conducted in China (September–December 2024) to investigate the impact of LLM-generated reference decisions and source disclosure on LLMs’ diagnostic performance. We used a Chinese clinical diagnostic task in a controlled comparative design, where three Chinese LLMs interpreted symptoms and conditions based on patient queries. LLMs’ outcomes were evaluated through accuracy and weighted F1 score metrics, with statistical analysis to determine significance. Results Analysis of variance on LLMs’ diagnostic accuracy scores demonstrated that incorporating LLM-generated decisions as a reference significantly improved diagnostic outcomes, with source disclosure amplifying this improvement. Conclusion Our findings underscore the potential of LLM collaboration in healthcare, offering strategies to refine response generation and decision-making across various applications.https://doi.org/10.1177/20552076251342078 |
| spellingShingle | Yongxiang Zhang Zhaobin Liu Shaosen Bai Ting Xu Raymond YK Lau Reference decisions enhance LLM performance, amplified by source disclosure Digital Health |
| title | Reference decisions enhance LLM performance, amplified by source disclosure |
| title_full | Reference decisions enhance LLM performance, amplified by source disclosure |
| title_fullStr | Reference decisions enhance LLM performance, amplified by source disclosure |
| title_full_unstemmed | Reference decisions enhance LLM performance, amplified by source disclosure |
| title_short | Reference decisions enhance LLM performance, amplified by source disclosure |
| title_sort | reference decisions enhance llm performance amplified by source disclosure |
| url | https://doi.org/10.1177/20552076251342078 |
| work_keys_str_mv | AT yongxiangzhang referencedecisionsenhancellmperformanceamplifiedbysourcedisclosure AT zhaobinliu referencedecisionsenhancellmperformanceamplifiedbysourcedisclosure AT shaosenbai referencedecisionsenhancellmperformanceamplifiedbysourcedisclosure AT tingxu referencedecisionsenhancellmperformanceamplifiedbysourcedisclosure AT raymondyklau referencedecisionsenhancellmperformanceamplifiedbysourcedisclosure |