Reference decisions enhance LLM performance, amplified by source disclosure
Objective The rapid integration of large language models (LLMs) has propelled advancements in automated dialog technologies, improving the public's access to healthcare services. Drawing inspiration from the collaborative decision-making practices of medical professionals in complex cases, we i...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2025-05-01
|
| Series: | Digital Health |
| Online Access: | https://doi.org/10.1177/20552076251342078 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Objective The rapid integration of large language models (LLMs) has propelled advancements in automated dialog technologies, improving the public's access to healthcare services. Drawing inspiration from the collaborative decision-making practices of medical professionals in complex cases, we investigated whether LLMs could enhance their diagnostic accuracy through interaction. Methods An experimental study was conducted in China (September–December 2024) to investigate the impact of LLM-generated reference decisions and source disclosure on LLMs’ diagnostic performance. We used a Chinese clinical diagnostic task in a controlled comparative design, where three Chinese LLMs interpreted symptoms and conditions based on patient queries. LLMs’ outcomes were evaluated through accuracy and weighted F1 score metrics, with statistical analysis to determine significance. Results Analysis of variance on LLMs’ diagnostic accuracy scores demonstrated that incorporating LLM-generated decisions as a reference significantly improved diagnostic outcomes, with source disclosure amplifying this improvement. Conclusion Our findings underscore the potential of LLM collaboration in healthcare, offering strategies to refine response generation and decision-making across various applications. |
|---|---|
| ISSN: | 2055-2076 |