Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study
BackgroundSepsis is a complex, life-threatening condition characterized by significant heterogeneity and vast amounts of unstructured data, posing substantial challenges for traditional knowledge graph construction methods. The integration of large language models (LLMs) with...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-03-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e65537 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850207483399766016 |
|---|---|
| author | Hao Yang Jiaxi Li Chi Zhang Alejandro Pazos Sierra Bairong Shen |
| author_facet | Hao Yang Jiaxi Li Chi Zhang Alejandro Pazos Sierra Bairong Shen |
| author_sort | Hao Yang |
| collection | DOAJ |
| description |
BackgroundSepsis is a complex, life-threatening condition characterized by significant heterogeneity and vast amounts of unstructured data, posing substantial challenges for traditional knowledge graph construction methods. The integration of large language models (LLMs) with real-world data offers a promising avenue to address these challenges and enhance the understanding and management of sepsis.
ObjectiveThis study aims to develop a comprehensive sepsis knowledge graph by leveraging the capabilities of LLMs, specifically GPT-4.0, in conjunction with multicenter clinical databases. The goal is to improve the understanding of sepsis and provide actionable insights for clinical decision-making. We also established a multicenter sepsis database (MSD) to support this effort.
MethodsWe collected clinical guidelines, public databases, and real-world data from 3 major hospitals in Western China, encompassing 10,544 patients diagnosed with sepsis. Using GPT-4.0, we used advanced prompt engineering techniques for entity recognition and relationship extraction, which facilitated the construction of a nuanced sepsis knowledge graph.
ResultsWe established a sepsis database with 10,544 patient records, including 8497 from West China Hospital, 690 from Shangjin Hospital, and 357 from Tianfu Hospital. The sepsis knowledge graph comprises of 1894 nodes and 2021 distinct relationships, encompassing nine entity concepts (diseases, symptoms, biomarkers, imaging examinations, etc) and 8 semantic relationships (complications, recommended medications, laboratory tests, etc). GPT-4.0 demonstrated superior performance in entity recognition and relationship extraction, achieving an F1-score of 76.76 on a sepsis-specific dataset, outperforming other models such as Qwen2 (43.77) and Llama3 (48.39). On the CMeEE dataset, GPT-4.0 achieved an F1-score of 65.42 using few-shot learning, surpassing traditional models such as BERT-CRF (62.11) and Med-BERT (60.66). Building upon this, we compiled a comprehensive sepsis knowledge graph, comprising of 1894 nodes and 2021 distinct relationships.
ConclusionsThis study represents a pioneering effort in using LLMs, particularly GPT-4.0, to construct a comprehensive sepsis knowledge graph. The innovative application of prompt engineering, combined with the integration of multicenter real-world data, has significantly enhanced the efficiency and accuracy of knowledge graph construction. The resulting knowledge graph provides a robust framework for understanding sepsis, supporting clinical decision-making, and facilitating further research. The success of this approach underscores the potential of LLMs in medical research and sets a new benchmark for future studies in sepsis and other complex medical conditions. |
| format | Article |
| id | doaj-art-fd0b3ccd5de346e6a2676f15d0fd076c |
| institution | OA Journals |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-fd0b3ccd5de346e6a2676f15d0fd076c2025-08-20T02:10:31ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-03-0127e6553710.2196/65537Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability StudyHao Yanghttps://orcid.org/0000-0002-3505-9403Jiaxi Lihttps://orcid.org/0000-0002-9820-8461Chi Zhanghttps://orcid.org/0009-0000-7838-9114Alejandro Pazos Sierrahttps://orcid.org/0000-0003-2324-238XBairong Shenhttps://orcid.org/0000-0003-2899-1531 BackgroundSepsis is a complex, life-threatening condition characterized by significant heterogeneity and vast amounts of unstructured data, posing substantial challenges for traditional knowledge graph construction methods. The integration of large language models (LLMs) with real-world data offers a promising avenue to address these challenges and enhance the understanding and management of sepsis. ObjectiveThis study aims to develop a comprehensive sepsis knowledge graph by leveraging the capabilities of LLMs, specifically GPT-4.0, in conjunction with multicenter clinical databases. The goal is to improve the understanding of sepsis and provide actionable insights for clinical decision-making. We also established a multicenter sepsis database (MSD) to support this effort. MethodsWe collected clinical guidelines, public databases, and real-world data from 3 major hospitals in Western China, encompassing 10,544 patients diagnosed with sepsis. Using GPT-4.0, we used advanced prompt engineering techniques for entity recognition and relationship extraction, which facilitated the construction of a nuanced sepsis knowledge graph. ResultsWe established a sepsis database with 10,544 patient records, including 8497 from West China Hospital, 690 from Shangjin Hospital, and 357 from Tianfu Hospital. The sepsis knowledge graph comprises of 1894 nodes and 2021 distinct relationships, encompassing nine entity concepts (diseases, symptoms, biomarkers, imaging examinations, etc) and 8 semantic relationships (complications, recommended medications, laboratory tests, etc). GPT-4.0 demonstrated superior performance in entity recognition and relationship extraction, achieving an F1-score of 76.76 on a sepsis-specific dataset, outperforming other models such as Qwen2 (43.77) and Llama3 (48.39). On the CMeEE dataset, GPT-4.0 achieved an F1-score of 65.42 using few-shot learning, surpassing traditional models such as BERT-CRF (62.11) and Med-BERT (60.66). Building upon this, we compiled a comprehensive sepsis knowledge graph, comprising of 1894 nodes and 2021 distinct relationships. ConclusionsThis study represents a pioneering effort in using LLMs, particularly GPT-4.0, to construct a comprehensive sepsis knowledge graph. The innovative application of prompt engineering, combined with the integration of multicenter real-world data, has significantly enhanced the efficiency and accuracy of knowledge graph construction. The resulting knowledge graph provides a robust framework for understanding sepsis, supporting clinical decision-making, and facilitating further research. The success of this approach underscores the potential of LLMs in medical research and sets a new benchmark for future studies in sepsis and other complex medical conditions.https://www.jmir.org/2025/1/e65537 |
| spellingShingle | Hao Yang Jiaxi Li Chi Zhang Alejandro Pazos Sierra Bairong Shen Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study Journal of Medical Internet Research |
| title | Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study |
| title_full | Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study |
| title_fullStr | Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study |
| title_full_unstemmed | Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study |
| title_short | Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study |
| title_sort | large language model driven knowledge graph construction in sepsis care using multicenter clinical databases development and usability study |
| url | https://www.jmir.org/2025/1/e65537 |
| work_keys_str_mv | AT haoyang largelanguagemodeldrivenknowledgegraphconstructioninsepsiscareusingmulticenterclinicaldatabasesdevelopmentandusabilitystudy AT jiaxili largelanguagemodeldrivenknowledgegraphconstructioninsepsiscareusingmulticenterclinicaldatabasesdevelopmentandusabilitystudy AT chizhang largelanguagemodeldrivenknowledgegraphconstructioninsepsiscareusingmulticenterclinicaldatabasesdevelopmentandusabilitystudy AT alejandropazossierra largelanguagemodeldrivenknowledgegraphconstructioninsepsiscareusingmulticenterclinicaldatabasesdevelopmentandusabilitystudy AT bairongshen largelanguagemodeldrivenknowledgegraphconstructioninsepsiscareusingmulticenterclinicaldatabasesdevelopmentandusabilitystudy |