Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema
Abstract Data Harmonization is an important yet time-consuming process. With the recent popularity of applications using Language Models (LMs) due to their high capabilities in text understanding, we investigated whether LMs could facilitate data harmonization for clinical use cases. To evaluate thi...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-06-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-06447-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850113080538693632 |
|---|---|
| author | Yasamin Salimi Tim Adams Mehmet Can Ay Helena Balabin Marc Jacobs Martin Hofmann-Apitius |
| author_facet | Yasamin Salimi Tim Adams Mehmet Can Ay Helena Balabin Marc Jacobs Martin Hofmann-Apitius |
| author_sort | Yasamin Salimi |
| collection | DOAJ |
| description | Abstract Data Harmonization is an important yet time-consuming process. With the recent popularity of applications using Language Models (LMs) due to their high capabilities in text understanding, we investigated whether LMs could facilitate data harmonization for clinical use cases. To evaluate this, we created PASSIONATE, a novel Parkinson’s disease (PD) variable mapping schema as a ground truth source for pairwise cohort harmonization using LLMs. Additionally, we extended our investigation using an existing Alzheimer’s disease (AD) CDM. We computed text embeddings based on two language models to perform automated cohort harmonization for both AD and PD. We additionally compared the results to a baseline method using fuzzy string matching to determine the degree to which the semantic capabilities of language models can be utilized for automated cohort harmonization. We found that mappings based on text embeddings performed significantly better than those generated by fuzzy string matching, reaching an average accuracy of over 80% for almost all tested PD cohorts. When extended to a further neighborhood of possible matches, the accuracy could be improved to up to 96%. Our results suggest that language models can be used for automated harmonization with a high accuracy that can potentially be improved in the future by applying domain-trained models. |
| format | Article |
| id | doaj-art-13a27ffcc79b4a0ebd6bc7f7374c0a23 |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-13a27ffcc79b4a0ebd6bc7f7374c0a232025-08-20T02:37:14ZengNature PortfolioScientific Reports2045-23222025-06-0115111310.1038/s41598-025-06447-2Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schemaYasamin Salimi0Tim Adams1Mehmet Can Ay2Helena Balabin3Marc Jacobs4Martin Hofmann-Apitius5Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)Laboratory for Cognitive Neurology, Department of Neurosciences, KU LeuvenDepartment of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)Abstract Data Harmonization is an important yet time-consuming process. With the recent popularity of applications using Language Models (LMs) due to their high capabilities in text understanding, we investigated whether LMs could facilitate data harmonization for clinical use cases. To evaluate this, we created PASSIONATE, a novel Parkinson’s disease (PD) variable mapping schema as a ground truth source for pairwise cohort harmonization using LLMs. Additionally, we extended our investigation using an existing Alzheimer’s disease (AD) CDM. We computed text embeddings based on two language models to perform automated cohort harmonization for both AD and PD. We additionally compared the results to a baseline method using fuzzy string matching to determine the degree to which the semantic capabilities of language models can be utilized for automated cohort harmonization. We found that mappings based on text embeddings performed significantly better than those generated by fuzzy string matching, reaching an average accuracy of over 80% for almost all tested PD cohorts. When extended to a further neighborhood of possible matches, the accuracy could be improved to up to 96%. Our results suggest that language models can be used for automated harmonization with a high accuracy that can potentially be improved in the future by applying domain-trained models.https://doi.org/10.1038/s41598-025-06447-2Alzheimer’s diseaseAutomatic data harmonizationParkinson’s diseaseLarge language modelsData stewardship |
| spellingShingle | Yasamin Salimi Tim Adams Mehmet Can Ay Helena Balabin Marc Jacobs Martin Hofmann-Apitius Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema Scientific Reports Alzheimer’s disease Automatic data harmonization Parkinson’s disease Large language models Data stewardship |
| title | Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema |
| title_full | Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema |
| title_fullStr | Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema |
| title_full_unstemmed | Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema |
| title_short | Evaluating language model embeddings for Parkinson’s disease cohort harmonization using a novel manually curated variable mapping schema |
| title_sort | evaluating language model embeddings for parkinson s disease cohort harmonization using a novel manually curated variable mapping schema |
| topic | Alzheimer’s disease Automatic data harmonization Parkinson’s disease Large language models Data stewardship |
| url | https://doi.org/10.1038/s41598-025-06447-2 |
| work_keys_str_mv | AT yasaminsalimi evaluatinglanguagemodelembeddingsforparkinsonsdiseasecohortharmonizationusinganovelmanuallycuratedvariablemappingschema AT timadams evaluatinglanguagemodelembeddingsforparkinsonsdiseasecohortharmonizationusinganovelmanuallycuratedvariablemappingschema AT mehmetcanay evaluatinglanguagemodelembeddingsforparkinsonsdiseasecohortharmonizationusinganovelmanuallycuratedvariablemappingschema AT helenabalabin evaluatinglanguagemodelembeddingsforparkinsonsdiseasecohortharmonizationusinganovelmanuallycuratedvariablemappingschema AT marcjacobs evaluatinglanguagemodelembeddingsforparkinsonsdiseasecohortharmonizationusinganovelmanuallycuratedvariablemappingschema AT martinhofmannapitius evaluatinglanguagemodelembeddingsforparkinsonsdiseasecohortharmonizationusinganovelmanuallycuratedvariablemappingschema |