376 Using a large language model to create lay summaries of clinical study descriptions
Objectives/Goals: We assessed the feasibility of using a large language model (LLM) to create lay language descriptions of study protocols for recruitment, which has the potential to improve accessibility and transparency of clinical studies and enable participants to make informed decisions. Method...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Cambridge University Press
2025-04-01
|
| Series: | Journal of Clinical and Translational Science |
| Online Access: | https://www.cambridge.org/core/product/identifier/S2059866124009993/type/journal_article |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849393729045004288 |
|---|---|
| author | Rebecca E Kaiser Farshad Sadr Trevor Yuen Till Krenz Lee Chin-Chin C Dominguez Sheela Daru LL Ransford Erin Kobetz |
| author_facet | Rebecca E Kaiser Farshad Sadr Trevor Yuen Till Krenz Lee Chin-Chin C Dominguez Sheela Daru LL Ransford Erin Kobetz |
| author_sort | Rebecca E Kaiser |
| collection | DOAJ |
| description | Objectives/Goals: We assessed the feasibility of using a large language model (LLM) to create lay language descriptions of study protocols for recruitment, which has the potential to improve accessibility and transparency of clinical studies and enable participants to make informed decisions. Methods/Study Population: All studies from a clinical research recruitment platform were included, which features human-written lay descriptions and titles for study recruitment. Corresponding protocol summaries in the IRB system were extracted and translated into lay language using a LLM (gpt-35-turbo-0613). A subset was used to develop prompt variations through an iterative process. Prompt strategies evaluated include chain-of-thought and few-shot prompting techniques. LLM-generated and human-written descriptions were compared for readability using Flesch–Kincaid and Simple Measure of Gobbledygook (SMOG) reading grade levels and information completeness using Word Movers’ Distance (WMD). Results/Anticipated Results: A total of 55 study descriptions were included – 10 were used to develop prompts and 45 were used for evaluation. The final LLM instructions included multistep prompts. The LLM was first instructed to produce a two- to three-sentence long description without using scientific jargon and included two pairs of examples. The LLM was then asked to shorten the description and finally to provide an engaging title. LLM-generated and human-written summaries were similar in length (median (IQR) 328 (278.5–360.5) vs. 342 (203–532.5) characters, respectively). LLM-generated summaries had lower Flesch-Kincaid grade level (5.15 vs. 8.28, p Discussion/Significance of Impact: An LLM can be used to generate lay language summaries that are readable at a lower grade level while maintaining semantic similarity. This approach can be used to improve the drafting of summaries for recruitment, thereby improving accessibility to potential participants. Future work includes human evaluation and implementation into practice. |
| format | Article |
| id | doaj-art-eb0f7bbe04eb483498c398451298e665 |
| institution | Kabale University |
| issn | 2059-8661 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Cambridge University Press |
| record_format | Article |
| series | Journal of Clinical and Translational Science |
| spelling | doaj-art-eb0f7bbe04eb483498c398451298e6652025-08-20T03:40:18ZengCambridge University PressJournal of Clinical and Translational Science2059-86612025-04-01911611610.1017/cts.2024.999376 Using a large language model to create lay summaries of clinical study descriptionsRebecca E Kaiser0Farshad Sadr1Trevor Yuen2Till Krenz3Lee Chin-Chin4C Dominguez Sheela5Daru LL Ransford6Erin Kobetz7University of Miami Health SystemUniversity of Miami Health SystemUniversity of Miami Health SystemUniversity of Miami Health SystemUniversity of Miami Health SystemUniversity of Miami Health SystemUniversity of Miami Health SystemUniversity of Miami Health SystemObjectives/Goals: We assessed the feasibility of using a large language model (LLM) to create lay language descriptions of study protocols for recruitment, which has the potential to improve accessibility and transparency of clinical studies and enable participants to make informed decisions. Methods/Study Population: All studies from a clinical research recruitment platform were included, which features human-written lay descriptions and titles for study recruitment. Corresponding protocol summaries in the IRB system were extracted and translated into lay language using a LLM (gpt-35-turbo-0613). A subset was used to develop prompt variations through an iterative process. Prompt strategies evaluated include chain-of-thought and few-shot prompting techniques. LLM-generated and human-written descriptions were compared for readability using Flesch–Kincaid and Simple Measure of Gobbledygook (SMOG) reading grade levels and information completeness using Word Movers’ Distance (WMD). Results/Anticipated Results: A total of 55 study descriptions were included – 10 were used to develop prompts and 45 were used for evaluation. The final LLM instructions included multistep prompts. The LLM was first instructed to produce a two- to three-sentence long description without using scientific jargon and included two pairs of examples. The LLM was then asked to shorten the description and finally to provide an engaging title. LLM-generated and human-written summaries were similar in length (median (IQR) 328 (278.5–360.5) vs. 342 (203–532.5) characters, respectively). LLM-generated summaries had lower Flesch-Kincaid grade level (5.15 vs. 8.28, p Discussion/Significance of Impact: An LLM can be used to generate lay language summaries that are readable at a lower grade level while maintaining semantic similarity. This approach can be used to improve the drafting of summaries for recruitment, thereby improving accessibility to potential participants. Future work includes human evaluation and implementation into practice.https://www.cambridge.org/core/product/identifier/S2059866124009993/type/journal_article |
| spellingShingle | Rebecca E Kaiser Farshad Sadr Trevor Yuen Till Krenz Lee Chin-Chin C Dominguez Sheela Daru LL Ransford Erin Kobetz 376 Using a large language model to create lay summaries of clinical study descriptions Journal of Clinical and Translational Science |
| title | 376 Using a large language model to create lay summaries of clinical study descriptions |
| title_full | 376 Using a large language model to create lay summaries of clinical study descriptions |
| title_fullStr | 376 Using a large language model to create lay summaries of clinical study descriptions |
| title_full_unstemmed | 376 Using a large language model to create lay summaries of clinical study descriptions |
| title_short | 376 Using a large language model to create lay summaries of clinical study descriptions |
| title_sort | 376 using a large language model to create lay summaries of clinical study descriptions |
| url | https://www.cambridge.org/core/product/identifier/S2059866124009993/type/journal_article |
| work_keys_str_mv | AT rebeccaekaiser 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions AT farshadsadr 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions AT trevoryuen 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions AT tillkrenz 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions AT leechinchin 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions AT cdominguezsheela 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions AT darullransford 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions AT erinkobetz 376usingalargelanguagemodeltocreatelaysummariesofclinicalstudydescriptions |