Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning
In the past decade, personalized treatment has become increasingly significant; however, the individualized care provided by physicians incurs substantial labor costs. The advancement of artificial intelligence, particularly the emergence of large language models, offers promising prospects for pers...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/5/2486 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850050824964669440 |
|---|---|
| author | Jinsheng Shi Yuyu Yuan Ao Wang Meng Nie |
| author_facet | Jinsheng Shi Yuyu Yuan Ao Wang Meng Nie |
| author_sort | Jinsheng Shi |
| collection | DOAJ |
| description | In the past decade, personalized treatment has become increasingly significant; however, the individualized care provided by physicians incurs substantial labor costs. The advancement of artificial intelligence, particularly the emergence of large language models, offers promising prospects for personalized treatment. These models have the potential to create intelligent agents capable of delivering tailored medical care. Nonetheless, current large language models face challenges in achieving precise treatment recommendations and optimizing treatment strategies. This study aims to enhance the accuracy of medical large language models by employing representation learning, inverse reinforcement learning, and offline reinforcement learning techniques for fine-tuning. The fine-tuned models are capable of generating personalized treatment plans. Through experiments on the mechanical ventilation therapy dataset constructed from the MIMIC-III database, we found that the similarity between the treatment actions suggested by the fine-tuned models and those of clinical physicians reaches 90.4% in the survival dataset, while it decreases to 72.3% in the mortality dataset. These results suggest that fine-tuned models are able to effectively capture individual patient information and provide treatment strategies that differ from routine clinical practice in complex situations, rather than merely mimicking physician behavior. Nevertheless, we emphasize that the recommendations of the model should be used as an auxiliary tool for physician decision making, and the final medical decision still needs to be comprehensively considered in combination with the patient’s clinical background, medical history, and autonomy. |
| format | Article |
| id | doaj-art-9faddc84b91e49cbb6d50b162d8248f5 |
| institution | DOAJ |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-9faddc84b91e49cbb6d50b162d8248f52025-08-20T02:53:19ZengMDPI AGApplied Sciences2076-34172025-02-01155248610.3390/app15052486Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement LearningJinsheng Shi0Yuyu Yuan1Ao Wang2Meng Nie3School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Xitucheng Road 10, Haidian, Beijing 100876, ChinaSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Xitucheng Road 10, Haidian, Beijing 100876, ChinaSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Xitucheng Road 10, Haidian, Beijing 100876, ChinaSchool of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Xitucheng Road 10, Haidian, Beijing 100876, ChinaIn the past decade, personalized treatment has become increasingly significant; however, the individualized care provided by physicians incurs substantial labor costs. The advancement of artificial intelligence, particularly the emergence of large language models, offers promising prospects for personalized treatment. These models have the potential to create intelligent agents capable of delivering tailored medical care. Nonetheless, current large language models face challenges in achieving precise treatment recommendations and optimizing treatment strategies. This study aims to enhance the accuracy of medical large language models by employing representation learning, inverse reinforcement learning, and offline reinforcement learning techniques for fine-tuning. The fine-tuned models are capable of generating personalized treatment plans. Through experiments on the mechanical ventilation therapy dataset constructed from the MIMIC-III database, we found that the similarity between the treatment actions suggested by the fine-tuned models and those of clinical physicians reaches 90.4% in the survival dataset, while it decreases to 72.3% in the mortality dataset. These results suggest that fine-tuned models are able to effectively capture individual patient information and provide treatment strategies that differ from routine clinical practice in complex situations, rather than merely mimicking physician behavior. Nevertheless, we emphasize that the recommendations of the model should be used as an auxiliary tool for physician decision making, and the final medical decision still needs to be comprehensively considered in combination with the patient’s clinical background, medical history, and autonomy.https://www.mdpi.com/2076-3417/15/5/2486representation learninginverse reinforcement learningoffline reinforcement learningmedical large language modelpersonalized treatment |
| spellingShingle | Jinsheng Shi Yuyu Yuan Ao Wang Meng Nie Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning Applied Sciences representation learning inverse reinforcement learning offline reinforcement learning medical large language model personalized treatment |
| title | Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning |
| title_full | Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning |
| title_fullStr | Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning |
| title_full_unstemmed | Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning |
| title_short | Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning |
| title_sort | fine tuning a personalized openbiollm using offline reinforcement learning |
| topic | representation learning inverse reinforcement learning offline reinforcement learning medical large language model personalized treatment |
| url | https://www.mdpi.com/2076-3417/15/5/2486 |
| work_keys_str_mv | AT jinshengshi finetuningapersonalizedopenbiollmusingofflinereinforcementlearning AT yuyuyuan finetuningapersonalizedopenbiollmusingofflinereinforcementlearning AT aowang finetuningapersonalizedopenbiollmusingofflinereinforcementlearning AT mengnie finetuningapersonalizedopenbiollmusingofflinereinforcementlearning |