A distributional reinforcement learning model for optimal glucose control after cardiac surgery
Abstract This study introduces Glucose Level Understanding and Control Optimized for Safety and Efficacy (GLUCOSE), a distributional offline reinforcement learning algorithm for optimizing insulin dosing after cardiac surgery. Trained on 5228 patients, tested on 920, and externally validated on 649,...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-05-01
|
| Series: | npj Digital Medicine |
| Online Access: | https://doi.org/10.1038/s41746-025-01709-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849687966526472192 |
|---|---|
| author | Jacob M. Desman Zhang-Wei Hong Moein Sabounchi Ashwin S. Sawant Jaskirat Gill Ana C. Costa Gagan Kumar Rajeev Sharma Arpeta Gupta Paul McCarthy Veena Nandwani Doug Powell Alexandra Carideo Donnie Goodwin Sanam Ahmed Umesh Gidwani Matthew A. Levin Robin Varghese Farzan Filsoufi Robert Freeman Avniel Shetreat-Klein Alexander W. Charney Ira Hofer Lili Chan David Reich Patricia Kovatch Roopa Kohli-Seth Monica Kraft Pulkit Agrawal John A. Kellum Girish N. Nadkarni Ankit Sakhuja |
| author_facet | Jacob M. Desman Zhang-Wei Hong Moein Sabounchi Ashwin S. Sawant Jaskirat Gill Ana C. Costa Gagan Kumar Rajeev Sharma Arpeta Gupta Paul McCarthy Veena Nandwani Doug Powell Alexandra Carideo Donnie Goodwin Sanam Ahmed Umesh Gidwani Matthew A. Levin Robin Varghese Farzan Filsoufi Robert Freeman Avniel Shetreat-Klein Alexander W. Charney Ira Hofer Lili Chan David Reich Patricia Kovatch Roopa Kohli-Seth Monica Kraft Pulkit Agrawal John A. Kellum Girish N. Nadkarni Ankit Sakhuja |
| author_sort | Jacob M. Desman |
| collection | DOAJ |
| description | Abstract This study introduces Glucose Level Understanding and Control Optimized for Safety and Efficacy (GLUCOSE), a distributional offline reinforcement learning algorithm for optimizing insulin dosing after cardiac surgery. Trained on 5228 patients, tested on 920, and externally validated on 649, GLUCOSE achieved a mean estimated reward of 0.0 [–0.07, 0.06] in internal testing and –0.63 [–0.74, –0.52] in external validation, outperforming clinician returns of –1.29 [–1.37, –1.20] and –1.02 [–1.16, –0.89]. In multi-phase human validation, GLUCOSE first showed a significantly lower mean absolute error (MAE) in insulin dosing, with 0.9 units MAE versus clinicians’ 1.97 units (p < 0.001) in internal testing and 1.90 versus 2.24 units (p = 0.003) in external validation. The second and third phases found GLUCOSE’s performance as comparable to or exceeding that of senior clinicians in MAE, safety, effectiveness, and acceptability. These findings suggest GLUCOSE as a robust tool for improving postoperative glucose management. |
| format | Article |
| id | doaj-art-a6be6a803b9240ce965adb2f2bd81eae |
| institution | DOAJ |
| issn | 2398-6352 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Digital Medicine |
| spelling | doaj-art-a6be6a803b9240ce965adb2f2bd81eae2025-08-20T03:22:11ZengNature Portfolionpj Digital Medicine2398-63522025-05-018111210.1038/s41746-025-01709-9A distributional reinforcement learning model for optimal glucose control after cardiac surgeryJacob M. Desman0Zhang-Wei Hong1Moein Sabounchi2Ashwin S. Sawant3Jaskirat Gill4Ana C. Costa5Gagan Kumar6Rajeev Sharma7Arpeta Gupta8Paul McCarthy9Veena Nandwani10Doug Powell11Alexandra Carideo12Donnie Goodwin13Sanam Ahmed14Umesh Gidwani15Matthew A. Levin16Robin Varghese17Farzan Filsoufi18Robert Freeman19Avniel Shetreat-Klein20Alexander W. Charney21Ira Hofer22Lili Chan23David Reich24Patricia Kovatch25Roopa Kohli-Seth26Monica Kraft27Pulkit Agrawal28John A. Kellum29Girish N. Nadkarni30Ankit Sakhuja31The Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiImprobable AI Lab, Massachusetts Institute of TechnologyThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiInstitute for Critical Care Medicine, Icahn School of Medicine at Mount SinaiDepartment of Cardiothoracic Surgery, Icahn School of Medicine at Mount SinaiDepartment of Pulmonary and Critical Care Medicine, Northeast Georgia Medical CenterDivision of Endocrinology, Hackensack University Medical CenterDivision of Endocrinology, Millenium Physician GroupSection of Cardiovascular Critical Care, Department of Cardiovascular and Thoracic Surgery, West Virginia UniversitySection of Cardiovascular Critical Care, Department of Cardiovascular and Thoracic Surgery, West Virginia UniversitySection of Cardiovascular Critical Care, Department of Cardiovascular and Thoracic Surgery, West Virginia UniversityInstitute for Critical Care Medicine, Icahn School of Medicine at Mount SinaiSection of Cardiovascular Critical Care, Department of Cardiovascular and Thoracic Surgery, West Virginia UniversityInstitute for Critical Care Medicine, Icahn School of Medicine at Mount SinaiInstitute for Critical Care Medicine, Icahn School of Medicine at Mount SinaiDepartment of Anesthesiology, Perioperative, and Pain Medicine, Icahn School of Medicine at Mount SinaiInstitute for Critical Care Medicine, Icahn School of Medicine at Mount SinaiDepartment of Cardiothoracic Surgery, Icahn School of Medicine at Mount SinaiThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiDepartment of Rehabilitation and Physical Medicine, Icahn School of Medicine at Mount SinaiThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiDepartment of Anesthesiology, Perioperative, and Pain Medicine, Icahn School of Medicine at Mount SinaiScientific Computing, Icahn School of Medicine at Mount SinaiInstitute for Critical Care Medicine, Icahn School of Medicine at Mount SinaiSamuel Bronfman Department of Medicine, Icahn School of Medicine at Mount SinaiImprobable AI Lab, Massachusetts Institute of TechnologyDepartment of Critical Care Medicine, University of Pittsburgh School of MedicineThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiThe Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount SinaiAbstract This study introduces Glucose Level Understanding and Control Optimized for Safety and Efficacy (GLUCOSE), a distributional offline reinforcement learning algorithm for optimizing insulin dosing after cardiac surgery. Trained on 5228 patients, tested on 920, and externally validated on 649, GLUCOSE achieved a mean estimated reward of 0.0 [–0.07, 0.06] in internal testing and –0.63 [–0.74, –0.52] in external validation, outperforming clinician returns of –1.29 [–1.37, –1.20] and –1.02 [–1.16, –0.89]. In multi-phase human validation, GLUCOSE first showed a significantly lower mean absolute error (MAE) in insulin dosing, with 0.9 units MAE versus clinicians’ 1.97 units (p < 0.001) in internal testing and 1.90 versus 2.24 units (p = 0.003) in external validation. The second and third phases found GLUCOSE’s performance as comparable to or exceeding that of senior clinicians in MAE, safety, effectiveness, and acceptability. These findings suggest GLUCOSE as a robust tool for improving postoperative glucose management.https://doi.org/10.1038/s41746-025-01709-9 |
| spellingShingle | Jacob M. Desman Zhang-Wei Hong Moein Sabounchi Ashwin S. Sawant Jaskirat Gill Ana C. Costa Gagan Kumar Rajeev Sharma Arpeta Gupta Paul McCarthy Veena Nandwani Doug Powell Alexandra Carideo Donnie Goodwin Sanam Ahmed Umesh Gidwani Matthew A. Levin Robin Varghese Farzan Filsoufi Robert Freeman Avniel Shetreat-Klein Alexander W. Charney Ira Hofer Lili Chan David Reich Patricia Kovatch Roopa Kohli-Seth Monica Kraft Pulkit Agrawal John A. Kellum Girish N. Nadkarni Ankit Sakhuja A distributional reinforcement learning model for optimal glucose control after cardiac surgery npj Digital Medicine |
| title | A distributional reinforcement learning model for optimal glucose control after cardiac surgery |
| title_full | A distributional reinforcement learning model for optimal glucose control after cardiac surgery |
| title_fullStr | A distributional reinforcement learning model for optimal glucose control after cardiac surgery |
| title_full_unstemmed | A distributional reinforcement learning model for optimal glucose control after cardiac surgery |
| title_short | A distributional reinforcement learning model for optimal glucose control after cardiac surgery |
| title_sort | distributional reinforcement learning model for optimal glucose control after cardiac surgery |
| url | https://doi.org/10.1038/s41746-025-01709-9 |
| work_keys_str_mv | AT jacobmdesman adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT zhangweihong adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT moeinsabounchi adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT ashwinssawant adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT jaskiratgill adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT anaccosta adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT gagankumar adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT rajeevsharma adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT arpetagupta adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT paulmccarthy adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT veenanandwani adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT dougpowell adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT alexandracarideo adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT donniegoodwin adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT sanamahmed adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT umeshgidwani adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT matthewalevin adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT robinvarghese adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT farzanfilsoufi adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT robertfreeman adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT avnielshetreatklein adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT alexanderwcharney adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT irahofer adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT lilichan adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT davidreich adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT patriciakovatch adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT roopakohliseth adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT monicakraft adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT pulkitagrawal adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT johnakellum adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT girishnnadkarni adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT ankitsakhuja adistributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT jacobmdesman distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT zhangweihong distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT moeinsabounchi distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT ashwinssawant distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT jaskiratgill distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT anaccosta distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT gagankumar distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT rajeevsharma distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT arpetagupta distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT paulmccarthy distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT veenanandwani distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT dougpowell distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT alexandracarideo distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT donniegoodwin distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT sanamahmed distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT umeshgidwani distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT matthewalevin distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT robinvarghese distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT farzanfilsoufi distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT robertfreeman distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT avnielshetreatklein distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT alexanderwcharney distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT irahofer distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT lilichan distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT davidreich distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT patriciakovatch distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT roopakohliseth distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT monicakraft distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT pulkitagrawal distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT johnakellum distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT girishnnadkarni distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery AT ankitsakhuja distributionalreinforcementlearningmodelforoptimalglucosecontrolaftercardiacsurgery |