Assessment of Racial Bias within the Risk Analysis Index of Frailty
Objective:. Our objective was to assess potential racial bias within the Risk Analysis Index (RAI). Background:. Patient risk measures are rarely tested for racial bias. Measures of frailty, like the RAI, need to be evaluated for poor predictive performance among Black patients. Methods:. Retrospect...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wolters Kluwer Health
2024-12-01
|
| Series: | Annals of Surgery Open |
| Online Access: | http://journals.lww.com/10.1097/AS9.0000000000000490 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589726411915264 |
|---|---|
| author | Michael A. Jacobs, MS Leslie R. M. Hausmann, PhD Robert M. Handzel, MD, MS Susanne Schmidt, PhD Carly A. Jacobs, MPH Daniel E. Hall, MD, MDiv, MHSc |
| author_facet | Michael A. Jacobs, MS Leslie R. M. Hausmann, PhD Robert M. Handzel, MD, MS Susanne Schmidt, PhD Carly A. Jacobs, MPH Daniel E. Hall, MD, MDiv, MHSc |
| author_sort | Michael A. Jacobs, MS |
| collection | DOAJ |
| description | Objective:. Our objective was to assess potential racial bias within the Risk Analysis Index (RAI).
Background:. Patient risk measures are rarely tested for racial bias. Measures of frailty, like the RAI, need to be evaluated for poor predictive performance among Black patients.
Methods:. Retrospective cohort study using April 2010–March 2019 Veterans Affairs Surgical Quality Improvement Program and 2010–2019 National Surgical Quality Improvement Program data. The performance of the RAI and several potential variants were compared between Black and White cases using various metrics to predict mortality (180-day for Veterans Affairs Surgical Quality Improvement Program, 30-day for National Surgical Quality Improvement Program).
Results:. Using the current, clinical threshold, the RAI performed as good or better among Black cases across various performance metrics versus White. When a higher threshold was used, Black cases had higher true positive rates but lower true negative rates, yielding 2.0% higher balanced accuracy. No RAI variant noticeably eliminated bias, improved parity across both true positives and true negatives, or improved overall model performance.
Conclusions:. The RAI tends to predict mortality among Black patients better than it predicts mortality among White patients. As existing bias-reducing techniques were not effective, further research into bias-reducing techniques is needed, especially for clinical risk predictions. We recommend using the RAI for both statistical analysis of surgical cohorts and quality improvement programs, such as the Surgical Pause. |
| format | Article |
| id | doaj-art-cb6b8a6f61bb4752822d20930e3c8b86 |
| institution | Kabale University |
| issn | 2691-3593 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Wolters Kluwer Health |
| record_format | Article |
| series | Annals of Surgery Open |
| spelling | doaj-art-cb6b8a6f61bb4752822d20930e3c8b862025-01-24T09:18:58ZengWolters Kluwer HealthAnnals of Surgery Open2691-35932024-12-0154e49010.1097/AS9.0000000000000490202412000-00003Assessment of Racial Bias within the Risk Analysis Index of FrailtyMichael A. Jacobs, MS0Leslie R. M. Hausmann, PhD1Robert M. Handzel, MD, MS2Susanne Schmidt, PhD3Carly A. Jacobs, MPH4Daniel E. Hall, MD, MDiv, MHSc5From the * Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PAFrom the * Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PAFrom the * Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA§ Department of Population Health Sciences, University of Texas Health San Antonio, San Antonio, TXFrom the * Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PAFrom the * Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PAObjective:. Our objective was to assess potential racial bias within the Risk Analysis Index (RAI). Background:. Patient risk measures are rarely tested for racial bias. Measures of frailty, like the RAI, need to be evaluated for poor predictive performance among Black patients. Methods:. Retrospective cohort study using April 2010–March 2019 Veterans Affairs Surgical Quality Improvement Program and 2010–2019 National Surgical Quality Improvement Program data. The performance of the RAI and several potential variants were compared between Black and White cases using various metrics to predict mortality (180-day for Veterans Affairs Surgical Quality Improvement Program, 30-day for National Surgical Quality Improvement Program). Results:. Using the current, clinical threshold, the RAI performed as good or better among Black cases across various performance metrics versus White. When a higher threshold was used, Black cases had higher true positive rates but lower true negative rates, yielding 2.0% higher balanced accuracy. No RAI variant noticeably eliminated bias, improved parity across both true positives and true negatives, or improved overall model performance. Conclusions:. The RAI tends to predict mortality among Black patients better than it predicts mortality among White patients. As existing bias-reducing techniques were not effective, further research into bias-reducing techniques is needed, especially for clinical risk predictions. We recommend using the RAI for both statistical analysis of surgical cohorts and quality improvement programs, such as the Surgical Pause.http://journals.lww.com/10.1097/AS9.0000000000000490 |
| spellingShingle | Michael A. Jacobs, MS Leslie R. M. Hausmann, PhD Robert M. Handzel, MD, MS Susanne Schmidt, PhD Carly A. Jacobs, MPH Daniel E. Hall, MD, MDiv, MHSc Assessment of Racial Bias within the Risk Analysis Index of Frailty Annals of Surgery Open |
| title | Assessment of Racial Bias within the Risk Analysis Index of Frailty |
| title_full | Assessment of Racial Bias within the Risk Analysis Index of Frailty |
| title_fullStr | Assessment of Racial Bias within the Risk Analysis Index of Frailty |
| title_full_unstemmed | Assessment of Racial Bias within the Risk Analysis Index of Frailty |
| title_short | Assessment of Racial Bias within the Risk Analysis Index of Frailty |
| title_sort | assessment of racial bias within the risk analysis index of frailty |
| url | http://journals.lww.com/10.1097/AS9.0000000000000490 |
| work_keys_str_mv | AT michaelajacobsms assessmentofracialbiaswithintheriskanalysisindexoffrailty AT lesliermhausmannphd assessmentofracialbiaswithintheriskanalysisindexoffrailty AT robertmhandzelmdms assessmentofracialbiaswithintheriskanalysisindexoffrailty AT susanneschmidtphd assessmentofracialbiaswithintheriskanalysisindexoffrailty AT carlyajacobsmph assessmentofracialbiaswithintheriskanalysisindexoffrailty AT danielehallmdmdivmhsc assessmentofracialbiaswithintheriskanalysisindexoffrailty |