Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping Review
BackgroundClinical risk prediction models integrated into digitized health care informatics systems hold promise for personalized primary prevention and care, a core goal of precision health. Fairness metrics are important tools for evaluating potential disparities across sen...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-03-01
|
| Series: | Online Journal of Public Health Informatics |
| Online Access: | https://ojphi.jmir.org/2025/1/e66598 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849390109016719360 |
|---|---|
| author | Lillian Rountree Yi-Ting Lin Chuyu Liu Maxwell Salvatore Andrew Admon Brahmajee Nallamothu Karandeep Singh Anirban Basu Fan Bu Bhramar Mukherjee |
| author_facet | Lillian Rountree Yi-Ting Lin Chuyu Liu Maxwell Salvatore Andrew Admon Brahmajee Nallamothu Karandeep Singh Anirban Basu Fan Bu Bhramar Mukherjee |
| author_sort | Lillian Rountree |
| collection | DOAJ |
| description |
BackgroundClinical risk prediction models integrated into digitized health care informatics systems hold promise for personalized primary prevention and care, a core goal of precision health. Fairness metrics are important tools for evaluating potential disparities across sensitive features, such as sex and race or ethnicity, in the field of prediction modeling. However, fairness metric usage in clinical risk prediction models remains infrequent, sporadic, and rarely empirically evaluated.
ObjectiveWe seek to assess the uptake of fairness metrics in clinical risk prediction modeling through an empirical evaluation of popular prediction models for 2 diseases, 1 chronic and 1 infectious disease.
MethodsWe conducted a scoping literature review in November 2023 of recent high-impact publications on clinical risk prediction models for cardiovascular disease (CVD) and COVID-19 using Google Scholar.
ResultsOur review resulted in a shortlist of 23 CVD-focused articles and 22 COVID-19 pandemic–focused articles. No articles evaluated fairness metrics. Of the CVD-focused articles, 26% used a sex-stratified model, and of those with race or ethnicity data, 92% had study populations that were more than 50% from 1 race or ethnicity. Of the COVID-19 models, 9% used a sex-stratified model, and of those that included race or ethnicity data, 50% had study populations that were more than 50% from 1 race or ethnicity. No articles for either disease stratified their models by race or ethnicity.
ConclusionsOur review shows that the use of fairness metrics for evaluating differences across sensitive features is rare, despite their ability to identify inequality and flag potential gaps in prevention and care. We also find that training data remain largely racially and ethnically homogeneous, demonstrating an urgent need for diversifying study cohorts and data collection. We propose an implementation framework to initiate change, calling for better connections between theory and practice when it comes to the adoption of fairness metrics for clinical risk prediction. We hypothesize that this integration will lead to a more equitable prediction world. |
| format | Article |
| id | doaj-art-ab15f20c056e4ac48137ae45b48e8d40 |
| institution | Kabale University |
| issn | 1947-2579 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Online Journal of Public Health Informatics |
| spelling | doaj-art-ab15f20c056e4ac48137ae45b48e8d402025-08-20T03:41:46ZengJMIR PublicationsOnline Journal of Public Health Informatics1947-25792025-03-0117e6659810.2196/66598Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping ReviewLillian Rountreehttps://orcid.org/0009-0008-3428-3914Yi-Ting Linhttps://orcid.org/0009-0006-8605-4086Chuyu Liuhttps://orcid.org/0009-0006-5514-6025Maxwell Salvatorehttps://orcid.org/0000-0002-3659-1514Andrew Admonhttps://orcid.org/0000-0002-7432-3764Brahmajee Nallamothuhttps://orcid.org/0000-0003-4331-6649Karandeep Singhhttps://orcid.org/0000-0001-8980-2330Anirban Basuhttps://orcid.org/0000-0003-4238-7402Fan Buhttps://orcid.org/0000-0003-3697-1477Bhramar Mukherjeehttps://orcid.org/0000-0003-0118-4561 BackgroundClinical risk prediction models integrated into digitized health care informatics systems hold promise for personalized primary prevention and care, a core goal of precision health. Fairness metrics are important tools for evaluating potential disparities across sensitive features, such as sex and race or ethnicity, in the field of prediction modeling. However, fairness metric usage in clinical risk prediction models remains infrequent, sporadic, and rarely empirically evaluated. ObjectiveWe seek to assess the uptake of fairness metrics in clinical risk prediction modeling through an empirical evaluation of popular prediction models for 2 diseases, 1 chronic and 1 infectious disease. MethodsWe conducted a scoping literature review in November 2023 of recent high-impact publications on clinical risk prediction models for cardiovascular disease (CVD) and COVID-19 using Google Scholar. ResultsOur review resulted in a shortlist of 23 CVD-focused articles and 22 COVID-19 pandemic–focused articles. No articles evaluated fairness metrics. Of the CVD-focused articles, 26% used a sex-stratified model, and of those with race or ethnicity data, 92% had study populations that were more than 50% from 1 race or ethnicity. Of the COVID-19 models, 9% used a sex-stratified model, and of those that included race or ethnicity data, 50% had study populations that were more than 50% from 1 race or ethnicity. No articles for either disease stratified their models by race or ethnicity. ConclusionsOur review shows that the use of fairness metrics for evaluating differences across sensitive features is rare, despite their ability to identify inequality and flag potential gaps in prevention and care. We also find that training data remain largely racially and ethnically homogeneous, demonstrating an urgent need for diversifying study cohorts and data collection. We propose an implementation framework to initiate change, calling for better connections between theory and practice when it comes to the adoption of fairness metrics for clinical risk prediction. We hypothesize that this integration will lead to a more equitable prediction world.https://ojphi.jmir.org/2025/1/e66598 |
| spellingShingle | Lillian Rountree Yi-Ting Lin Chuyu Liu Maxwell Salvatore Andrew Admon Brahmajee Nallamothu Karandeep Singh Anirban Basu Fan Bu Bhramar Mukherjee Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping Review Online Journal of Public Health Informatics |
| title | Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping Review |
| title_full | Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping Review |
| title_fullStr | Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping Review |
| title_full_unstemmed | Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping Review |
| title_short | Reporting of Fairness Metrics in Clinical Risk Prediction Models Used for Precision Health: Scoping Review |
| title_sort | reporting of fairness metrics in clinical risk prediction models used for precision health scoping review |
| url | https://ojphi.jmir.org/2025/1/e66598 |
| work_keys_str_mv | AT lillianrountree reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT yitinglin reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT chuyuliu reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT maxwellsalvatore reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT andrewadmon reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT brahmajeenallamothu reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT karandeepsingh reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT anirbanbasu reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT fanbu reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview AT bhramarmukherjee reportingoffairnessmetricsinclinicalriskpredictionmodelsusedforprecisionhealthscopingreview |