Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancer
Abstract Background Recent advancements in immunotherapy, particularly pembrolizumab, have shown promising results in treating metastatic colorectal cancer (CRC) and triple-negative breast cancer (TNBC). Accurate detection of predictive biomarkers, such as microsatellite instability (MSI)/mismatch r...
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-08-01
|
| Series: | Communications Medicine |
| Online Access: | https://doi.org/10.1038/s43856-025-01045-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849226009202655232 |
|---|---|
| author | Yating Cheng Norsang Lama Ming Chen Eghbal Amidi Mohammadreza Ramzanpour Md Ashequr Rahman Joanne Xiu Anthony Helmstetter Lauren Dickman Jennifer R. Ribeiro Hassan Ghani Matthew Oberley David Spetzler George W. Sledge |
| author_facet | Yating Cheng Norsang Lama Ming Chen Eghbal Amidi Mohammadreza Ramzanpour Md Ashequr Rahman Joanne Xiu Anthony Helmstetter Lauren Dickman Jennifer R. Ribeiro Hassan Ghani Matthew Oberley David Spetzler George W. Sledge |
| author_sort | Yating Cheng |
| collection | DOAJ |
| description | Abstract Background Recent advancements in immunotherapy, particularly pembrolizumab, have shown promising results in treating metastatic colorectal cancer (CRC) and triple-negative breast cancer (TNBC). Accurate detection of predictive biomarkers, such as microsatellite instability (MSI)/mismatch repair deficiency (MMRd) and programmed death-ligand 1 (PD-L1), is key to efficacy of these treatments. Traditional methods like immunohistochemistry (IHC) and next-generation sequencing are effective but are labor intensive and require subjective interpretation. Methods We developed a dual-modality transformer-based model for predicting MSI/MMRd and PD-L1 status using hematoxylin & eosin and IHC stained whole slide images. We evaluated the model using area under the receiver operating curve (AUROC). Time-on-treatment (TOT) and overall survival (OS) were derived from insurance claims and analyzed by Kaplan–Meier method. Hazard ratios (HR) were determined using the Cox proportional hazard model. Results Our AI framework achieves clinical-grade performance, with AUROC exceeding 0.97 for MSI/MMRd prediction in CRC and 0.96 for PD-L1 prediction in breast cancer. Patients with biomarker-positive model predictions demonstrated prolonged TOT and OS when treated with pembrolizumab. For breast cancer patients, the model’s predictions were superior to PD-L1 IHC in stratifying patients with improved outcomes on pembrolizumab, suggesting a reevaluation of existing PD-L1 status thresholds. Conclusions This study promotes the integration of advanced AI tools in clinical pathology, aiming to enhance the precision and efficiency of cancer biomarker evaluation and offering a customizable framework for varied clinical scenarios. Our model enhances predictive accuracy, integrating features from both staining methods, and exhibits superior prognostic precision compared to current biomarker assessments. |
| format | Article |
| id | doaj-art-aba32e06c14d4bcc89193bbae5614f24 |
| institution | Kabale University |
| issn | 2730-664X |
| language | English |
| publishDate | 2025-08-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Medicine |
| spelling | doaj-art-aba32e06c14d4bcc89193bbae5614f242025-08-24T11:47:36ZengNature PortfolioCommunications Medicine2730-664X2025-08-015111510.1038/s43856-025-01045-9Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancerYating Cheng0Norsang Lama1Ming Chen2Eghbal Amidi3Mohammadreza Ramzanpour4Md Ashequr Rahman5Joanne Xiu6Anthony Helmstetter7Lauren Dickman8Jennifer R. Ribeiro9Hassan Ghani10Matthew Oberley11David Spetzler12George W. Sledge13Caris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesCaris Life SciencesAbstract Background Recent advancements in immunotherapy, particularly pembrolizumab, have shown promising results in treating metastatic colorectal cancer (CRC) and triple-negative breast cancer (TNBC). Accurate detection of predictive biomarkers, such as microsatellite instability (MSI)/mismatch repair deficiency (MMRd) and programmed death-ligand 1 (PD-L1), is key to efficacy of these treatments. Traditional methods like immunohistochemistry (IHC) and next-generation sequencing are effective but are labor intensive and require subjective interpretation. Methods We developed a dual-modality transformer-based model for predicting MSI/MMRd and PD-L1 status using hematoxylin & eosin and IHC stained whole slide images. We evaluated the model using area under the receiver operating curve (AUROC). Time-on-treatment (TOT) and overall survival (OS) were derived from insurance claims and analyzed by Kaplan–Meier method. Hazard ratios (HR) were determined using the Cox proportional hazard model. Results Our AI framework achieves clinical-grade performance, with AUROC exceeding 0.97 for MSI/MMRd prediction in CRC and 0.96 for PD-L1 prediction in breast cancer. Patients with biomarker-positive model predictions demonstrated prolonged TOT and OS when treated with pembrolizumab. For breast cancer patients, the model’s predictions were superior to PD-L1 IHC in stratifying patients with improved outcomes on pembrolizumab, suggesting a reevaluation of existing PD-L1 status thresholds. Conclusions This study promotes the integration of advanced AI tools in clinical pathology, aiming to enhance the precision and efficiency of cancer biomarker evaluation and offering a customizable framework for varied clinical scenarios. Our model enhances predictive accuracy, integrating features from both staining methods, and exhibits superior prognostic precision compared to current biomarker assessments.https://doi.org/10.1038/s43856-025-01045-9 |
| spellingShingle | Yating Cheng Norsang Lama Ming Chen Eghbal Amidi Mohammadreza Ramzanpour Md Ashequr Rahman Joanne Xiu Anthony Helmstetter Lauren Dickman Jennifer R. Ribeiro Hassan Ghani Matthew Oberley David Spetzler George W. Sledge Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancer Communications Medicine |
| title | Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancer |
| title_full | Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancer |
| title_fullStr | Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancer |
| title_full_unstemmed | Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancer |
| title_short | Synergistic H&E and IHC image analysis by AI predicts cancer biomarkers and survival outcomes in colorectal and breast cancer |
| title_sort | synergistic h e and ihc image analysis by ai predicts cancer biomarkers and survival outcomes in colorectal and breast cancer |
| url | https://doi.org/10.1038/s43856-025-01045-9 |
| work_keys_str_mv | AT yatingcheng synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT norsanglama synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT mingchen synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT eghbalamidi synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT mohammadrezaramzanpour synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT mdashequrrahman synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT joannexiu synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT anthonyhelmstetter synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT laurendickman synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT jenniferrribeiro synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT hassanghani synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT matthewoberley synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT davidspetzler synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer AT georgewsledge synergisticheandihcimageanalysisbyaipredictscancerbiomarkersandsurvivaloutcomesincolorectalandbreastcancer |