Augmented reality microscopy to bridge trust between AI and pathologists
Abstract Diagnostic certainty is the cornerstone of modern medicine and critical for maximal treatment benefit. When evaluating biomarker expression by immunohistochemistry (IHC), however, pathologists are hindered by complex scoring methodologies, unique positivity cut-offs and subjective staining...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-05-01
|
| Series: | npj Precision Oncology |
| Online Access: | https://doi.org/10.1038/s41698-025-00899-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850272000801505280 |
|---|---|
| author | Sunil Badve George L. Kumar Tobias Lang Eli Peigin James Pratt Robert Anders Deyali Chatterjee Raul S. Gonzalez Rondell P. Graham Alyssa M. Krasinskas Xiuli Liu Alexander Quaas Romil Saxena Namrata Setia Laura Tang Hanlin L. Wang Josef Rüschoff Hans-Ulrich Schildhaus Khalid Daifalla Marc Päpper Patrick Frey Felix Faber Maria Karasarides |
| author_facet | Sunil Badve George L. Kumar Tobias Lang Eli Peigin James Pratt Robert Anders Deyali Chatterjee Raul S. Gonzalez Rondell P. Graham Alyssa M. Krasinskas Xiuli Liu Alexander Quaas Romil Saxena Namrata Setia Laura Tang Hanlin L. Wang Josef Rüschoff Hans-Ulrich Schildhaus Khalid Daifalla Marc Päpper Patrick Frey Felix Faber Maria Karasarides |
| author_sort | Sunil Badve |
| collection | DOAJ |
| description | Abstract Diagnostic certainty is the cornerstone of modern medicine and critical for maximal treatment benefit. When evaluating biomarker expression by immunohistochemistry (IHC), however, pathologists are hindered by complex scoring methodologies, unique positivity cut-offs and subjective staining interpretation. Artificial intelligence (AI) can potentially eliminate diagnostic uncertainty, especially when AI “trustworthiness” is proven by expert pathologists in the context of real-world clinical practice. Building on an IHC foundation model, we employed pathologists-in-the-loop finetuning to produce a programmed cell death ligand 1 (PD-L1) CPS AI Model. We devised a multi-head augmented reality microscope (ARM) system overlayed with the PD-L1 CPS AI Model to assess interobserver variability and gauge the pathologists’ trust in AI model outputs. Using difficult to interpret regions on gastroesophageal biopsies, we show that AI-assistance improved case agreement between any 2 pathologists by 14% (agreement on 77% vs 91%) and among 11 pathologists by 26% (agreement on 43% vs 69%). At a clinical cutoff of PD-L1 CPS ≥ 5, the number of cases diagnosed as positive by all 11 pathologists increased by 31%. Our findings underscore the benefits of fully engaging pathologists as active participants in the development and deployment of IHC AI models and frame the roadmap for trustworthy AI as a bridge to increased adoption in routine pathology practice. |
| format | Article |
| id | doaj-art-9c1cf13975604bb4bc5abf3a510d95ad |
| institution | OA Journals |
| issn | 2397-768X |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Precision Oncology |
| spelling | doaj-art-9c1cf13975604bb4bc5abf3a510d95ad2025-08-20T01:51:59ZengNature Portfolionpj Precision Oncology2397-768X2025-05-019111310.1038/s41698-025-00899-5Augmented reality microscopy to bridge trust between AI and pathologistsSunil Badve0George L. Kumar1Tobias Lang2Eli Peigin3James Pratt4Robert Anders5Deyali Chatterjee6Raul S. Gonzalez7Rondell P. Graham8Alyssa M. Krasinskas9Xiuli Liu10Alexander Quaas11Romil Saxena12Namrata Setia13Laura Tang14Hanlin L. Wang15Josef Rüschoff16Hans-Ulrich Schildhaus17Khalid Daifalla18Marc Päpper19Patrick Frey20Felix Faber21Maria Karasarides22Emory University School of MedicineBristol Myers SquibbMindpeakAugmentiqs, D.NBristol Myers SquibbJohns Hopkins University BaltimoreMD Anderson Cancer CenterEmory University School of MedicineMayo ClinicEmory University School of MedicineWashington University School of MedicineCologne University HospitalEmory University School of MedicineUniversity of ChicagoMemorial Sloan Kettering Cancer CenterUCLA David Geffen School of MedicineDiscovery Life Sciences Biomarker Services GmbHDiscovery Life Sciences Biomarker Services GmbHMindpeakMindpeakMindpeakMindpeakBristol Myers SquibbAbstract Diagnostic certainty is the cornerstone of modern medicine and critical for maximal treatment benefit. When evaluating biomarker expression by immunohistochemistry (IHC), however, pathologists are hindered by complex scoring methodologies, unique positivity cut-offs and subjective staining interpretation. Artificial intelligence (AI) can potentially eliminate diagnostic uncertainty, especially when AI “trustworthiness” is proven by expert pathologists in the context of real-world clinical practice. Building on an IHC foundation model, we employed pathologists-in-the-loop finetuning to produce a programmed cell death ligand 1 (PD-L1) CPS AI Model. We devised a multi-head augmented reality microscope (ARM) system overlayed with the PD-L1 CPS AI Model to assess interobserver variability and gauge the pathologists’ trust in AI model outputs. Using difficult to interpret regions on gastroesophageal biopsies, we show that AI-assistance improved case agreement between any 2 pathologists by 14% (agreement on 77% vs 91%) and among 11 pathologists by 26% (agreement on 43% vs 69%). At a clinical cutoff of PD-L1 CPS ≥ 5, the number of cases diagnosed as positive by all 11 pathologists increased by 31%. Our findings underscore the benefits of fully engaging pathologists as active participants in the development and deployment of IHC AI models and frame the roadmap for trustworthy AI as a bridge to increased adoption in routine pathology practice.https://doi.org/10.1038/s41698-025-00899-5 |
| spellingShingle | Sunil Badve George L. Kumar Tobias Lang Eli Peigin James Pratt Robert Anders Deyali Chatterjee Raul S. Gonzalez Rondell P. Graham Alyssa M. Krasinskas Xiuli Liu Alexander Quaas Romil Saxena Namrata Setia Laura Tang Hanlin L. Wang Josef Rüschoff Hans-Ulrich Schildhaus Khalid Daifalla Marc Päpper Patrick Frey Felix Faber Maria Karasarides Augmented reality microscopy to bridge trust between AI and pathologists npj Precision Oncology |
| title | Augmented reality microscopy to bridge trust between AI and pathologists |
| title_full | Augmented reality microscopy to bridge trust between AI and pathologists |
| title_fullStr | Augmented reality microscopy to bridge trust between AI and pathologists |
| title_full_unstemmed | Augmented reality microscopy to bridge trust between AI and pathologists |
| title_short | Augmented reality microscopy to bridge trust between AI and pathologists |
| title_sort | augmented reality microscopy to bridge trust between ai and pathologists |
| url | https://doi.org/10.1038/s41698-025-00899-5 |
| work_keys_str_mv | AT sunilbadve augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT georgelkumar augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT tobiaslang augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT elipeigin augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT jamespratt augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT robertanders augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT deyalichatterjee augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT raulsgonzalez augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT rondellpgraham augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT alyssamkrasinskas augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT xiuliliu augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT alexanderquaas augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT romilsaxena augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT namratasetia augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT lauratang augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT hanlinlwang augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT josefruschoff augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT hansulrichschildhaus augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT khaliddaifalla augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT marcpapper augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT patrickfrey augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT felixfaber augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists AT mariakarasarides augmentedrealitymicroscopytobridgetrustbetweenaiandpathologists |