A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slides
Abstract Mitotic activity is an important feature for grading several cancer types. However, counting mitotic figures (cells in division) is a time-consuming and laborious task prone to inter-observer variation. Inaccurate recognition of MFs can lead to incorrect grading and hence potential suboptim...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | Communications Biology |
| Online Access: | https://doi.org/10.1038/s42003-024-07398-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850102771827605504 |
|---|---|
| author | Zhuoyan Shen Mikaël Simard Douglas Brand Vanghelita Andrei Ali Al-Khader Fatine Oumlil Katherine Trevers Thomas Butters Simon Haefliger Eleanna Kara Fernanda Amary Roberto Tirabosco Paul Cool Gary Royle Maria A. Hawkins Adrienne M. Flanagan Charles-Antoine Collins-Fekete |
| author_facet | Zhuoyan Shen Mikaël Simard Douglas Brand Vanghelita Andrei Ali Al-Khader Fatine Oumlil Katherine Trevers Thomas Butters Simon Haefliger Eleanna Kara Fernanda Amary Roberto Tirabosco Paul Cool Gary Royle Maria A. Hawkins Adrienne M. Flanagan Charles-Antoine Collins-Fekete |
| author_sort | Zhuoyan Shen |
| collection | DOAJ |
| description | Abstract Mitotic activity is an important feature for grading several cancer types. However, counting mitotic figures (cells in division) is a time-consuming and laborious task prone to inter-observer variation. Inaccurate recognition of MFs can lead to incorrect grading and hence potential suboptimal treatment. This study presents an artificial intelligence-based approach to detect mitotic figures in digitised whole-slide images stained with haematoxylin and eosin. Advances in this area are hampered by the small size and variety of datasets available. To address this, we create the largest dataset of mitotic figures (N = 74,620), combining an in-house dataset of soft tissue tumours with five open-source datasets. We then employ a two-stage framework, named the Optimised Mitoses Generator Network (OMG-Net), to identify mitotic figures. This framework first deploys the Segment Anything Model to automatically outline cells, followed by an adapted ResNet18 that distinguishes mitotic figures. OMG-Net achieves an F1 score of 0.84 in detecting pan-cancer mitotic figures, including human breast carcinoma, neuroendocrine tumours, and melanoma. It outperforms previous state-of-the-art models in hold-out test sets. To summarise, our study introduces a generalisable data creation and curation pipeline and a high-performance detection model, which can largely contribute to the field of computer-aided mitotic figure detection. |
| format | Article |
| id | doaj-art-862ddc50bb82412983a0b201e5558c21 |
| institution | DOAJ |
| issn | 2399-3642 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Biology |
| spelling | doaj-art-862ddc50bb82412983a0b201e5558c212025-08-20T02:39:41ZengNature PortfolioCommunications Biology2399-36422024-12-017111110.1038/s42003-024-07398-6A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slidesZhuoyan Shen0Mikaël Simard1Douglas Brand2Vanghelita Andrei3Ali Al-Khader4Fatine Oumlil5Katherine Trevers6Thomas Butters7Simon Haefliger8Eleanna Kara9Fernanda Amary10Roberto Tirabosco11Paul Cool12Gary Royle13Maria A. Hawkins14Adrienne M. Flanagan15Charles-Antoine Collins-Fekete16Department of Medical Physics and Biomedical Engineering, University College LondonDepartment of Medical Physics and Biomedical Engineering, University College LondonDepartment of Medical Physics and Biomedical Engineering, University College LondonResearch Department of Pathology, University College London Cancer InstituteResearch Department of Pathology, University College London Cancer InstituteCellular and Molecular Pathology, Royal National Orthopaedic Hospital NHS Foundation TrustResearch Department of Pathology, University College London Cancer InstituteResearch Department of Pathology, University College London Cancer InstituteResearch Department of Pathology, University College London Cancer InstituteDepartment of Neurology, Rutgers Biomedical and Health SciencesResearch Department of Pathology, University College London Cancer InstituteResearch Department of Pathology, University College London Cancer InstituteDepartment of Orthopaedics, The Robert Jones and Agnes Hunt Orthopaedic HospitalDepartment of Medical Physics and Biomedical Engineering, University College LondonDepartment of Medical Physics and Biomedical Engineering, University College LondonResearch Department of Pathology, University College London Cancer InstituteDepartment of Medical Physics and Biomedical Engineering, University College LondonAbstract Mitotic activity is an important feature for grading several cancer types. However, counting mitotic figures (cells in division) is a time-consuming and laborious task prone to inter-observer variation. Inaccurate recognition of MFs can lead to incorrect grading and hence potential suboptimal treatment. This study presents an artificial intelligence-based approach to detect mitotic figures in digitised whole-slide images stained with haematoxylin and eosin. Advances in this area are hampered by the small size and variety of datasets available. To address this, we create the largest dataset of mitotic figures (N = 74,620), combining an in-house dataset of soft tissue tumours with five open-source datasets. We then employ a two-stage framework, named the Optimised Mitoses Generator Network (OMG-Net), to identify mitotic figures. This framework first deploys the Segment Anything Model to automatically outline cells, followed by an adapted ResNet18 that distinguishes mitotic figures. OMG-Net achieves an F1 score of 0.84 in detecting pan-cancer mitotic figures, including human breast carcinoma, neuroendocrine tumours, and melanoma. It outperforms previous state-of-the-art models in hold-out test sets. To summarise, our study introduces a generalisable data creation and curation pipeline and a high-performance detection model, which can largely contribute to the field of computer-aided mitotic figure detection.https://doi.org/10.1038/s42003-024-07398-6 |
| spellingShingle | Zhuoyan Shen Mikaël Simard Douglas Brand Vanghelita Andrei Ali Al-Khader Fatine Oumlil Katherine Trevers Thomas Butters Simon Haefliger Eleanna Kara Fernanda Amary Roberto Tirabosco Paul Cool Gary Royle Maria A. Hawkins Adrienne M. Flanagan Charles-Antoine Collins-Fekete A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slides Communications Biology |
| title | A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slides |
| title_full | A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slides |
| title_fullStr | A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slides |
| title_full_unstemmed | A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slides |
| title_short | A deep learning framework deploying segment anything to detect pan-cancer mitotic figures from haematoxylin and eosin-stained slides |
| title_sort | deep learning framework deploying segment anything to detect pan cancer mitotic figures from haematoxylin and eosin stained slides |
| url | https://doi.org/10.1038/s42003-024-07398-6 |
| work_keys_str_mv | AT zhuoyanshen adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT mikaelsimard adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT douglasbrand adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT vanghelitaandrei adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT alialkhader adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT fatineoumlil adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT katherinetrevers adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT thomasbutters adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT simonhaefliger adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT eleannakara adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT fernandaamary adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT robertotirabosco adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT paulcool adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT garyroyle adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT mariaahawkins adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT adriennemflanagan adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT charlesantoinecollinsfekete adeeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT zhuoyanshen deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT mikaelsimard deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT douglasbrand deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT vanghelitaandrei deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT alialkhader deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT fatineoumlil deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT katherinetrevers deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT thomasbutters deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT simonhaefliger deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT eleannakara deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT fernandaamary deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT robertotirabosco deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT paulcool deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT garyroyle deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT mariaahawkins deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT adriennemflanagan deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides AT charlesantoinecollinsfekete deeplearningframeworkdeployingsegmentanythingtodetectpancancermitoticfiguresfromhaematoxylinandeosinstainedslides |