Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of Interest

<b>Background:</b> Breast cancer remains a critical health concern worldwide, with histopathological analysis of tissue biopsies serving as the clinical gold standard for diagnosis. Manual evaluation of histopathology images is time-intensive and requires specialized expertise, often res...

Full description

Saved in:
Bibliographic Details
Main Authors: Zaka Ur Rehman, Mohammad Faizal Ahmad Fauzi, Wan Siti Halimatul Munirah Wan Ahmad, Fazly Salleh Abas, Phaik-Leng Cheah, Seow-Fan Chiew, Lai-Meng Looi
Format: Article
Language:English
Published: MDPI AG 2025-06-01
Series:Diagnostics
Subjects:
Online Access:https://www.mdpi.com/2075-4418/15/13/1584
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1850118520026693632
author Zaka Ur Rehman
Mohammad Faizal Ahmad Fauzi
Wan Siti Halimatul Munirah Wan Ahmad
Fazly Salleh Abas
Phaik-Leng Cheah
Seow-Fan Chiew
Lai-Meng Looi
author_facet Zaka Ur Rehman
Mohammad Faizal Ahmad Fauzi
Wan Siti Halimatul Munirah Wan Ahmad
Fazly Salleh Abas
Phaik-Leng Cheah
Seow-Fan Chiew
Lai-Meng Looi
author_sort Zaka Ur Rehman
collection DOAJ
description <b>Background:</b> Breast cancer remains a critical health concern worldwide, with histopathological analysis of tissue biopsies serving as the clinical gold standard for diagnosis. Manual evaluation of histopathology images is time-intensive and requires specialized expertise, often resulting in variability in diagnostic outcomes. In silver in situ hybridization (SISH) images, accurate nuclei detection is essential for precise histo-scoring of HER2 gene expression, directly impacting treatment decisions. <b>Methods:</b> This study presents a scalable and automated deep learning framework for nuclei detection in HER2-SISH whole slide images (WSIs), utilizing a novel dataset of 100 expert-marked regions extracted from 20 WSIs collected at the University of Malaya Medical Center (UMMC). The proposed two-stage approach combines a pretrained Stardist model with image processing-based annotations, followed by fine tuning on our domain-specific dataset to improve generalization. <b>Results:</b> The fine-tuned model achieved substantial improvements over both the pretrained Stardist model and a conventional watershed segmentation baseline. Quantitatively, the proposed method attained an average F1-score of 98.1% for visual assessments and 97.4% for expert-marked nuclei, outperforming baseline methods across all metrics. Additionally, training and validation performance curves demonstrate stable model convergence over 100 epochs. <b>Conclusions:</b> These results highlight the robustness of our approach in handling the complex morphological characteristics of SISH-stained nuclei. Our framework supports pathologists by offering reliable, automated nuclei detection in HER2 scoring workflows, contributing to diagnostic consistency and efficiency in clinical pathology.
format Article
id doaj-art-deef765c2459408488daa141fcac3beb
institution OA Journals
issn 2075-4418
language English
publishDate 2025-06-01
publisher MDPI AG
record_format Article
series Diagnostics
spelling doaj-art-deef765c2459408488daa141fcac3beb2025-08-20T02:35:51ZengMDPI AGDiagnostics2075-44182025-06-011513158410.3390/diagnostics15131584Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of InterestZaka Ur Rehman0Mohammad Faizal Ahmad Fauzi1Wan Siti Halimatul Munirah Wan Ahmad2Fazly Salleh Abas3Phaik-Leng Cheah4Seow-Fan Chiew5Lai-Meng Looi6Centre for Image and Vision Computing, CoE for Artificial Intelligence, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, MalaysiaCentre for Image and Vision Computing, CoE for Artificial Intelligence, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, MalaysiaCentre for Image and Vision Computing, CoE for Artificial Intelligence, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, MalaysiaCentre for Image and Vision Computing, CoE for Artificial Intelligence, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, MalaysiaDepartment of Pathology, University Malaya-Medical Center, Kuala Lumpur 59100, MalaysiaDepartment of Pathology, University Malaya-Medical Center, Kuala Lumpur 59100, MalaysiaDepartment of Pathology, University Malaya-Medical Center, Kuala Lumpur 59100, Malaysia<b>Background:</b> Breast cancer remains a critical health concern worldwide, with histopathological analysis of tissue biopsies serving as the clinical gold standard for diagnosis. Manual evaluation of histopathology images is time-intensive and requires specialized expertise, often resulting in variability in diagnostic outcomes. In silver in situ hybridization (SISH) images, accurate nuclei detection is essential for precise histo-scoring of HER2 gene expression, directly impacting treatment decisions. <b>Methods:</b> This study presents a scalable and automated deep learning framework for nuclei detection in HER2-SISH whole slide images (WSIs), utilizing a novel dataset of 100 expert-marked regions extracted from 20 WSIs collected at the University of Malaya Medical Center (UMMC). The proposed two-stage approach combines a pretrained Stardist model with image processing-based annotations, followed by fine tuning on our domain-specific dataset to improve generalization. <b>Results:</b> The fine-tuned model achieved substantial improvements over both the pretrained Stardist model and a conventional watershed segmentation baseline. Quantitatively, the proposed method attained an average F1-score of 98.1% for visual assessments and 97.4% for expert-marked nuclei, outperforming baseline methods across all metrics. Additionally, training and validation performance curves demonstrate stable model convergence over 100 epochs. <b>Conclusions:</b> These results highlight the robustness of our approach in handling the complex morphological characteristics of SISH-stained nuclei. Our framework supports pathologists by offering reliable, automated nuclei detection in HER2 scoring workflows, contributing to diagnostic consistency and efficiency in clinical pathology.https://www.mdpi.com/2075-4418/15/13/1584deep learningdigital pathologyhuman epidermal growth factor receptor 2 (HER2)silver-enhanced in situ hybridization (SISH)
spellingShingle Zaka Ur Rehman
Mohammad Faizal Ahmad Fauzi
Wan Siti Halimatul Munirah Wan Ahmad
Fazly Salleh Abas
Phaik-Leng Cheah
Seow-Fan Chiew
Lai-Meng Looi
Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of Interest
Diagnostics
deep learning
digital pathology
human epidermal growth factor receptor 2 (HER2)
silver-enhanced in situ hybridization (SISH)
title Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of Interest
title_full Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of Interest
title_fullStr Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of Interest
title_full_unstemmed Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of Interest
title_short Scalable Nuclei Detection in HER2-SISH Whole Slide Images via Fine-Tuned Stardist with Expert-Annotated Regions of Interest
title_sort scalable nuclei detection in her2 sish whole slide images via fine tuned stardist with expert annotated regions of interest
topic deep learning
digital pathology
human epidermal growth factor receptor 2 (HER2)
silver-enhanced in situ hybridization (SISH)
url https://www.mdpi.com/2075-4418/15/13/1584
work_keys_str_mv AT zakaurrehman scalablenucleidetectioninher2sishwholeslideimagesviafinetunedstardistwithexpertannotatedregionsofinterest
AT mohammadfaizalahmadfauzi scalablenucleidetectioninher2sishwholeslideimagesviafinetunedstardistwithexpertannotatedregionsofinterest
AT wansitihalimatulmunirahwanahmad scalablenucleidetectioninher2sishwholeslideimagesviafinetunedstardistwithexpertannotatedregionsofinterest
AT fazlysallehabas scalablenucleidetectioninher2sishwholeslideimagesviafinetunedstardistwithexpertannotatedregionsofinterest
AT phaiklengcheah scalablenucleidetectioninher2sishwholeslideimagesviafinetunedstardistwithexpertannotatedregionsofinterest
AT seowfanchiew scalablenucleidetectioninher2sishwholeslideimagesviafinetunedstardistwithexpertannotatedregionsofinterest
AT laimenglooi scalablenucleidetectioninher2sishwholeslideimagesviafinetunedstardistwithexpertannotatedregionsofinterest