AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images

The acurate segmentation and classification of nuclei in histological images are crucial for the diagnosis and treatment of colorectal cancer. However, the aggregation of nuclei and intra-class variability in histology images present significant challenges for nuclei segmentation and classification....

Full description

Saved in:
Bibliographic Details
Main Authors: Ruifen Cao, Qingbin Meng, Dayu Tan, Pijing Wei, Yun Ding, Chunhou Zheng
Format: Article
Language:English
Published: MDPI AG 2024-11-01
Series:Sensors
Subjects:
Online Access:https://www.mdpi.com/1424-8220/24/22/7208
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1850267086762278912
author Ruifen Cao
Qingbin Meng
Dayu Tan
Pijing Wei
Yun Ding
Chunhou Zheng
author_facet Ruifen Cao
Qingbin Meng
Dayu Tan
Pijing Wei
Yun Ding
Chunhou Zheng
author_sort Ruifen Cao
collection DOAJ
description The acurate segmentation and classification of nuclei in histological images are crucial for the diagnosis and treatment of colorectal cancer. However, the aggregation of nuclei and intra-class variability in histology images present significant challenges for nuclei segmentation and classification. In addition, the imbalance of various nuclei classes exacerbates the difficulty of nuclei classification and segmentation using deep learning models. To address these challenges, we present a novel attention-enhanced residual refinement network (AER-Net), which consists of one encoder and three decoder branches that have same network structure. In addition to the nuclei instance segmentation branch and nuclei classification branch, one branch is used to predict the vertical and horizontal distance from each pixel to its nuclear center, which is combined with output by the segmentation branch to improve the final segmentation results. The AER-Net utilizes an attention-enhanced encoder module to focus on more valuable features. To further refine predictions and achieve more accurate results, an attention-enhancing residual refinement module is employed at the end of each encoder branch. Moreover, the coarse predictions and refined predictions are combined by using a loss function that employs cross-entropy loss and generalized dice loss to efficiently tackle the challenge of class imbalance among nuclei in histology images. Compared with other state-of-the-art methods on two colorectal cancer datasets and a pan-cancer dataset, AER-Net demonstrates outstanding performance, validating its effectiveness in nuclear segmentation and classification.
format Article
id doaj-art-26d9080d232242eb9729fab76e04efb0
institution OA Journals
issn 1424-8220
language English
publishDate 2024-11-01
publisher MDPI AG
record_format Article
series Sensors
spelling doaj-art-26d9080d232242eb9729fab76e04efb02025-08-20T01:53:56ZengMDPI AGSensors1424-82202024-11-012422720810.3390/s24227208AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology ImagesRuifen Cao0Qingbin Meng1Dayu Tan2Pijing Wei3Yun Ding4Chunhou Zheng5The Information Materials and Intelligent Sensing Laboratory of Anhui Province, School of Computer Science and Technology, Anhui University, Hefei 230601, ChinaThe Information Materials and Intelligent Sensing Laboratory of Anhui Province, School of Computer Science and Technology, Anhui University, Hefei 230601, ChinaInstitutes of Physical Science and Information Technology, Anhui University, Hefei 230601, ChinaInstitutes of Physical Science and Information Technology, Anhui University, Hefei 230601, ChinaSchool of Artificial Intelligence, Anhui University, Hefei 230601, ChinaSchool of Artificial Intelligence, Anhui University, Hefei 230601, ChinaThe acurate segmentation and classification of nuclei in histological images are crucial for the diagnosis and treatment of colorectal cancer. However, the aggregation of nuclei and intra-class variability in histology images present significant challenges for nuclei segmentation and classification. In addition, the imbalance of various nuclei classes exacerbates the difficulty of nuclei classification and segmentation using deep learning models. To address these challenges, we present a novel attention-enhanced residual refinement network (AER-Net), which consists of one encoder and three decoder branches that have same network structure. In addition to the nuclei instance segmentation branch and nuclei classification branch, one branch is used to predict the vertical and horizontal distance from each pixel to its nuclear center, which is combined with output by the segmentation branch to improve the final segmentation results. The AER-Net utilizes an attention-enhanced encoder module to focus on more valuable features. To further refine predictions and achieve more accurate results, an attention-enhancing residual refinement module is employed at the end of each encoder branch. Moreover, the coarse predictions and refined predictions are combined by using a loss function that employs cross-entropy loss and generalized dice loss to efficiently tackle the challenge of class imbalance among nuclei in histology images. Compared with other state-of-the-art methods on two colorectal cancer datasets and a pan-cancer dataset, AER-Net demonstrates outstanding performance, validating its effectiveness in nuclear segmentation and classification.https://www.mdpi.com/1424-8220/24/22/7208histology imagesdeep learningnuclei segmentationnuclei classification
spellingShingle Ruifen Cao
Qingbin Meng
Dayu Tan
Pijing Wei
Yun Ding
Chunhou Zheng
AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images
Sensors
histology images
deep learning
nuclei segmentation
nuclei classification
title AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images
title_full AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images
title_fullStr AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images
title_full_unstemmed AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images
title_short AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images
title_sort aer net attention enhanced residual refinement network for nuclei segmentation and classification in histology images
topic histology images
deep learning
nuclei segmentation
nuclei classification
url https://www.mdpi.com/1424-8220/24/22/7208
work_keys_str_mv AT ruifencao aernetattentionenhancedresidualrefinementnetworkfornucleisegmentationandclassificationinhistologyimages
AT qingbinmeng aernetattentionenhancedresidualrefinementnetworkfornucleisegmentationandclassificationinhistologyimages
AT dayutan aernetattentionenhancedresidualrefinementnetworkfornucleisegmentationandclassificationinhistologyimages
AT pijingwei aernetattentionenhancedresidualrefinementnetworkfornucleisegmentationandclassificationinhistologyimages
AT yunding aernetattentionenhancedresidualrefinementnetworkfornucleisegmentationandclassificationinhistologyimages
AT chunhouzheng aernetattentionenhancedresidualrefinementnetworkfornucleisegmentationandclassificationinhistologyimages