Three-Blind Validation Strategy of Deep Learning Models for Image Segmentation
Image segmentation plays a central role in computer vision applications such as medical imaging, industrial inspection, and environmental monitoring. However, evaluating segmentation performance can be particularly challenging when ground truth is not clearly defined, as is often the case in tasks i...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-05-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/11/5/170 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849710854104154112 |
|---|---|
| author | Andrés Larroza Francisco Javier Pérez-Benito Raquel Tendero Juan Carlos Perez-Cortes Marta Román Rafael Llobet |
| author_facet | Andrés Larroza Francisco Javier Pérez-Benito Raquel Tendero Juan Carlos Perez-Cortes Marta Román Rafael Llobet |
| author_sort | Andrés Larroza |
| collection | DOAJ |
| description | Image segmentation plays a central role in computer vision applications such as medical imaging, industrial inspection, and environmental monitoring. However, evaluating segmentation performance can be particularly challenging when ground truth is not clearly defined, as is often the case in tasks involving subjective interpretation. These challenges are amplified by inter- and intra-observer variability, which complicates the use of human annotations as a reliable reference. To address this, we propose a novel validation framework—referred to as the three-blind validation strategy—that enables rigorous assessment of segmentation models in contexts where subjectivity and label variability are significant. The core idea is to have a third independent expert, blind to the labeler identities, assess a shuffled set of segmentations produced by multiple human annotators and/or automated models. This allows for the unbiased evaluation of model performance and helps uncover patterns of disagreement that may indicate systematic issues with either human or machine annotations. The primary objective of this study is to introduce and demonstrate this validation strategy as a generalizable framework for robust model evaluation in subjective segmentation tasks. We illustrate its practical implementation in a mammography use case involving dense tissue segmentation while emphasizing its potential applicability to a broad range of segmentation scenarios. |
| format | Article |
| id | doaj-art-b93ee28b0fdb431da26d405b39e2f355 |
| institution | DOAJ |
| issn | 2313-433X |
| language | English |
| publishDate | 2025-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj-art-b93ee28b0fdb431da26d405b39e2f3552025-08-20T03:14:46ZengMDPI AGJournal of Imaging2313-433X2025-05-0111517010.3390/jimaging11050170Three-Blind Validation Strategy of Deep Learning Models for Image SegmentationAndrés Larroza0Francisco Javier Pérez-Benito1Raquel Tendero2Juan Carlos Perez-Cortes3Marta Román4Rafael Llobet5Instituto Tecnológico de la Informática, Universitat Politècnica de València, Camino de Vera, s/n, 46022 València, SpainInstituto Tecnológico de la Informática, Universitat Politècnica de València, Camino de Vera, s/n, 46022 València, SpainInstituto Tecnológico de la Informática, Universitat Politècnica de València, Camino de Vera, s/n, 46022 València, SpainInstituto Tecnológico de la Informática, Universitat Politècnica de València, Camino de Vera, s/n, 46022 València, SpainDepartment of Epidemiology and Evaluation, IMIM (Hospital del Mar Research Institute), Passeig Marítim 25-29, 08003 Barcelona, SpainInstituto Tecnológico de la Informática, Universitat Politècnica de València, Camino de Vera, s/n, 46022 València, SpainImage segmentation plays a central role in computer vision applications such as medical imaging, industrial inspection, and environmental monitoring. However, evaluating segmentation performance can be particularly challenging when ground truth is not clearly defined, as is often the case in tasks involving subjective interpretation. These challenges are amplified by inter- and intra-observer variability, which complicates the use of human annotations as a reliable reference. To address this, we propose a novel validation framework—referred to as the three-blind validation strategy—that enables rigorous assessment of segmentation models in contexts where subjectivity and label variability are significant. The core idea is to have a third independent expert, blind to the labeler identities, assess a shuffled set of segmentations produced by multiple human annotators and/or automated models. This allows for the unbiased evaluation of model performance and helps uncover patterns of disagreement that may indicate systematic issues with either human or machine annotations. The primary objective of this study is to introduce and demonstrate this validation strategy as a generalizable framework for robust model evaluation in subjective segmentation tasks. We illustrate its practical implementation in a mammography use case involving dense tissue segmentation while emphasizing its potential applicability to a broad range of segmentation scenarios.https://www.mdpi.com/2313-433X/11/5/170deep learningimage segmentationmammography |
| spellingShingle | Andrés Larroza Francisco Javier Pérez-Benito Raquel Tendero Juan Carlos Perez-Cortes Marta Román Rafael Llobet Three-Blind Validation Strategy of Deep Learning Models for Image Segmentation Journal of Imaging deep learning image segmentation mammography |
| title | Three-Blind Validation Strategy of Deep Learning Models for Image Segmentation |
| title_full | Three-Blind Validation Strategy of Deep Learning Models for Image Segmentation |
| title_fullStr | Three-Blind Validation Strategy of Deep Learning Models for Image Segmentation |
| title_full_unstemmed | Three-Blind Validation Strategy of Deep Learning Models for Image Segmentation |
| title_short | Three-Blind Validation Strategy of Deep Learning Models for Image Segmentation |
| title_sort | three blind validation strategy of deep learning models for image segmentation |
| topic | deep learning image segmentation mammography |
| url | https://www.mdpi.com/2313-433X/11/5/170 |
| work_keys_str_mv | AT andreslarroza threeblindvalidationstrategyofdeeplearningmodelsforimagesegmentation AT franciscojavierperezbenito threeblindvalidationstrategyofdeeplearningmodelsforimagesegmentation AT raqueltendero threeblindvalidationstrategyofdeeplearningmodelsforimagesegmentation AT juancarlosperezcortes threeblindvalidationstrategyofdeeplearningmodelsforimagesegmentation AT martaroman threeblindvalidationstrategyofdeeplearningmodelsforimagesegmentation AT rafaelllobet threeblindvalidationstrategyofdeeplearningmodelsforimagesegmentation |