Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstruction
Abstract Scanner-related changes in data quality are common in medical imaging, yet monitoring their impact on diagnostic AI performance remains challenging. In this study, we performed standardized consistency testing of an FDA-cleared and CE-marked AI for triage and notification of intracranial an...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-06-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-04830-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849434313408380928 |
|---|---|
| author | Leonie Goelz Angelo Laudani Ulrich Genske Michael Scheel Georg Bohner Hans-Christian Bauknecht Sven Mutze Bernd Hamm Paul Jahnke |
| author_facet | Leonie Goelz Angelo Laudani Ulrich Genske Michael Scheel Georg Bohner Hans-Christian Bauknecht Sven Mutze Bernd Hamm Paul Jahnke |
| author_sort | Leonie Goelz |
| collection | DOAJ |
| description | Abstract Scanner-related changes in data quality are common in medical imaging, yet monitoring their impact on diagnostic AI performance remains challenging. In this study, we performed standardized consistency testing of an FDA-cleared and CE-marked AI for triage and notification of intracranial aneurysms across changes in image data quality caused by dose and image reconstruction. Our assessment was based on repeated examinations of a head CT phantom designed for AI evaluation, replicating a patient with three intracranial aneurysms in the anterior, middle and posterior circulation. We show that the AI maintains stable performance within the medium dose range but produces inconsistent results at reduced dose and, unexpectedly, at higher dose when filtered back projection is used. Data quality standards required for AI are stricter than those for neuroradiologists, who report higher aneurysm visibility rates and experience performance degradation only at substantially lower doses, with no decline at higher doses. |
| format | Article |
| id | doaj-art-03794751fcc64635a39da34c5f85cebb |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-03794751fcc64635a39da34c5f85cebb2025-08-20T03:26:42ZengNature PortfolioScientific Reports2045-23222025-06-011511910.1038/s41598-025-04830-7Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstructionLeonie Goelz0Angelo Laudani1Ulrich Genske2Michael Scheel3Georg Bohner4Hans-Christian Bauknecht5Sven Mutze6Bernd Hamm7Paul Jahnke8Department of Radiology and Neuroradiology, Unfallkrankenhaus BerlinDepartment of Radiology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of HealthDepartment of Radiology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of HealthDepartment of Neuroradiology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of HealthDepartment of Neuroradiology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of HealthDepartment of Neuroradiology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of HealthDepartment of Radiology and Neuroradiology, Unfallkrankenhaus BerlinDepartment of Radiology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of HealthDepartment of Radiology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of HealthAbstract Scanner-related changes in data quality are common in medical imaging, yet monitoring their impact on diagnostic AI performance remains challenging. In this study, we performed standardized consistency testing of an FDA-cleared and CE-marked AI for triage and notification of intracranial aneurysms across changes in image data quality caused by dose and image reconstruction. Our assessment was based on repeated examinations of a head CT phantom designed for AI evaluation, replicating a patient with three intracranial aneurysms in the anterior, middle and posterior circulation. We show that the AI maintains stable performance within the medium dose range but produces inconsistent results at reduced dose and, unexpectedly, at higher dose when filtered back projection is used. Data quality standards required for AI are stricter than those for neuroradiologists, who report higher aneurysm visibility rates and experience performance degradation only at substantially lower doses, with no decline at higher doses.https://doi.org/10.1038/s41598-025-04830-7Intracranial aneurysmArtificial intelligenceComputed tomography angiographyReproducibility of findingsImaging phantoms |
| spellingShingle | Leonie Goelz Angelo Laudani Ulrich Genske Michael Scheel Georg Bohner Hans-Christian Bauknecht Sven Mutze Bernd Hamm Paul Jahnke Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstruction Scientific Reports Intracranial aneurysm Artificial intelligence Computed tomography angiography Reproducibility of findings Imaging phantoms |
| title | Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstruction |
| title_full | Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstruction |
| title_fullStr | Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstruction |
| title_full_unstemmed | Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstruction |
| title_short | Inconsistency of AI in intracranial aneurysm detection with varying dose and image reconstruction |
| title_sort | inconsistency of ai in intracranial aneurysm detection with varying dose and image reconstruction |
| topic | Intracranial aneurysm Artificial intelligence Computed tomography angiography Reproducibility of findings Imaging phantoms |
| url | https://doi.org/10.1038/s41598-025-04830-7 |
| work_keys_str_mv | AT leoniegoelz inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT angelolaudani inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT ulrichgenske inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT michaelscheel inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT georgbohner inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT hanschristianbauknecht inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT svenmutze inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT berndhamm inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction AT pauljahnke inconsistencyofaiinintracranialaneurysmdetectionwithvaryingdoseandimagereconstruction |