Gaze Assistance for Efficient Segmentation Correction of Medical Images
The segmentation of medical images is an important step in various diagnostic applications, including abnormality detection and radiotherapy planning. Recent developments in Artificial Intelligence (AI) have significantly advanced the field of segmentation automation. However, expert-level accuracy...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10843670/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832586896133324800 |
|---|---|
| author | Leila Khaertdinova Tatyana Shmykova Ilya Pershin Andrey Laryukov Albert Khanov Damir Zidikhanov Bulat Ibragimov |
| author_facet | Leila Khaertdinova Tatyana Shmykova Ilya Pershin Andrey Laryukov Albert Khanov Damir Zidikhanov Bulat Ibragimov |
| author_sort | Leila Khaertdinova |
| collection | DOAJ |
| description | The segmentation of medical images is an important step in various diagnostic applications, including abnormality detection and radiotherapy planning. Recent developments in Artificial Intelligence (AI) have significantly advanced the field of segmentation automation. However, expert-level accuracy has not been achieved for most segmentation tasks, which significantly hampers the adoption of fully-automated medical image segmentation. This paper investigates the idea of efficient correction of medical image segmentation by using not manual controller commands, which can be time-consuming, but gaze movements. We propose a lightweight fine-tuning approach of the Segment Anything Model in medical images, known as MedSAM, to interactively adjust segmentation masks based on gaze point prompts. Our model is specifically trained for the abdominal CT imaging task using the publicly available WORD database. While surpassing state-of-the-art segmentation models, comprehensive studies with medical experts demonstrated that our gaze-assisted interactive approach led to significant improvements in segmentation quality. Specifically, the gaze-assisted corrections increased the average segmentation performance by nearly 62% for difficult medical cases, compared to traditional segmentation methods based on bounding boxes. The main findings of our proposed work include: 1) the substantial improvement in segmentation quality through gaze interaction, 2) the development of an efficient correction mechanism leveraging eye movements, and 3) the demonstration of gaze-assisted segmentation’s superior performance in abdominal imaging tasks. Our innovative approach shows promise for interactive segmentation of medical images and opens the door for further advancements in human-AI interaction in medicine using eye-tracking technology. |
| format | Article |
| id | doaj-art-e4afa81457294ceb80379e24dac75338 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e4afa81457294ceb80379e24dac753382025-01-25T00:02:07ZengIEEEIEEE Access2169-35362025-01-0113141991421310.1109/ACCESS.2025.353070110843670Gaze Assistance for Efficient Segmentation Correction of Medical ImagesLeila Khaertdinova0https://orcid.org/0009-0009-0663-3312Tatyana Shmykova1https://orcid.org/0009-0004-7961-6808Ilya Pershin2https://orcid.org/0000-0001-7052-2355Andrey Laryukov3Albert Khanov4Damir Zidikhanov5Bulat Ibragimov6https://orcid.org/0000-0001-7739-7788Research Center of the Artificial Intelligence Institute, Innopolis University, Innopolis, RussiaResearch Center of the Artificial Intelligence Institute, Innopolis University, Innopolis, RussiaResearch Center of the Artificial Intelligence Institute, Innopolis University, Innopolis, RussiaKazan Federal University, Kazan, RussiaKazan Federal University, Kazan, RussiaKazan State Medical Academy, Kazan, RussiaUniversity of Copenhagen, Copenhagen, DenmarkThe segmentation of medical images is an important step in various diagnostic applications, including abnormality detection and radiotherapy planning. Recent developments in Artificial Intelligence (AI) have significantly advanced the field of segmentation automation. However, expert-level accuracy has not been achieved for most segmentation tasks, which significantly hampers the adoption of fully-automated medical image segmentation. This paper investigates the idea of efficient correction of medical image segmentation by using not manual controller commands, which can be time-consuming, but gaze movements. We propose a lightweight fine-tuning approach of the Segment Anything Model in medical images, known as MedSAM, to interactively adjust segmentation masks based on gaze point prompts. Our model is specifically trained for the abdominal CT imaging task using the publicly available WORD database. While surpassing state-of-the-art segmentation models, comprehensive studies with medical experts demonstrated that our gaze-assisted interactive approach led to significant improvements in segmentation quality. Specifically, the gaze-assisted corrections increased the average segmentation performance by nearly 62% for difficult medical cases, compared to traditional segmentation methods based on bounding boxes. The main findings of our proposed work include: 1) the substantial improvement in segmentation quality through gaze interaction, 2) the development of an efficient correction mechanism leveraging eye movements, and 3) the demonstration of gaze-assisted segmentation’s superior performance in abdominal imaging tasks. Our innovative approach shows promise for interactive segmentation of medical images and opens the door for further advancements in human-AI interaction in medicine using eye-tracking technology.https://ieeexplore.ieee.org/document/10843670/Eye gazeeye trackinginteractive segmentationmedical image segmentationsegment anything2D segmentation |
| spellingShingle | Leila Khaertdinova Tatyana Shmykova Ilya Pershin Andrey Laryukov Albert Khanov Damir Zidikhanov Bulat Ibragimov Gaze Assistance for Efficient Segmentation Correction of Medical Images IEEE Access Eye gaze eye tracking interactive segmentation medical image segmentation segment anything 2D segmentation |
| title | Gaze Assistance for Efficient Segmentation Correction of Medical Images |
| title_full | Gaze Assistance for Efficient Segmentation Correction of Medical Images |
| title_fullStr | Gaze Assistance for Efficient Segmentation Correction of Medical Images |
| title_full_unstemmed | Gaze Assistance for Efficient Segmentation Correction of Medical Images |
| title_short | Gaze Assistance for Efficient Segmentation Correction of Medical Images |
| title_sort | gaze assistance for efficient segmentation correction of medical images |
| topic | Eye gaze eye tracking interactive segmentation medical image segmentation segment anything 2D segmentation |
| url | https://ieeexplore.ieee.org/document/10843670/ |
| work_keys_str_mv | AT leilakhaertdinova gazeassistanceforefficientsegmentationcorrectionofmedicalimages AT tatyanashmykova gazeassistanceforefficientsegmentationcorrectionofmedicalimages AT ilyapershin gazeassistanceforefficientsegmentationcorrectionofmedicalimages AT andreylaryukov gazeassistanceforefficientsegmentationcorrectionofmedicalimages AT albertkhanov gazeassistanceforefficientsegmentationcorrectionofmedicalimages AT damirzidikhanov gazeassistanceforefficientsegmentationcorrectionofmedicalimages AT bulatibragimov gazeassistanceforefficientsegmentationcorrectionofmedicalimages |