BrainAR: Automated Brain Tumor Diagnosis With Deep Learning and 3D Augmented Reality Visualization

BrainAR: Automated Brain Tumor Diagnosis With Deep Learning and 3D Augmented Reality Visualization

Augmented Reality (AR) technology offers promising applications in healthcare by enabling interactive 3D visualization of anatomical structures. However, current AR implementations often lack patient-specific detail, limiting their effectiveness in clinical settings. In this paper, we present BrainA...

Full description

Saved in:
Bibliographic Details
Main Authors: Meriem Khedir, Kahina Amara, Nassima Dif, Oussama Kerdjidj, Shadi Atalla, Naeem Ramzan
Format: Article
Language:English
Published: IEEE 2025-01-01
Series:IEEE Access
Subjects:
Brain tumor segmentation
computer-aided diagnosis
patient-specific 3D visualization
tumor localization
interaction
3D U-Net
Online Access:https://ieeexplore.ieee.org/document/11083598/
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Augmented Reality (AR) technology offers promising applications in healthcare by enabling interactive 3D visualization of anatomical structures. However, current AR implementations often lack patient-specific detail, limiting their effectiveness in clinical settings. In this paper, we present BrainAR, an innovative mobile AR-based application designed for the automatic segmentation, 3D visualization, localization, and interaction with brain tumors using multiparametric 3D Magnetic Resonance Imaging (MRI) data. Our method leverages a 3D Residual U-Net, trained on the BraTS2021 dataset, achieving a mean Dice score of 0.886 for accurate tumor segmentation. The segmentation outputs are integrated into a real-time 3D engine to enable precise and dynamic visualization of brain tumors. Key contributions of our work include: 1) a server-side deployment of the segmentation model for online, patient-specific inference; 2) seamless AR integration enabling interactive exploration through hand gestures and voice commands; and 3) a mobile-based platform aimed at enhancing accessibility and usability in clinical environments. The proposed solution facilitates early detection and diagnosis by providing clinicians with an intuitive, immersive, and patient-specific tool for enhanced medical imaging interaction.
ISSN:2169-3536

Similar Items