Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance Indicators
BackgroundImmersive virtual reality (iVR) has emerged as a training method to prepare medical first responders (MFRs) for mass casualty incidents (MCIs) and disasters in a resource-efficient, flexible, and safe manner. However, systematic evaluations and validations of potent...
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-01-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e63241 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832584021453832192 |
|---|---|
| author | Anke Sabine Baetzner Yannick Hill Benjamin Roszipal Solène Gerwann Matthias Beutel Tanja Birrenbach Markus Karlseder Stefan Mohr Gabriel Alexander Salg Helmut Schrom-Feiertag Marie Ottilie Frenkel Cornelia Wrzus |
| author_facet | Anke Sabine Baetzner Yannick Hill Benjamin Roszipal Solène Gerwann Matthias Beutel Tanja Birrenbach Markus Karlseder Stefan Mohr Gabriel Alexander Salg Helmut Schrom-Feiertag Marie Ottilie Frenkel Cornelia Wrzus |
| author_sort | Anke Sabine Baetzner |
| collection | DOAJ |
| description |
BackgroundImmersive virtual reality (iVR) has emerged as a training method to prepare medical first responders (MFRs) for mass casualty incidents (MCIs) and disasters in a resource-efficient, flexible, and safe manner. However, systematic evaluations and validations of potential performance indicators for virtual MCI training are still lacking.
ObjectiveThis study aimed to investigate whether different performance indicators based on visual attention, triage performance, and information transmission can be effectively extended to MCI training in iVR by testing if they can discriminate between different levels of expertise. Furthermore, the study examined the extent to which such objective indicators correlate with subjective performance assessments.
MethodsA total of 76 participants (mean age 25.54, SD 6.01 y; 45/76, 59% male) with different medical expertise (MFRs: paramedics and emergency physicians; non-MFRs: medical students, in-hospital nurses, and other physicians) participated in 5 virtual MCI scenarios of varying complexity in a randomized order. Tasks involved assessing the situation, triaging virtual patients, and transmitting relevant information to a control center. Performance indicators included eye-tracking–based visual attention, triage accuracy, triage speed, information transmission efficiency, and self-assessment of performance. Expertise was determined based on the occupational group (39/76, 51% MFRs vs 37/76, 49% non-MFRs) and a knowledge test with patient vignettes.
ResultsTriage accuracy (d=0.48), triage speed (d=0.42), and information transmission efficiency (d=1.13) differentiated significantly between MFRs and non-MFRs. In addition, higher triage accuracy was significantly associated with higher triage knowledge test scores (Spearman ρ=0.40). Visual attention was not significantly associated with expertise. Furthermore, subjective performance was not correlated with any other performance indicator.
ConclusionsiVR-based MCI scenarios proved to be a valuable tool for assessing the performance of MFRs. The results suggest that iVR could be integrated into current MCI training curricula to provide frequent, objective, and potentially (partly) automated performance assessments in a controlled environment. In particular, performance indicators, such as triage accuracy, triage speed, and information transmission efficiency, capture multiple aspects of performance and are recommended for integration. While the examined visual attention indicators did not function as valid performance indicators in this study, future research could further explore visual attention in MCI training and examine other indicators, such as holistic gaze patterns. Overall, the results underscore the importance of integrating objective indicators to enhance trainers’ feedback and provide trainees with guidance on evaluating and reflecting on their own performance. |
| format | Article |
| id | doaj-art-acf9639787b24bf1954a72df2646a045 |
| institution | Kabale University |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-acf9639787b24bf1954a72df2646a0452025-01-27T21:30:52ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-01-0127e6324110.2196/63241Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance IndicatorsAnke Sabine Baetznerhttps://orcid.org/0000-0001-5366-8184Yannick Hillhttps://orcid.org/0000-0002-4382-2149Benjamin Roszipalhttps://orcid.org/0000-0003-4276-9507Solène Gerwannhttps://orcid.org/0009-0004-5473-5982Matthias Beutelhttps://orcid.org/0009-0000-7013-6868Tanja Birrenbachhttps://orcid.org/0000-0002-3046-0900Markus Karlsederhttps://orcid.org/0000-0003-2773-0727Stefan Mohrhttps://orcid.org/0000-0003-4292-663XGabriel Alexander Salghttps://orcid.org/0000-0002-3964-3527Helmut Schrom-Feiertaghttps://orcid.org/0000-0002-5327-2494Marie Ottilie Frenkelhttps://orcid.org/0000-0003-0646-2612Cornelia Wrzushttps://orcid.org/0000-0002-6290-959X BackgroundImmersive virtual reality (iVR) has emerged as a training method to prepare medical first responders (MFRs) for mass casualty incidents (MCIs) and disasters in a resource-efficient, flexible, and safe manner. However, systematic evaluations and validations of potential performance indicators for virtual MCI training are still lacking. ObjectiveThis study aimed to investigate whether different performance indicators based on visual attention, triage performance, and information transmission can be effectively extended to MCI training in iVR by testing if they can discriminate between different levels of expertise. Furthermore, the study examined the extent to which such objective indicators correlate with subjective performance assessments. MethodsA total of 76 participants (mean age 25.54, SD 6.01 y; 45/76, 59% male) with different medical expertise (MFRs: paramedics and emergency physicians; non-MFRs: medical students, in-hospital nurses, and other physicians) participated in 5 virtual MCI scenarios of varying complexity in a randomized order. Tasks involved assessing the situation, triaging virtual patients, and transmitting relevant information to a control center. Performance indicators included eye-tracking–based visual attention, triage accuracy, triage speed, information transmission efficiency, and self-assessment of performance. Expertise was determined based on the occupational group (39/76, 51% MFRs vs 37/76, 49% non-MFRs) and a knowledge test with patient vignettes. ResultsTriage accuracy (d=0.48), triage speed (d=0.42), and information transmission efficiency (d=1.13) differentiated significantly between MFRs and non-MFRs. In addition, higher triage accuracy was significantly associated with higher triage knowledge test scores (Spearman ρ=0.40). Visual attention was not significantly associated with expertise. Furthermore, subjective performance was not correlated with any other performance indicator. ConclusionsiVR-based MCI scenarios proved to be a valuable tool for assessing the performance of MFRs. The results suggest that iVR could be integrated into current MCI training curricula to provide frequent, objective, and potentially (partly) automated performance assessments in a controlled environment. In particular, performance indicators, such as triage accuracy, triage speed, and information transmission efficiency, capture multiple aspects of performance and are recommended for integration. While the examined visual attention indicators did not function as valid performance indicators in this study, future research could further explore visual attention in MCI training and examine other indicators, such as holistic gaze patterns. Overall, the results underscore the importance of integrating objective indicators to enhance trainers’ feedback and provide trainees with guidance on evaluating and reflecting on their own performance.https://www.jmir.org/2025/1/e63241 |
| spellingShingle | Anke Sabine Baetzner Yannick Hill Benjamin Roszipal Solène Gerwann Matthias Beutel Tanja Birrenbach Markus Karlseder Stefan Mohr Gabriel Alexander Salg Helmut Schrom-Feiertag Marie Ottilie Frenkel Cornelia Wrzus Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance Indicators Journal of Medical Internet Research |
| title | Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance Indicators |
| title_full | Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance Indicators |
| title_fullStr | Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance Indicators |
| title_full_unstemmed | Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance Indicators |
| title_short | Mass Casualty Incident Training in Immersive Virtual Reality: Quasi-Experimental Evaluation of Multimethod Performance Indicators |
| title_sort | mass casualty incident training in immersive virtual reality quasi experimental evaluation of multimethod performance indicators |
| url | https://www.jmir.org/2025/1/e63241 |
| work_keys_str_mv | AT ankesabinebaetzner masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT yannickhill masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT benjaminroszipal masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT solenegerwann masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT matthiasbeutel masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT tanjabirrenbach masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT markuskarlseder masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT stefanmohr masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT gabrielalexandersalg masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT helmutschromfeiertag masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT marieottiliefrenkel masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators AT corneliawrzus masscasualtyincidenttraininginimmersivevirtualrealityquasiexperimentalevaluationofmultimethodperformanceindicators |