Computer vision detects covert voluntary facial movements in unresponsive brain injury patients
Abstract Background Many brain injury patients who appear unresponsive retain subtle, purposeful motor behaviors, signaling capacity for recovery. We hypothesized that low-amplitude movements precede larger-amplitude voluntary movements detectable by clinicians after acute brain injury. To test this...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-08-01
|
| Series: | Communications Medicine |
| Online Access: | https://doi.org/10.1038/s43856-025-01042-y |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849226023398277120 |
|---|---|
| author | Xi Cheng Sujith Swarna Jermaine Robertson Nathaniel A. Cleri Jordan R. Saadon Chiemeka Uwakwe Yindong Hua Seyed Morsal Mosallami Aghili Cassie Wang Robert S. Kleyner Xuwen Zheng Ariana Forohar John Servider Kurt Butler Chao Chen Jordane Dimidschstein Petar M. Djurić Charles B. Mikell Sima Mofakham |
| author_facet | Xi Cheng Sujith Swarna Jermaine Robertson Nathaniel A. Cleri Jordan R. Saadon Chiemeka Uwakwe Yindong Hua Seyed Morsal Mosallami Aghili Cassie Wang Robert S. Kleyner Xuwen Zheng Ariana Forohar John Servider Kurt Butler Chao Chen Jordane Dimidschstein Petar M. Djurić Charles B. Mikell Sima Mofakham |
| author_sort | Xi Cheng |
| collection | DOAJ |
| description | Abstract Background Many brain injury patients who appear unresponsive retain subtle, purposeful motor behaviors, signaling capacity for recovery. We hypothesized that low-amplitude movements precede larger-amplitude voluntary movements detectable by clinicians after acute brain injury. To test this hypothesis, we developed a novel, as far as we are aware, computer vision-based tool (SeeMe) that detects and quantifies low-amplitude facial movements in response to auditory commands. Methods We enrolled 16 healthy volunteers and 37 comatose acute brain injury patients (Glasgow Coma Scale ≤8) aged 18–85 with no prior neurological diagnoses. We measured facial movements to command assessed using SeeMe and compared them to clinicians’ exams. The primary outcome was the detection of facial movement in response to auditory commands. To assess comprehension, we tested whether movements were specific to command type (i.e., eye-opening to open your eyes and not stick out your tongue) with a machine learning-based classifier. Results Here we show that SeeMe detects eye-opening in comatose patients 4.1 days earlier than clinicians. SeeMe also detects eye-opening in more comatose patients (30/36, 85.7%) than clinical examination (25/36, 71.4%). In patients without an obscuring endotracheal tube, SeeMe detects mouth movements in 16/17 (94.1%) patients. The amplitude and number of SeeMe-detected responses correlate with clinical outcome at discharge. Using our classifier, eye-opening is specific (81%) to the command open your eyes. Conclusion Acute brain injury patients have low-amplitude movements before overt movements. Thus, many covertly conscious patients may have motor behavior currently undetected by clinicians. |
| format | Article |
| id | doaj-art-dd3b6ca7b6624cb6a1269104527bbe76 |
| institution | Kabale University |
| issn | 2730-664X |
| language | English |
| publishDate | 2025-08-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Medicine |
| spelling | doaj-art-dd3b6ca7b6624cb6a1269104527bbe762025-08-24T11:47:41ZengNature PortfolioCommunications Medicine2730-664X2025-08-01511910.1038/s43856-025-01042-yComputer vision detects covert voluntary facial movements in unresponsive brain injury patientsXi Cheng0Sujith Swarna1Jermaine Robertson2Nathaniel A. Cleri3Jordan R. Saadon4Chiemeka Uwakwe5Yindong Hua6Seyed Morsal Mosallami Aghili7Cassie Wang8Robert S. Kleyner9Xuwen Zheng10Ariana Forohar11John Servider12Kurt Butler13Chao Chen14Jordane Dimidschstein15Petar M. Djurić16Charles B. Mikell17Sima Mofakham18Department of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Electrical and Computer Engineering, Stony Brook UniversityDepartment of Biomedical Informatics, Stony Brook UniversityStanley Center for Psychiatric Research, Broad Institute of Harvard and MITDepartment of Electrical and Computer Engineering, Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityDepartment of Neurosurgery, Renaissance School of Medicine at Stony Brook UniversityAbstract Background Many brain injury patients who appear unresponsive retain subtle, purposeful motor behaviors, signaling capacity for recovery. We hypothesized that low-amplitude movements precede larger-amplitude voluntary movements detectable by clinicians after acute brain injury. To test this hypothesis, we developed a novel, as far as we are aware, computer vision-based tool (SeeMe) that detects and quantifies low-amplitude facial movements in response to auditory commands. Methods We enrolled 16 healthy volunteers and 37 comatose acute brain injury patients (Glasgow Coma Scale ≤8) aged 18–85 with no prior neurological diagnoses. We measured facial movements to command assessed using SeeMe and compared them to clinicians’ exams. The primary outcome was the detection of facial movement in response to auditory commands. To assess comprehension, we tested whether movements were specific to command type (i.e., eye-opening to open your eyes and not stick out your tongue) with a machine learning-based classifier. Results Here we show that SeeMe detects eye-opening in comatose patients 4.1 days earlier than clinicians. SeeMe also detects eye-opening in more comatose patients (30/36, 85.7%) than clinical examination (25/36, 71.4%). In patients without an obscuring endotracheal tube, SeeMe detects mouth movements in 16/17 (94.1%) patients. The amplitude and number of SeeMe-detected responses correlate with clinical outcome at discharge. Using our classifier, eye-opening is specific (81%) to the command open your eyes. Conclusion Acute brain injury patients have low-amplitude movements before overt movements. Thus, many covertly conscious patients may have motor behavior currently undetected by clinicians.https://doi.org/10.1038/s43856-025-01042-y |
| spellingShingle | Xi Cheng Sujith Swarna Jermaine Robertson Nathaniel A. Cleri Jordan R. Saadon Chiemeka Uwakwe Yindong Hua Seyed Morsal Mosallami Aghili Cassie Wang Robert S. Kleyner Xuwen Zheng Ariana Forohar John Servider Kurt Butler Chao Chen Jordane Dimidschstein Petar M. Djurić Charles B. Mikell Sima Mofakham Computer vision detects covert voluntary facial movements in unresponsive brain injury patients Communications Medicine |
| title | Computer vision detects covert voluntary facial movements in unresponsive brain injury patients |
| title_full | Computer vision detects covert voluntary facial movements in unresponsive brain injury patients |
| title_fullStr | Computer vision detects covert voluntary facial movements in unresponsive brain injury patients |
| title_full_unstemmed | Computer vision detects covert voluntary facial movements in unresponsive brain injury patients |
| title_short | Computer vision detects covert voluntary facial movements in unresponsive brain injury patients |
| title_sort | computer vision detects covert voluntary facial movements in unresponsive brain injury patients |
| url | https://doi.org/10.1038/s43856-025-01042-y |
| work_keys_str_mv | AT xicheng computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT sujithswarna computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT jermainerobertson computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT nathanielacleri computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT jordanrsaadon computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT chiemekauwakwe computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT yindonghua computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT seyedmorsalmosallamiaghili computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT cassiewang computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT robertskleyner computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT xuwenzheng computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT arianaforohar computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT johnservider computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT kurtbutler computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT chaochen computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT jordanedimidschstein computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT petarmdjuric computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT charlesbmikell computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients AT simamofakham computervisiondetectscovertvoluntaryfacialmovementsinunresponsivebraininjurypatients |