Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality
Head-worn augmented reality (AR) offers exciting possibilities to help users learn new information. By blending digital content with the learner's real-world environment, AR can create engaging and enjoyable experiences that may improve knowledge retention. Electroencephalography (EEG) allows f...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-05-01
|
| Series: | Computers in Human Behavior Reports |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2451958825000922 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850110860861636608 |
|---|---|
| author | Tom Bullock Emily Machniak Joyce Passananti Kangyou Yu Radha Kumaran You-Jin Kim Jordan Garrett Madhav Viswesvaran Julia Ram Tobias Höllerer Barry Giesbrecht |
| author_facet | Tom Bullock Emily Machniak Joyce Passananti Kangyou Yu Radha Kumaran You-Jin Kim Jordan Garrett Madhav Viswesvaran Julia Ram Tobias Höllerer Barry Giesbrecht |
| author_sort | Tom Bullock |
| collection | DOAJ |
| description | Head-worn augmented reality (AR) offers exciting possibilities to help users learn new information. By blending digital content with the learner's real-world environment, AR can create engaging and enjoyable experiences that may improve knowledge retention. Electroencephalography (EEG) allows for discreet, continuous monitoring of brain activity at the scalp. This study examined whether the N400 event-related potential (ERP), a brain response linked to semantic processing, could be incorporated into a system combining AR and EEG. While the N400 is reliably elicited by a mismatch in meaning between two sequentially presented stimuli, there are two key outstanding questions. First, how do 3D objects presented in AR impact semantic processing as measured by the N400? Second, is there a reliable N400 to mismatches between an object and its name, in addition to mismatches in meaning? Twenty-four young adults viewed sequential pairs of stimuli through an AR headset while EEG was recorded. We manipulated whether the first stimulus was a 3D object or written word and whether the second stimulus matched or mismatched the first in terms of meaning or name. Participants' reaction times were slower for mismatching pairs when compared to matching pairs in all conditions. Averaged ERP and single-trial classification analyses showed robust differences in brain responses. Additionally, participants were more accurate and exhibited faster behavioral and brain responses for naming compared to meaning judgments. These results suggest the N400 could be used in a combined AR-EEG system to provide feedback on semantic understanding, potentially opening exciting new avenues for enhancing learning. |
| format | Article |
| id | doaj-art-e5acb3e9eb284b63befc9e38ec1bcbf9 |
| institution | OA Journals |
| issn | 2451-9588 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Computers in Human Behavior Reports |
| spelling | doaj-art-e5acb3e9eb284b63befc9e38ec1bcbf92025-08-20T02:37:45ZengElsevierComputers in Human Behavior Reports2451-95882025-05-011810067710.1016/j.chbr.2025.100677Electrophysiological evidence for differential semantic processing of words and objects presented in augmented realityTom Bullock0Emily Machniak1Joyce Passananti2Kangyou Yu3Radha Kumaran4You-Jin Kim5Jordan Garrett6Madhav Viswesvaran7Julia Ram8Tobias Höllerer9Barry Giesbrecht10Department of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States; Institute for Collaborative Biotechnologies, University of California, Santa Barbara, Santa Barbara, CA, United States; Corresponding author. Department of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States.Department of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States; Institute for Collaborative Biotechnologies, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Computer Science, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Computer Science, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Computer Science, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Computer Science, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States; Institute for Collaborative Biotechnologies, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United StatesInstitute for Collaborative Biotechnologies, University of California, Santa Barbara, Santa Barbara, CA, United States; Department of Computer Science, University of California, Santa Barbara, Santa Barbara, CA, United StatesDepartment of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States; Institute for Collaborative Biotechnologies, University of California, Santa Barbara, Santa Barbara, CA, United StatesHead-worn augmented reality (AR) offers exciting possibilities to help users learn new information. By blending digital content with the learner's real-world environment, AR can create engaging and enjoyable experiences that may improve knowledge retention. Electroencephalography (EEG) allows for discreet, continuous monitoring of brain activity at the scalp. This study examined whether the N400 event-related potential (ERP), a brain response linked to semantic processing, could be incorporated into a system combining AR and EEG. While the N400 is reliably elicited by a mismatch in meaning between two sequentially presented stimuli, there are two key outstanding questions. First, how do 3D objects presented in AR impact semantic processing as measured by the N400? Second, is there a reliable N400 to mismatches between an object and its name, in addition to mismatches in meaning? Twenty-four young adults viewed sequential pairs of stimuli through an AR headset while EEG was recorded. We manipulated whether the first stimulus was a 3D object or written word and whether the second stimulus matched or mismatched the first in terms of meaning or name. Participants' reaction times were slower for mismatching pairs when compared to matching pairs in all conditions. Averaged ERP and single-trial classification analyses showed robust differences in brain responses. Additionally, participants were more accurate and exhibited faster behavioral and brain responses for naming compared to meaning judgments. These results suggest the N400 could be used in a combined AR-EEG system to provide feedback on semantic understanding, potentially opening exciting new avenues for enhancing learning.http://www.sciencedirect.com/science/article/pii/S2451958825000922Augmented realityEEGERPClassificationN4003D models |
| spellingShingle | Tom Bullock Emily Machniak Joyce Passananti Kangyou Yu Radha Kumaran You-Jin Kim Jordan Garrett Madhav Viswesvaran Julia Ram Tobias Höllerer Barry Giesbrecht Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality Computers in Human Behavior Reports Augmented reality EEG ERP Classification N400 3D models |
| title | Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality |
| title_full | Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality |
| title_fullStr | Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality |
| title_full_unstemmed | Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality |
| title_short | Electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality |
| title_sort | electrophysiological evidence for differential semantic processing of words and objects presented in augmented reality |
| topic | Augmented reality EEG ERP Classification N400 3D models |
| url | http://www.sciencedirect.com/science/article/pii/S2451958825000922 |
| work_keys_str_mv | AT tombullock electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT emilymachniak electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT joycepassananti electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT kangyouyu electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT radhakumaran electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT youjinkim electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT jordangarrett electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT madhavviswesvaran electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT juliaram electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT tobiashollerer electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality AT barrygiesbrecht electrophysiologicalevidencefordifferentialsemanticprocessingofwordsandobjectspresentedinaugmentedreality |