Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognition
Traditional models of reading lack a realistic simulation of the early visual processing stages, taking input in the form of letter banks and predefined line segments, making them unsuitable for modeling early brain responses. We used variations of the VGG-11 convolutional neural network (CNN) to cr...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
eLife Sciences Publications Ltd
2025-05-01
|
| Series: | eLife |
| Subjects: | |
| Online Access: | https://elifesciences.org/articles/96217 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850136272799006720 |
|---|---|
| author | Marijn van Vliet Oona Rinkinen Takao Shimizu Anni-Mari Niskanen Barry Devereux Riitta Salmelin |
| author_facet | Marijn van Vliet Oona Rinkinen Takao Shimizu Anni-Mari Niskanen Barry Devereux Riitta Salmelin |
| author_sort | Marijn van Vliet |
| collection | DOAJ |
| description | Traditional models of reading lack a realistic simulation of the early visual processing stages, taking input in the form of letter banks and predefined line segments, making them unsuitable for modeling early brain responses. We used variations of the VGG-11 convolutional neural network (CNN) to create models of visual word recognition that starts from the pixel-level and performs the macro-scale computations needed for the detection and segmentation of letter shapes to word-form identification of large vocabulary of 10k Finnish words, regardless of letter size, shape, or rotation. The models were evaluated based on an existing magnetoencephalography (MEG) study where participants viewed regular words, pseudowords, noise-embedded words, symbol strings, and consonant strings. The original images used in the study were presented to the models and the activity in the layers was compared to MEG evoked response amplitudes. Through a few alterations to make the network more biologically plausible, we found an CNN architecture that can correctly simulate the behavior of three prominent responses, namely the type I (early visual response), type II (the ‘letter string’ response), and the N400m. In conclusion, starting a model of reading with convolution-and-pooling steps enables the flexibility and realism crucial for a direct model-to-brain comparison. |
| format | Article |
| id | doaj-art-951fe2ab7eb44df0895aed5d97f71b93 |
| institution | OA Journals |
| issn | 2050-084X |
| language | English |
| publishDate | 2025-05-01 |
| publisher | eLife Sciences Publications Ltd |
| record_format | Article |
| series | eLife |
| spelling | doaj-art-951fe2ab7eb44df0895aed5d97f71b932025-08-20T02:31:11ZengeLife Sciences Publications LtdeLife2050-084X2025-05-011310.7554/eLife.96217Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognitionMarijn van Vliet0https://orcid.org/0000-0002-6537-6899Oona Rinkinen1Takao Shimizu2Anni-Mari Niskanen3Barry Devereux4Riitta Salmelin5https://orcid.org/0000-0003-2499-193XDepartment of Neuroscience and Biomedical Engineering, Aalto University, Espoo, FinlandDepartment of Neuroscience and Biomedical Engineering, Aalto University, Espoo, FinlandDepartment of Neuroscience and Biomedical Engineering, Aalto University, Espoo, FinlandDepartment of Neuroscience and Biomedical Engineering, Aalto University, Espoo, FinlandSchool of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, Belfast, United KingdomDepartment of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland; Aalto NeuroImaging, Aalto University, Espo, FinlandTraditional models of reading lack a realistic simulation of the early visual processing stages, taking input in the form of letter banks and predefined line segments, making them unsuitable for modeling early brain responses. We used variations of the VGG-11 convolutional neural network (CNN) to create models of visual word recognition that starts from the pixel-level and performs the macro-scale computations needed for the detection and segmentation of letter shapes to word-form identification of large vocabulary of 10k Finnish words, regardless of letter size, shape, or rotation. The models were evaluated based on an existing magnetoencephalography (MEG) study where participants viewed regular words, pseudowords, noise-embedded words, symbol strings, and consonant strings. The original images used in the study were presented to the models and the activity in the layers was compared to MEG evoked response amplitudes. Through a few alterations to make the network more biologically plausible, we found an CNN architecture that can correctly simulate the behavior of three prominent responses, namely the type I (early visual response), type II (the ‘letter string’ response), and the N400m. In conclusion, starting a model of reading with convolution-and-pooling steps enables the flexibility and realism crucial for a direct model-to-brain comparison.https://elifesciences.org/articles/96217computational modelingmagnetoencephalographyreadingevoked potentials |
| spellingShingle | Marijn van Vliet Oona Rinkinen Takao Shimizu Anni-Mari Niskanen Barry Devereux Riitta Salmelin Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognition eLife computational modeling magnetoencephalography reading evoked potentials |
| title | Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognition |
| title_full | Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognition |
| title_fullStr | Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognition |
| title_full_unstemmed | Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognition |
| title_short | Convolutional networks can model the functional modulation of the MEG responses associated with feed-forward processes during visual word recognition |
| title_sort | convolutional networks can model the functional modulation of the meg responses associated with feed forward processes during visual word recognition |
| topic | computational modeling magnetoencephalography reading evoked potentials |
| url | https://elifesciences.org/articles/96217 |
| work_keys_str_mv | AT marijnvanvliet convolutionalnetworkscanmodelthefunctionalmodulationofthemegresponsesassociatedwithfeedforwardprocessesduringvisualwordrecognition AT oonarinkinen convolutionalnetworkscanmodelthefunctionalmodulationofthemegresponsesassociatedwithfeedforwardprocessesduringvisualwordrecognition AT takaoshimizu convolutionalnetworkscanmodelthefunctionalmodulationofthemegresponsesassociatedwithfeedforwardprocessesduringvisualwordrecognition AT annimariniskanen convolutionalnetworkscanmodelthefunctionalmodulationofthemegresponsesassociatedwithfeedforwardprocessesduringvisualwordrecognition AT barrydevereux convolutionalnetworkscanmodelthefunctionalmodulationofthemegresponsesassociatedwithfeedforwardprocessesduringvisualwordrecognition AT riittasalmelin convolutionalnetworkscanmodelthefunctionalmodulationofthemegresponsesassociatedwithfeedforwardprocessesduringvisualwordrecognition |