Evidence for compositionality in fMRI visual representations via Brain Algebra
Abstract Electrophysiological and neuroimaging studies have revealed how the brain encodes various visual categories and concepts. An open question is how combinations of multiple visual concepts are represented in terms of the component brain patterns: are brain responses to individual concepts com...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-08-01
|
| Series: | Communications Biology |
| Online Access: | https://doi.org/10.1038/s42003-025-08706-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849226006590652416 |
|---|---|
| author | Matteo Ferrante Tommaso Boccato Nicola Toschi Rufin VanRullen |
| author_facet | Matteo Ferrante Tommaso Boccato Nicola Toschi Rufin VanRullen |
| author_sort | Matteo Ferrante |
| collection | DOAJ |
| description | Abstract Electrophysiological and neuroimaging studies have revealed how the brain encodes various visual categories and concepts. An open question is how combinations of multiple visual concepts are represented in terms of the component brain patterns: are brain responses to individual concepts composed according to algebraic rules? To explore this, we generated “conceptual perturbations" in neural space by averaging fMRI responses to images with a shared concept (e.g., “winter" or “summer"). After thresholding to ensure specificity, we applied these perturbations to the neural pattern associated with a base image, forming new brain patterns that incorporate the added concept. These modified brain patterns were then decoded into images using a pretrained fMRI-to-image decoding model. Qualitative and quantitative inspection of the resulting images provides insight into how the brain might combine visual concepts. For example, adding a “winter" perturbation to the brain pattern of a man on a skateboard yields a new pattern representing a man on a snowboard in a winter scene—even when the perturbation modifies only a small subset of voxels. Our findings reveal that compositional processes in neural representations may lead to predictable perceptual outcomes, as interpreted by our decoding model. This suggests that the brain’s combinatory encoding of concepts may follow a systematic, algebraic-like process—what we term “brain algebra." Although our study is model-driven, it opens avenues for future empirical work into the mechanisms of compositionality in the brain. |
| format | Article |
| id | doaj-art-a2b5c662064f44eaa95e91048e6fdb05 |
| institution | Kabale University |
| issn | 2399-3642 |
| language | English |
| publishDate | 2025-08-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Biology |
| spelling | doaj-art-a2b5c662064f44eaa95e91048e6fdb052025-08-24T11:45:45ZengNature PortfolioCommunications Biology2399-36422025-08-018111110.1038/s42003-025-08706-4Evidence for compositionality in fMRI visual representations via Brain AlgebraMatteo Ferrante0Tommaso Boccato1Nicola Toschi2Rufin VanRullen3Department of Biomedicine and Prevention, University of Rome, Tor Vergata (IT)Department of Biomedicine and Prevention, University of Rome, Tor Vergata (IT)Martinos Center For Biomedical Imaging, MGH and Harvard Medical School (USA)CerCo, CNRS UMR5549Abstract Electrophysiological and neuroimaging studies have revealed how the brain encodes various visual categories and concepts. An open question is how combinations of multiple visual concepts are represented in terms of the component brain patterns: are brain responses to individual concepts composed according to algebraic rules? To explore this, we generated “conceptual perturbations" in neural space by averaging fMRI responses to images with a shared concept (e.g., “winter" or “summer"). After thresholding to ensure specificity, we applied these perturbations to the neural pattern associated with a base image, forming new brain patterns that incorporate the added concept. These modified brain patterns were then decoded into images using a pretrained fMRI-to-image decoding model. Qualitative and quantitative inspection of the resulting images provides insight into how the brain might combine visual concepts. For example, adding a “winter" perturbation to the brain pattern of a man on a skateboard yields a new pattern representing a man on a snowboard in a winter scene—even when the perturbation modifies only a small subset of voxels. Our findings reveal that compositional processes in neural representations may lead to predictable perceptual outcomes, as interpreted by our decoding model. This suggests that the brain’s combinatory encoding of concepts may follow a systematic, algebraic-like process—what we term “brain algebra." Although our study is model-driven, it opens avenues for future empirical work into the mechanisms of compositionality in the brain.https://doi.org/10.1038/s42003-025-08706-4 |
| spellingShingle | Matteo Ferrante Tommaso Boccato Nicola Toschi Rufin VanRullen Evidence for compositionality in fMRI visual representations via Brain Algebra Communications Biology |
| title | Evidence for compositionality in fMRI visual representations via Brain Algebra |
| title_full | Evidence for compositionality in fMRI visual representations via Brain Algebra |
| title_fullStr | Evidence for compositionality in fMRI visual representations via Brain Algebra |
| title_full_unstemmed | Evidence for compositionality in fMRI visual representations via Brain Algebra |
| title_short | Evidence for compositionality in fMRI visual representations via Brain Algebra |
| title_sort | evidence for compositionality in fmri visual representations via brain algebra |
| url | https://doi.org/10.1038/s42003-025-08706-4 |
| work_keys_str_mv | AT matteoferrante evidenceforcompositionalityinfmrivisualrepresentationsviabrainalgebra AT tommasoboccato evidenceforcompositionalityinfmrivisualrepresentationsviabrainalgebra AT nicolatoschi evidenceforcompositionalityinfmrivisualrepresentationsviabrainalgebra AT rufinvanrullen evidenceforcompositionalityinfmrivisualrepresentationsviabrainalgebra |