Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex.
The role of the dorsal anterior cingulate cortex (dACC) in cognition is a frequently studied yet highly debated topic in neuroscience. Most authors agree that the dACC is involved in either cognitive control (e.g., voluntary inhibition of automatic responses) or monitoring (e.g., comparing expectati...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-04-01
|
| Series: | PLoS Computational Biology |
| Online Access: | https://doi.org/10.1371/journal.pcbi.1013025 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850193571986014208 |
|---|---|
| author | Tim Vriens Eliana Vassena Giovanni Pezzulo Gianluca Baldassarre Massimo Silvetti |
| author_facet | Tim Vriens Eliana Vassena Giovanni Pezzulo Gianluca Baldassarre Massimo Silvetti |
| author_sort | Tim Vriens |
| collection | DOAJ |
| description | The role of the dorsal anterior cingulate cortex (dACC) in cognition is a frequently studied yet highly debated topic in neuroscience. Most authors agree that the dACC is involved in either cognitive control (e.g., voluntary inhibition of automatic responses) or monitoring (e.g., comparing expectations with outcomes, detecting errors, tracking surprise). A consensus on which theoretical perspective best explains dACC contribution to behaviour is still lacking, as two distinct sets of studies report dACC activation in tasks requiring surprise tracking for performance monitoring and cognitive control without involving surprise monitoring, respectively. This creates a theoretical impasse, as no single current account can reconcile these findings. Here we propose a novel hypothesis on dACC function that integrates both the monitoring and the cognitive control perspectives in a unifying, meta-Reinforcement Learning framework, in which cognitive control is optimized by meta-learning based on tracking Bayesian surprise. We tested the quantitative predictions from our theory in three different functional neuroimaging experiments at the basis of the current theory crisis. We show that the meta-Reinforcement Learning perspective successfully captures all the neuroimaging results by predicting both cognitive control and monitoring functions, proposing a solution to the theory crisis about dACC function within an integrative framework. In sum, our results suggest that dACC function can be framed as a meta-learning optimisation of cognitive control, providing an integrative perspective on its roles in cognitive control, surprise tracking, and performance monitoring. |
| format | Article |
| id | doaj-art-e47fcf2520b7446aa9624cacd4e82d65 |
| institution | OA Journals |
| issn | 1553-734X 1553-7358 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS Computational Biology |
| spelling | doaj-art-e47fcf2520b7446aa9624cacd4e82d652025-08-20T02:14:15ZengPublic Library of Science (PLoS)PLoS Computational Biology1553-734X1553-73582025-04-01214e101302510.1371/journal.pcbi.1013025Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex.Tim VriensEliana VassenaGiovanni PezzuloGianluca BaldassarreMassimo SilvettiThe role of the dorsal anterior cingulate cortex (dACC) in cognition is a frequently studied yet highly debated topic in neuroscience. Most authors agree that the dACC is involved in either cognitive control (e.g., voluntary inhibition of automatic responses) or monitoring (e.g., comparing expectations with outcomes, detecting errors, tracking surprise). A consensus on which theoretical perspective best explains dACC contribution to behaviour is still lacking, as two distinct sets of studies report dACC activation in tasks requiring surprise tracking for performance monitoring and cognitive control without involving surprise monitoring, respectively. This creates a theoretical impasse, as no single current account can reconcile these findings. Here we propose a novel hypothesis on dACC function that integrates both the monitoring and the cognitive control perspectives in a unifying, meta-Reinforcement Learning framework, in which cognitive control is optimized by meta-learning based on tracking Bayesian surprise. We tested the quantitative predictions from our theory in three different functional neuroimaging experiments at the basis of the current theory crisis. We show that the meta-Reinforcement Learning perspective successfully captures all the neuroimaging results by predicting both cognitive control and monitoring functions, proposing a solution to the theory crisis about dACC function within an integrative framework. In sum, our results suggest that dACC function can be framed as a meta-learning optimisation of cognitive control, providing an integrative perspective on its roles in cognitive control, surprise tracking, and performance monitoring.https://doi.org/10.1371/journal.pcbi.1013025 |
| spellingShingle | Tim Vriens Eliana Vassena Giovanni Pezzulo Gianluca Baldassarre Massimo Silvetti Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. PLoS Computational Biology |
| title | Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. |
| title_full | Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. |
| title_fullStr | Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. |
| title_full_unstemmed | Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. |
| title_short | Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. |
| title_sort | meta reinforcement learning reconciles surprise value and control in the anterior cingulate cortex |
| url | https://doi.org/10.1371/journal.pcbi.1013025 |
| work_keys_str_mv | AT timvriens metareinforcementlearningreconcilessurprisevalueandcontrolintheanteriorcingulatecortex AT elianavassena metareinforcementlearningreconcilessurprisevalueandcontrolintheanteriorcingulatecortex AT giovannipezzulo metareinforcementlearningreconcilessurprisevalueandcontrolintheanteriorcingulatecortex AT gianlucabaldassarre metareinforcementlearningreconcilessurprisevalueandcontrolintheanteriorcingulatecortex AT massimosilvetti metareinforcementlearningreconcilessurprisevalueandcontrolintheanteriorcingulatecortex |