Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model.
Because most rewarding events are probabilistic and changing, the extinction of probabilistic rewards is important for survival. It has been proposed that the extinction of probabilistic rewards depends on arousal and the amount of learning of reward values. Midbrain dopamine neurons were suggested...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2014-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0089494&type=printable |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849720540719218688 |
|---|---|
| author | Minryung R Song Jean-Marc Fellous |
| author_facet | Minryung R Song Jean-Marc Fellous |
| author_sort | Minryung R Song |
| collection | DOAJ |
| description | Because most rewarding events are probabilistic and changing, the extinction of probabilistic rewards is important for survival. It has been proposed that the extinction of probabilistic rewards depends on arousal and the amount of learning of reward values. Midbrain dopamine neurons were suggested to play a role in both arousal and learning reward values. Despite extensive research on modeling dopaminergic activity in reward learning (e.g. temporal difference models), few studies have been done on modeling its role in arousal. Although temporal difference models capture key characteristics of dopaminergic activity during the extinction of deterministic rewards, they have been less successful at simulating the extinction of probabilistic rewards. By adding an arousal signal to a temporal difference model, we were able to simulate the extinction of probabilistic rewards and its dependence on the amount of learning. Our simulations propose that arousal allows the probability of reward to have lasting effects on the updating of reward value, which slows the extinction of low probability rewards. Using this model, we predicted that, by signaling the prediction error, dopamine determines the learned reward value that has to be extinguished during extinction and participates in regulating the size of the arousal signal that controls the learning rate. These predictions were supported by pharmacological experiments in rats. |
| format | Article |
| id | doaj-art-8bc00aede3fc4ceeb690759a94b96cf0 |
| institution | DOAJ |
| issn | 1932-6203 |
| language | English |
| publishDate | 2014-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-8bc00aede3fc4ceeb690759a94b96cf02025-08-20T03:11:54ZengPublic Library of Science (PLoS)PLoS ONE1932-62032014-01-0192e8949410.1371/journal.pone.0089494Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model.Minryung R SongJean-Marc FellousBecause most rewarding events are probabilistic and changing, the extinction of probabilistic rewards is important for survival. It has been proposed that the extinction of probabilistic rewards depends on arousal and the amount of learning of reward values. Midbrain dopamine neurons were suggested to play a role in both arousal and learning reward values. Despite extensive research on modeling dopaminergic activity in reward learning (e.g. temporal difference models), few studies have been done on modeling its role in arousal. Although temporal difference models capture key characteristics of dopaminergic activity during the extinction of deterministic rewards, they have been less successful at simulating the extinction of probabilistic rewards. By adding an arousal signal to a temporal difference model, we were able to simulate the extinction of probabilistic rewards and its dependence on the amount of learning. Our simulations propose that arousal allows the probability of reward to have lasting effects on the updating of reward value, which slows the extinction of low probability rewards. Using this model, we predicted that, by signaling the prediction error, dopamine determines the learned reward value that has to be extinguished during extinction and participates in regulating the size of the arousal signal that controls the learning rate. These predictions were supported by pharmacological experiments in rats.https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0089494&type=printable |
| spellingShingle | Minryung R Song Jean-Marc Fellous Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model. PLoS ONE |
| title | Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model. |
| title_full | Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model. |
| title_fullStr | Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model. |
| title_full_unstemmed | Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model. |
| title_short | Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model. |
| title_sort | value learning and arousal in the extinction of probabilistic rewards the role of dopamine in a modified temporal difference model |
| url | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0089494&type=printable |
| work_keys_str_mv | AT minryungrsong valuelearningandarousalintheextinctionofprobabilisticrewardstheroleofdopamineinamodifiedtemporaldifferencemodel AT jeanmarcfellous valuelearningandarousalintheextinctionofprobabilisticrewardstheroleofdopamineinamodifiedtemporaldifferencemodel |