Averaged Soft Actor-Critic for Deep Reinforcement Learning
With the advent of the era of artificial intelligence, deep reinforcement learning (DRL) has achieved unprecedented success in high-dimensional and large-scale artificial intelligence tasks. However, the insecurity and instability of the DRL algorithm have an important impact on its performance. The...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-01-01
|
| Series: | Complexity |
| Online Access: | http://dx.doi.org/10.1155/2021/6658724 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849309174037479424 |
|---|---|
| author | Feng Ding Guanfeng Ma Zhikui Chen Jing Gao Peng Li |
| author_facet | Feng Ding Guanfeng Ma Zhikui Chen Jing Gao Peng Li |
| author_sort | Feng Ding |
| collection | DOAJ |
| description | With the advent of the era of artificial intelligence, deep reinforcement learning (DRL) has achieved unprecedented success in high-dimensional and large-scale artificial intelligence tasks. However, the insecurity and instability of the DRL algorithm have an important impact on its performance. The Soft Actor-Critic (SAC) algorithm uses advanced functions to update the policy and value network to alleviate some of these problems. However, SAC still has some problems. In order to reduce the error caused by the overestimation of SAC, we propose a new SAC algorithm called Averaged-SAC. By averaging the previously learned action-state estimates, it reduces the overestimation problem of soft Q-learning, thereby contributing to a more stable training process and improving performance. We evaluate the performance of Averaged-SAC through some games in the MuJoCo environment. The experimental results show that the Averaged-SAC algorithm effectively improves the performance of the SAC algorithm and the stability of the training process. |
| format | Article |
| id | doaj-art-03969153b56d499d950ce34cd5b8e312 |
| institution | Kabale University |
| issn | 1076-2787 1099-0526 |
| language | English |
| publishDate | 2021-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Complexity |
| spelling | doaj-art-03969153b56d499d950ce34cd5b8e3122025-08-20T03:54:14ZengWileyComplexity1076-27871099-05262021-01-01202110.1155/2021/66587246658724Averaged Soft Actor-Critic for Deep Reinforcement LearningFeng Ding0Guanfeng Ma1Zhikui Chen2Jing Gao3Peng Li4School of Software Technology, Dalian University of Technology, Dalian, ChinaSchool of Software Technology, Dalian University of Technology, Dalian, ChinaSchool of Software Technology, Dalian University of Technology, Dalian, ChinaSchool of Software Technology, Dalian University of Technology, Dalian, ChinaSchool of Software Technology, Dalian University of Technology, Dalian, ChinaWith the advent of the era of artificial intelligence, deep reinforcement learning (DRL) has achieved unprecedented success in high-dimensional and large-scale artificial intelligence tasks. However, the insecurity and instability of the DRL algorithm have an important impact on its performance. The Soft Actor-Critic (SAC) algorithm uses advanced functions to update the policy and value network to alleviate some of these problems. However, SAC still has some problems. In order to reduce the error caused by the overestimation of SAC, we propose a new SAC algorithm called Averaged-SAC. By averaging the previously learned action-state estimates, it reduces the overestimation problem of soft Q-learning, thereby contributing to a more stable training process and improving performance. We evaluate the performance of Averaged-SAC through some games in the MuJoCo environment. The experimental results show that the Averaged-SAC algorithm effectively improves the performance of the SAC algorithm and the stability of the training process.http://dx.doi.org/10.1155/2021/6658724 |
| spellingShingle | Feng Ding Guanfeng Ma Zhikui Chen Jing Gao Peng Li Averaged Soft Actor-Critic for Deep Reinforcement Learning Complexity |
| title | Averaged Soft Actor-Critic for Deep Reinforcement Learning |
| title_full | Averaged Soft Actor-Critic for Deep Reinforcement Learning |
| title_fullStr | Averaged Soft Actor-Critic for Deep Reinforcement Learning |
| title_full_unstemmed | Averaged Soft Actor-Critic for Deep Reinforcement Learning |
| title_short | Averaged Soft Actor-Critic for Deep Reinforcement Learning |
| title_sort | averaged soft actor critic for deep reinforcement learning |
| url | http://dx.doi.org/10.1155/2021/6658724 |
| work_keys_str_mv | AT fengding averagedsoftactorcriticfordeepreinforcementlearning AT guanfengma averagedsoftactorcriticfordeepreinforcementlearning AT zhikuichen averagedsoftactorcriticfordeepreinforcementlearning AT jinggao averagedsoftactorcriticfordeepreinforcementlearning AT pengli averagedsoftactorcriticfordeepreinforcementlearning |