TD algorithm based on double-layer fuzzy partitioning
When dealing with the continuous space problems,the traditional Q-iteration algorithms based on lookup-table or function approximation converge slowly and are diff lt to get a continuous policy.To overcome the above weak-nesses,an on-policy TD algorithm named DFP-OPTD was proposed based on double-la...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Journal on Communications
2013-10-01
|
| Series: | Tongxin xuebao |
| Subjects: | |
| Online Access: | http://www.joconline.com.cn/zh/article/doi/10.3969/j.issn.1000-436x.2013.10.011/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841539835719843840 |
|---|---|
| author | Xiang MU Quan LIU Qi-ming FU Hong-kun SUN Xin ZHOU |
| author_facet | Xiang MU Quan LIU Qi-ming FU Hong-kun SUN Xin ZHOU |
| author_sort | Xiang MU |
| collection | DOAJ |
| description | When dealing with the continuous space problems,the traditional Q-iteration algorithms based on lookup-table or function approximation converge slowly and are diff lt to get a continuous policy.To overcome the above weak-nesses,an on-policy TD algorithm named DFP-OPTD was proposed based on double-layer fuzzy partitioning and its convergence was proved.The first layer of fuzzy partitioning was applied for state space,the second layer of fuzzy parti-tioning was applied for action space,and Q-value functions were computed by the combination of the two layer fuzzy partitioning.Based on the Q-value function,the consequent parameters of fuzzy rules were updated by gradient descent method.Applying DFP-OPTD on two classical reinforcement learning problems,experimental results show that the algo-rithm not only can be used to get a continuous action policy,but also has a better convergence performance. |
| format | Article |
| id | doaj-art-bf4e4005ccfa4516855d7a5f564d1717 |
| institution | Kabale University |
| issn | 1000-436X |
| language | zho |
| publishDate | 2013-10-01 |
| publisher | Editorial Department of Journal on Communications |
| record_format | Article |
| series | Tongxin xuebao |
| spelling | doaj-art-bf4e4005ccfa4516855d7a5f564d17172025-01-14T06:41:28ZzhoEditorial Department of Journal on CommunicationsTongxin xuebao1000-436X2013-10-0134929959675807TD algorithm based on double-layer fuzzy partitioningXiang MUQuan LIUQi-ming FUHong-kun SUNXin ZHOUWhen dealing with the continuous space problems,the traditional Q-iteration algorithms based on lookup-table or function approximation converge slowly and are diff lt to get a continuous policy.To overcome the above weak-nesses,an on-policy TD algorithm named DFP-OPTD was proposed based on double-layer fuzzy partitioning and its convergence was proved.The first layer of fuzzy partitioning was applied for state space,the second layer of fuzzy parti-tioning was applied for action space,and Q-value functions were computed by the combination of the two layer fuzzy partitioning.Based on the Q-value function,the consequent parameters of fuzzy rules were updated by gradient descent method.Applying DFP-OPTD on two classical reinforcement learning problems,experimental results show that the algo-rithm not only can be used to get a continuous action policy,but also has a better convergence performance.http://www.joconline.com.cn/zh/article/doi/10.3969/j.issn.1000-436x.2013.10.011/reinforcement learningon-policygradient descentdouble layer fuzzy partitioningcontinuous action policy |
| spellingShingle | Xiang MU Quan LIU Qi-ming FU Hong-kun SUN Xin ZHOU TD algorithm based on double-layer fuzzy partitioning Tongxin xuebao reinforcement learning on-policy gradient descent double layer fuzzy partitioning continuous action policy |
| title | TD algorithm based on double-layer fuzzy partitioning |
| title_full | TD algorithm based on double-layer fuzzy partitioning |
| title_fullStr | TD algorithm based on double-layer fuzzy partitioning |
| title_full_unstemmed | TD algorithm based on double-layer fuzzy partitioning |
| title_short | TD algorithm based on double-layer fuzzy partitioning |
| title_sort | td algorithm based on double layer fuzzy partitioning |
| topic | reinforcement learning on-policy gradient descent double layer fuzzy partitioning continuous action policy |
| url | http://www.joconline.com.cn/zh/article/doi/10.3969/j.issn.1000-436x.2013.10.011/ |
| work_keys_str_mv | AT xiangmu tdalgorithmbasedondoublelayerfuzzypartitioning AT quanliu tdalgorithmbasedondoublelayerfuzzypartitioning AT qimingfu tdalgorithmbasedondoublelayerfuzzypartitioning AT hongkunsun tdalgorithmbasedondoublelayerfuzzypartitioning AT xinzhou tdalgorithmbasedondoublelayerfuzzypartitioning |