Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes
This paper is concerned with the asymptotic optimality of quantized stationary policies for continuous-time Markov decision processes (CTMDPs) in Polish spaces with state-dependent discount factors, where the transition rates and reward rates are allowed to be unbounded. Using the dynamic programmin...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-01-01
|
| Series: | Discrete Dynamics in Nature and Society |
| Online Access: | http://dx.doi.org/10.1155/2022/1080946 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832555161040453632 |
|---|---|
| author | Xiao Wu Yanqiu Tang |
| author_facet | Xiao Wu Yanqiu Tang |
| author_sort | Xiao Wu |
| collection | DOAJ |
| description | This paper is concerned with the asymptotic optimality of quantized stationary policies for continuous-time Markov decision processes (CTMDPs) in Polish spaces with state-dependent discount factors, where the transition rates and reward rates are allowed to be unbounded. Using the dynamic programming approach, we first establish the discounted optimal equation and the existence of its solutions. Then, we obtain the existence of optimal deterministic stationary policies under suitable conditions by more concise proofs. Furthermore, we discretize and incentivize the action space and construct a sequence of quantizer policies, which is the approximation of the optimal stationary policies of the CTMDPs, and get the approximation result and the rates of convergence on the expected discounted rewards of the quantized stationary policies. Also, we give an iteration algorithm on the approximate optimal policies. Finally, we give an example to illustrate the asymptotic optimality. |
| format | Article |
| id | doaj-art-00b1158acfad44d6b4ea046be09a5fa2 |
| institution | Kabale University |
| issn | 1607-887X |
| language | English |
| publishDate | 2022-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Discrete Dynamics in Nature and Society |
| spelling | doaj-art-00b1158acfad44d6b4ea046be09a5fa22025-02-03T05:49:21ZengWileyDiscrete Dynamics in Nature and Society1607-887X2022-01-01202210.1155/2022/1080946Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision ProcessesXiao Wu0Yanqiu Tang1School of Mathematics and StatisticsSchool of Mathematics and StatisticsThis paper is concerned with the asymptotic optimality of quantized stationary policies for continuous-time Markov decision processes (CTMDPs) in Polish spaces with state-dependent discount factors, where the transition rates and reward rates are allowed to be unbounded. Using the dynamic programming approach, we first establish the discounted optimal equation and the existence of its solutions. Then, we obtain the existence of optimal deterministic stationary policies under suitable conditions by more concise proofs. Furthermore, we discretize and incentivize the action space and construct a sequence of quantizer policies, which is the approximation of the optimal stationary policies of the CTMDPs, and get the approximation result and the rates of convergence on the expected discounted rewards of the quantized stationary policies. Also, we give an iteration algorithm on the approximate optimal policies. Finally, we give an example to illustrate the asymptotic optimality.http://dx.doi.org/10.1155/2022/1080946 |
| spellingShingle | Xiao Wu Yanqiu Tang Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes Discrete Dynamics in Nature and Society |
| title | Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes |
| title_full | Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes |
| title_fullStr | Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes |
| title_full_unstemmed | Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes |
| title_short | Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes |
| title_sort | asymptotic optimality and rates of convergence of quantized stationary policies in continuous time markov decision processes |
| url | http://dx.doi.org/10.1155/2022/1080946 |
| work_keys_str_mv | AT xiaowu asymptoticoptimalityandratesofconvergenceofquantizedstationarypoliciesincontinuoustimemarkovdecisionprocesses AT yanqiutang asymptoticoptimalityandratesofconvergenceofquantizedstationarypoliciesincontinuoustimemarkovdecisionprocesses |