Back to Search
Start Over
Stochastic Approximation for Risk-Aware Markov Decision Processes.
- Source :
-
IEEE Transactions on Automatic Control . Mar2021, Vol. 66 Issue 3, p1314-1320. 7p. - Publication Year :
- 2021
-
Abstract
- We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point problem. The outer loop performs Q-learning to compute an optimal risk-aware policy. Several widely investigated risk measures (e.g., conditional value-at-risk, optimized certainty equivalent, and absolute semideviation) are covered by our algorithm. Almost sure convergence and the convergence rate of the algorithm are established. For an error tolerance ε > 0 for optimal Q-value estimation gap and learning rate k ∈ (1/2,1], the overall convergence rate of our algorithm is Ω((ln(1/δε)/ε2)1/k + (ln(1/ε))1/(1−k)) with probability at least 1−ε. [ABSTRACT FROM AUTHOR]
Details
- Language :
- English
- ISSN :
- 00189286
- Volume :
- 66
- Issue :
- 3
- Database :
- Academic Search Index
- Journal :
- IEEE Transactions on Automatic Control
- Publication Type :
- Periodical
- Accession number :
- 148970686
- Full Text :
- https://doi.org/10.1109/TAC.2020.2989702