Search

Motivated by uncertain parameters encountered in Markov decision processes (MDPs) and stochastic games, we study the effect of parameter uncertainty on Bellman operator-based algorithms under a set-based framework. Specifically, we first consider a family of MDPs where the cost parameters are in a given compact set; we then define a Bellman operator acting on a set of value functions to produce a new set of value functions as the output under all possible variations in the cost parameter. We prove the existence of a fixed point of this set-based Bellman operator by showing that it is contractive on a complete metric space, and explore its relationship with the corresponding family of MDPs and stochastic games. Additionally, we show that given interval set bounded cost parameters, we can form exact bounds on the set of optimal value functions. Finally, we utilize our results to bound the value function trajectory of a player in a stochastic game., 15 pages, 4 figures

In this paper we study the dynamical behavior of the Chebyshev–Halley methods on the family of degree n polynomials z n + c . We prove that, despite increasing the degree, it is still possible to draw the parameter space by using the orbit of a single critical point. For the methods having z = ∞ as an attracting fixed point, we show how the basins of attraction of the roots become smaller as the value of n grows. We also demonstrate that, although the convergence order of the Chebyshev–Halley family is 3, there is a member of order 4 for each value of n. In the case of quadratic polynomials, we bound the set of parameters which correspond to iterative methods with stable behaviour other than the basins of attraction of the roots.