Computation Techniques for Large Scale Undiscounted Markov Decision Processes.

This paper considers computation techniques associated with the optimization of large scale Markov decision processes. Markov decision processes and successive approximation procedures are described. Then a procedure for scaling continuous time and renewal processes so that they are amenable to the second procedure is discussed. The effect of the scale factor value on the convergence rate of the procedure and insights into proper scale factor selection are given. Finally, various methods of achieving computational efficiency during execution of the optimization are considered.