Search

Abstract: We consider a denumerable state continuous-time controlled Markov chain (CMC) with possibly unbounded transition and reward rates. We deal with constrained optimality; that is, we want to maximize a discounted reward (an average reward) criterion subject to a constraint on a discounted cost (an average cost). We give conditions ensuring that the average constrained optimal reward and policies can be obtained as the limit, as the discount rate vanishes, of the corresponding discounted constrained optimal reward and policies. This extends to average constrained CMCs the standard results on the vanishing discount approach for the average unconstrained case. We also present an example showing that the vanishing discount results for constrained problems might not hold. [ABSTRACT FROM AUTHOR]

Abstract: This paper is concerned with random selection processes for which, in the previous literature, several sufficient conditions for a De Finetti-type theorem (granting the conditional independence of a family of random variables) have been proposed. In this paper, we prove that these conditions are also necessary. In this sense, we identify the extremes of the convex set of marginal distributions of a random selection process, and then we give a thorough description and classification of such processes. [Copyright &y& Elsevier]

In this paper we give conditions for the existence of bias optimal policies in a class of continuous-time controlled Markov chains with unbounded reward and transition rates. Several characterizations of bias optimality are proposed. We also introduce new sets of conditions ensuring uniform exponential ergodicity of continuous-time controlled Markov chains. [ABSTRACT FROM AUTHOR]

This paper deals with continuous-time zero-sum two-person Markov games with denumerable state space, general (Borel) action spaces and possibly unbounded transition and reward/cost rates. We analyze the bias optimality and the weakly overtaking optimality criteria. An example shows that, in contrast to control (or one-player) problems, these criteria are not equivalent for games. [ABSTRACT FROM AUTHOR]

Given a Markov process, we are interested in the numerical computation of the moments of the exit time from a bounded domain. We use a moment approach which, together with appropriate semidefinite positivity moment conditions, yields a sequence of semidefinite programs (or SDP relaxations), depending on the number of moments considered, that provide a sequence of nonincreasing (resp. nondecreasing) upper (resp. lower) bounds. The results are compared to the linear Hausdorff moment conditions approach considered for the LP relaxations in Helmes et al. [Helmes, K., R&ooml;hl, S., Stockbridge, R.H. Computing moments of the exit time distribution for Markov processes by linear programming. Oper. Res. 2001, 49, 5 16-530]. The SDP relaxations are shown to be more general and more precise than the LP relaxations. [ABSTRACT FROM AUTHOR]