Search

In this paper, we investigate an optimization problem for continuous-time Markov decision processes with both impulsive and continuous controls. We consider the so-called constrained problem where the objective of the controller is to minimize a total expected discounted optimality criterion associated with a cost rate function while keeping other performance criteria of the same form, but associated with different cost rate functions, below some given bounds. Our model allows multiple impulses at the same time moment. The main objective of this work is to study the associated linear program defined on a space of measures including the occupation measures of the controlled process and to provide sufficient conditions to ensure the existence of an optimal control. [ABSTRACT FROM AUTHOR]

The main results of the paper are concerned with determining the optimal stationary strategies for stochastic discrete control problems on networks with discounted costs. We ground polynomial time algorithms for determining the optimal strategies of this problem using a linear programming approach. Additionally, we show that the proposed approach can be extended for Markov decision processes with a total discounted cost optimization criterion. [ABSTRACT FROM AUTHOR]

The purpose of this paper is two fold. First to establish the theory of discounted constrained Markov decision processes with a countable state and action spaces with general multi-chain structure. Second, to introduce finite approximation methods. We define the occupation measures and obtain properties of the set of all achievable occupation measures under the different admissible policies. We establish the optimality of stationary policies for the constrained control problem, and obtain an LP with a countable number of decision variables through which stationary optimal policies are computed. Since for such an LP one cannot expect to find an optimal solution in a finite number of operations, we present two schemes for finite approximations and establish the convergence of optimal values and policies for both the discounted and the expected average cost, with unbounded cost. Sometimes it turns out to be easier to solve the problem with infinite state space than the problem with finite yet large state space. Based on the optimal policy for the problem with infinite state space, we construct policies which are almost optimal for the problem with truncated state space. This method is applied to obtain an €-optimal policy for a problem of optimal priority assignment under constraints for a system of K finite queues. [ABSTRACT FROM AUTHOR]