Search

This paper addresses a class of discrete-time Markov decision processes in Borel spaces with a finite number of cost constraints. The constrained control model considers costs of discounted type with state-dependent discount factors which are subject to external disturbances. Our objective is to prove the existence of optimal control policies and characterize them according to certain optimality criteria. Specifically, by rewriting appropriately our original constrained problem as a new one on a space of occupation measures, we apply the direct method to show solvability. Next, the problem is defined as a convex program, and we prove that the existence of a saddle point of the associated Lagrangian operator is equivalent to the existence of an optimal control policy for the constrained problem. Finally, we turn our attention to multi-objective optimization problems, where the existence of Pareto optimal policies can be obtained from the existence of saddle-points of the aforementioned Lagrangian or equivalently from the existence of optimal control policies of constrained problems. [ABSTRACT FROM AUTHOR]

In this paper we study a discrete-time switching control problem when the dynamic of the system evolves as a random walk. The payoff function is a discounted-type total cost whose discount factor may depend on the state and/or the action variables. This flexibility includes cases when the controller can stop the dynamics of the system whether is optimal to do it. To provide optimality results, we use the well-known dynamic programming method that leads to the study of certain functional equations. Under suitable conditions, we prove the existence of solutions to these equations, and we also show that the optimal value of the control problem becomes a minimal solution of these equations. Finally, we illustrate our results through an example about the control of epidemic processes. [ABSTRACT FROM AUTHOR]

This work addresses a switching control problem under which the cost associated with the changes of regimes is allowed to have discontinuities in time. Our main contribution is to show several characterizations of the optimal cost function as well as the existence of ε-optimal control policies. As a by-product, we also study the existence and uniqueness of solutions of a system of backward stochastic differential equations whose barriers (or obstacles) are discontinuous (in fact of càdlàg type) and depend itself on the unknown solution. In the last part of the paper, we study the case when an underlying diffusion is part of the dynamic of the system. In this special case, the optimal payoff becomes a weak solution of the HJB system of PDEs with discontinuous obstacles which is of quasi-variational type. This paper is somehow a continuation of the papers [B. Djehiche, S. Hamadène, and A. Popier, A finite horizon optimal multiple switching problem, SIAM J. Control Optim. 48 (2009), pp. 2751–2770; S. Hamadène and M.A. Morlais, Viscosity solutions of systems of PDEs with interconnected Obstacle and switching problem, Appl. Math. Optim. 67 (2013), pp. 163–196.] that consider continuous costs. [ABSTRACT FROM AUTHOR]

This paper deals with infinite-horizon non-zero-sum stochastic differential games with discounted and average payoffs and an additive structure. Our main objective is to give conditions for the existence of Nash equilibria in the set of stationary randomized strategies for the discounted payoff and then, by following the vanishing discount approach, to show the existence of Nash equilibria for the average payoff. To this end, we use standard dynamic programming techniques. [ABSTRACT FROM AUTHOR]

This paper deals with discrete-time stochastic systems composed of a large number of N interacting objects (a.k.a. agents or particles). There is a central controller whose decisions, at each stage, affect the system behavior. Each object evolves randomly among a finite set of classes, according to a transition law which depends on an unknown parameter. Such a parameter is possibly non observable and may change from stage to stage. Due to the lack of information and to the large number of agents, the control problem under study is rewritten as a game against nature according to the mean field theory; that is, we introduce a game model associated to the proportions of the objects in each class, whereas the values of the unknown parameter are now considered as "actions" selected by an opponent to the controller (the nature). Then, letting N 1 (the mean field limit) and considering a discounted optimality criterion, the objective for the controller is to minimize the maximum cost, where the maximum is taken over all possible strategies of the nature. [ABSTRACT FROM AUTHOR]

In this paper, we propose an application of the so-called games against nature for giving solution to an ergodic control problem governed by a general class of Markov diffusion processes whose coefficients depend on an unknown and non-observable parameter. To this end, we assume that the values of the parameter are taken by means of ‘actions’ made by some opposite player of the controller (the nature). Then, the problem reduces to finding optimality for the controller given that the nature has chosen its best strategy. Such a control is also known as the worst case optimal control. Our analysis is based on the use of the dynamic programming technique by showing, among other facts, the existence of classical (twice differentiable) solutions of the so called Hamilton Jacobi Bellman equation. We also provide an example on economic welfare to illustrate our results. [ABSTRACT FROM PUBLISHER]

This article concernsn-dimensional controlled diffusion processes. The main problem is to maximize a certain long-run average reward (also known as an ergodic reward) in such a way that a given long-run average cost is bounded above by a constant. Under suitable assumptions, the existence of optimal controls for such constrained control problems is a well-known fact. In this article we go a bit further and our goal is to introduce a technique to compute optimal controls. To this end, we follow the Lagrange multipliers approach. An example on a linear-quadratic system illustrates our results. [ABSTRACT FROM AUTHOR]

In this paper, we study the optimal ergodic control problem with minimum variance for a general class of controlled Markov diffusion processes. To this end, we follow a lexicographical approach. Namely, we first identify the class of average optimal control policies, and then within this class, we search policies that minimize the limiting average variance. To do this, a key intermediate step is to show that the limiting average variance is a constant independent of the initial state. Our proof of this latter fact gives a result stronger than the central limit theorem for diffusions. An application to manufacturing systems illustrates our results. [ABSTRACT FROM PUBLISHER]