Search

We consider infinite horizon optimal control problems with time averaging and time discounting criteria and give estimates for the Cesàro and Abel limits of their optimal values in the case when they depend on the initial conditions. We establish that these limits are bounded from above by the optimal value of a certain infinite dimensional (ID) linear programming (LP) problem and that they are bounded from below by the optimal value of the corresponding dual problem. (These estimates imply, in particular, that the Cesàro and Abel limits exist and are equal to each other if there is no duality gap). In addition, we obtain IDLP-based optimality conditions for the long run average optimal control problem, and we illustrate these conditions by an example.

We consider stabilization of an equilibrium point via infinite horizon discounted optimal control in discrete-time. In addition to applications in economics and social sciences, discounted optimal control is a commonly used numerical technique guaranteeing solvability of certain classes of optimal control problems . In this paper, we present conditions based on strict dissipativity that ensure that the optimally controlled system is asymptotically stable or practically asymptotically stable. These conditions are shown to be complementary to recently proposed conditions based on a detectability property. Illustrative examples are provided.

This paper is devoted to the study of infinite horizon optimal control problems with time discounting and time averaging criteria. These problems are related to certain infinite-dimensional linear programming (IDLP) problems, and this relation is used to obtain properties of the optimal value functions. We focus on the general non-ergodic case, where the optimal value functions may depend on the initial condition of the system. We also obtain IDLP-based sufficient and necessary optimality conditions for optimal control problems.

We study a non-zero sum game considered on the solutions of a hybrid dynamical system that evolves in continuous time and that is subjected to abrupt changes of parameters. The changes of the parameters are synchronized with (and determined by) the changes of the states/actions of two Markov decision processes, each of which is controlled by a player that aims at minimizing his or her objective function. The lengths of the time intervals between the " jumps " of the parameters are assumed to be small. We show that an asymptotic Nash equilibrium of such hybrid game can be constructed on the basis of a Nash equilibrium of a deterministic averaged dynamic game.

We formulate and study the infinite-dimensional linear programming problem associated with the deterministic long-run average cost control problem. Along with its dual, it allows one to characterize the optimal value of this control problem. The novelty of our approach is that we focus on the general case wherein the optimal value may depend on the initial condition of the system.

It has been recently established that a deterministic infinite horizon discounted optimal control problem in discrete time is closely related to a certain infinite dimensional linear programming problem and its dual. In the present paper, we use these results to establish necessary and sufficient optimality conditions for this optimal control problem and apply them to construct a near optimal control., 25 pages

This paper is devoted to a study of infinite horizon optimal control problems with time discounting and time averaging criteria in discrete time. We establish that these problems are related to certain infinite-dimensional linear programming (IDLP) problems. We also establish asymptotic relationships between the optimal values of problems with time discounting and long-run average criteria.

A problem of minimization of L1-penalized conditional value-at-risk (CVaR) is considered. It is shown that there exists a non-negative threshold value of the penalty parameter such that the optimal value of the penalized problem is unbounded if the penalty parameter is less than the threshold value, and it is bounded if the penalty parameter is greater or equal than this value. It is established that the threshold value can be found via the solution of a linear programming problem, and, therefore, readily computable. Theoretical results are illustrated by numerical examples. Refereed/Peer-reviewed

This paper is devoted to a study of a discrete time infinite horizon optimal control problem with time discounting criterion. We introduce an infinite-dimensional linear programming (IDLP) problem closely related to this problem. We derive necessary and sufficient conditions of optimality for the optimal control problem in terms of the solution of the dual to this IDLP problem.

International audience; Linear programming formulations for the discounted and long-run average MDPs have evolved along separate trajectories. In 2006, E. Altman conjectured that the two linear programming formulations of discounted and long-run average MDPs are, most likely, a manifestation of general properties of singularly perturbed linear programs. In this note we demonstrate that this is, indeed, the case.

It has been established recently that, under mild conditions, deterministic long run average problems of optimal control are “asymptotically equivalent” to infinite-dimensional linear programming problems (LPPs) and that these LPPs can be approximated by finite-dimensional LPPs. In this paper we introduce the corresponding infinite- and finite-dimensional dual problems and study duality relationships. We also investigate the possibility of using solutions of finite-dimensional LPPs and their duals for numerical construction of the optimal controls in periodic optimization problems. The construction is illustrated with a numerical example.

Deterministic long run average problems of optimal control are ''asymptotically equivalent" to infinite-dimensional linear programming problems ( LPP ) and the latter are approximated by finite dimensional LPP. The solutions of this finite dimensional LPP can be used for numerical analysis of periodic optimization problems. In the present paper we establish the convergence of controls constructed on the basis of the solution of the finite dimensional LPP to the optimal control of a periodic optimization problem. Results are illustrated with a numerical example.