Search

This paper investigates the use of lattice point sets as an efficient method to sample uniformly the state space of discrete-time dynamic systems for the solution of finite-horizon optimal control problems using approximate dynamic programming. Lattice point sets are a kind of discretization method, commonly employed for efficient numerical integration, providing a regular and balanced sampling of the state space based on the repetition of elementary unit cells. A convergence analysis of the approximate solution of the control problem to the optimal one is provided, pointing out that such sampling schemes allow one to efficiently exploit possible regularities of the cost-to-go functions. Furthermore, it is shown that a higher accuracy may be obtained through suitable transformations of the state vector of the dynamic system. Another advantage of lattice point sets over other sampling schemes is the possibility of evaluating a priori the goodness of a given set over another through the explicit computation of a specific parameter. Simulation results concerning the optimal control of a water reservoirs system are presented to show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Approximate dynamic programming (ADP) is the standard tool to solve Markovian decision problems under general hypotheses on the system and the cost equations. It is known that one of the key issues of the procedure is how to generate an efficient sampling of the state space, needed for the approximation of the value function, in order to cope with the well-known phenomenon of the curse of dimensionality. The most common approaches in the literature are either aimed at a uniform covering of the state space or driven by the actual evolution of the system trajectories. Concerning the latter approach, F -discrepancy, a quantity closely related to the Kolmogorov–Smirnov statistic, that measures how strictly a set of random points represents a probability distribution, has been recently proposed for an efficient ADP framework in the finite-horizon case. In this paper, we extend this framework to infinite-horizon discounted problems, providing a constructive algorithm to generate efficient sampling points driven by the system behavior. Then, the algorithm is refined with the aim of acquiring a more balanced covering of the state space, thus addressing possible drawbacks of a pure system-driven sampling approach to obtain, in fact, an efficient hybrid between the latter and the pure uniform design. A theoretical analysis is provided through the introduction of an original notion of the F -discrepancy and the proof of its properties. Simulation tests are provided to showcase the behavior of the proposed sampling method. [ABSTRACT FROM PUBLISHER]

In this paper, we address the problem of generating efficient state sample points for the solution of continuous-state finite-horizon Markovian decision problems through approximate dynamic programming. It is known that the selection of sampling points at which the value function is observed is a key factor when such function is approximated by a model based on a finite number of evaluations. A standard approach consists in generating these points through a random or deterministic procedure, aiming at a balanced covering of the state space. Yet, this solution may not be efficient if the state trajectories are not uniformly distributed. Here, we propose to exploit F -discrepancy, a quantity that measures how closely a set of random points represents a probability distribution, and introduce an example of an algorithm based on such concept to automatically select point sets that are efficient with respect to the underlying Markovian process. An error analysis of the approximate solution is provided, showing how the proposed algorithm enables convergence under suitable regularity hypotheses. Then, simulation results are provided concerning an inventory forecasting test problem. The tests confirm in general the important role of F -discrepancy, and show how the proposed algorithm is able to yield better results than uniform sampling, using sets even 50 times smaller. [ABSTRACT FROM PUBLISHER]

A methodology to design state estimators for a class of nonlinear continuous-time dynamic systems that is based on neural networks and nonlinear programming is proposed. The estimator has the structure of a Luenberger observer with a linear gain and a parameterized (in general, nonlinear) function, whose argument is an innovation term representing the difference between the current measurement and its prediction. The problem of the estimator design consists in finding the values of the gain and of the parameters that guarantee the asymptotic stability of the estimation error. Toward this end, if a neural network is used to take on this function, the parameters (i.e., the neural weights) are chosen, together with the gain, by constraining the derivative of a quadratic Lyapunov function for the estimation error to be negative definite on a given compact set. It is proved that it is sufficient to impose the negative definiteness of such a derivative only on a suitably dense grid of sampling points. The gain is determined by solving a Lyapunov equation. The neural weights are searched for via non-linear programming by minimizing a cost penalizing grid-point constraints that are not satisfied. Techniques based on low-discrepancy sequences are applied to deal with a small number of sampling points, and, hence, to reduce the computational burden required to optimize the parameters. Numerical results are reported and comparisons with those obtained by the extended Kalman filter are made. [ABSTRACT FROM AUTHOR]