Search

Within the framework of finite-horizon optimal control problems involving nonlinear, input-affine dynamics, a connection between the costate variable and generating functions of the annihilating codistribution of the underlying Hamiltonian vector field is established. It is shown that the inverse mapping of any collection of n, such generating functions coincide, for any time and for a certain constant vector, with the costate of the optimal process. In particular, the corresponding constant vector is determined by solving a parameterized boundary value problem in the state space of the original plant alone, rather than in the extended state/costate space of the Hamiltonian dynamics.

A class of finite-horizon optimal control problems, the solution of which relies on a time-varying change of coordinates that incorporates the transition matrix of the system linearized along the current estimate of the optimal process, is studied. The transformed dynamics exhibit a constant optimal costate. Differently from existing methods that hinge upon similar tools, the proposed strategy does not require at each step the (numerical) solution of a two-point boundary value problem or of a time-varying Riccati equation, and only the solution of a linear initial value problem is needed. The method is firstly illustrated in the setting of linear dynamics and quadratic cost for which the construction permits the identification of a class of problems in which the solution to the underlying (quadratic) Differential Riccati Equation exhibit a separation between homogeneous and particular contributions.

Infinite-horizon optimal control problems for nonlinear systems are considered. Due to the nonlinear and intrinsically infinite-dimensional nature of the task, solving such optimal control problems is challenging. In this article, an exact finite-dimensional characterization of the optimal solution over the entire horizon is proposed. This is obtained via the (static) minimization of a suitably defined function of (projected) trajectories of the underlying Hamiltonian dynamics on a hypersphere of fixed radius. The result is achieved in the spirit of the so-called shooting methods by introducing, via simultaneous forward/backward propagation, an intermediate shooting point much closer to the origin, regardless of the actual initial state. A modified strategy allows one to determine an arbitrarily accurate approximate solution by means of standard gradient-descent algorithms over compact domains. Finally, to further increase the robustness of the control law, a receding-horizon architecture is envisioned by designing a sequence of shrinking hyperspheres. These aspects are illustrated by means of a benchmark numerical simulation.

The design of optimal control laws for nonlinear systems is tackled without knowledge of the underlying plant and of a functional description of the cost function. The proposed data-driven method is based only on real-time measurements of the state of the plant and of the (instantaneous) value of the reward signal and relies on a combination of ideas borrowed from the theories of optimal and adaptive control problems. As a result, the architecture implements a policy iteration strategy in which, hinging on the use of neural networks, the policy evaluation step and the computation of the relevant information instrumental for the policy improvement step are performed in a purely continuous-time fashion. Furthermore, the desirable features of the design method, including convergence rate and robustness properties, are discussed. Finally, the theory is validated via two benchmark numerical simulations.

Two data-driven strategies for value iteration in linear quadratic optimal control problems over an infinite horizon are proposed. The two architectures share common features, since they both consist of a purely continuous-time control architecture and are based on the forward integration of the differential Riccati equation (DRE). They profoundly differ, instead, in the estimation mechanism of the vector field of the underlying DRE from collected data: The first relies on a characterization of properties of the advantage function associated to the problem, whereas the second is inspired by tools from adaptive control theory and ensures semi-global exponential convergence to the optimal solution. Advantages and drawbacks of the architectures are discussed, while the performance is validated via a benchmark numerical example.