Search

A class of nonlinear finite-horizon optimal control problems is studied. We propose a solution based on an iterative strategy that relies on the linearization of the nonlinear dynamics and on the construction of the corresponding time-varying Hamiltonian dynamics. Differently from existing methods that hinge upon similar tools, the proposed strategy hinges upon the solution to a linear initial value problem and does not require at each step the (numerical) solution of a two-point boundary value problem or of a time-varying Riccati equation. The result is achieved by exploiting a time-varying change of coordinates with the objective of obtaining a constant optimal costate in the transformed coordinates.

The infinite-horizon optimal control problem for nonlinear systems is studied. In the context of model-based, iterative learning strategies we propose an alternative definition and construction of the temporal difference error arising in policy iteration strategies. In such architectures, the error is computed via the evolution of the Hamiltonian function (or, possibly, of its integral) along the trajectories of the closed-loop system. Herein the temporal difference error is instead obtained via two subsequent steps: first the dynamics of the underlying costate variable in the Hamiltonian system is steered by means of a (virtual) control input in such a way that the stable invariant manifold becomes externally attractive. Then, the distance-from-invariance of the manifold, induced by approximate solutions, yields a natural candidate measure for the policy evaluation step. The policy improvement phase is then performed by means of standard gradient descent methods that allows us to correctly update the weights of the underlying functional approximator. The above-mentioned architecture then yields an iterative (episodic) learning scheme based on a scalar, constant reward at each iteration, the value of which is insensitive to the length of the episode, as in the original spirit of reinforcement learning strategies for discrete-time systems. Finally, the theory is validated by means of a numerical simulation involving an automatic flight control problem.

A fixed-point characterization of the optimal costate in finite-horizon optimal control problems for nonlinear systems is presented. It is shown that the optimal initial condition of the costate variable must be a fixed-point, for any time, of the composition of the forward and backward flows of the underlying Hamiltonian dynamics. Such an abstract property is then translated into a constructive condition by relying on a sequence of repeated Lie brackets involving the Hamiltonian dynamics and evaluated at a single point in the state space. This leads to a system of algebraic equations in the unknown initial optimal costate that allows achieving a desired degree of accuracy of the approximation while always consisting of a number of equations equal to the dimension of the state of the underlying system, regardless of the achieved accuracy. A dual characterization of the optimal terminal value of the state is also discussed, together with a few computational aspects of the proposed strategy.

To study optimal control and disturbance attenuation problems, two prominent—and somewhat alternative—strategies have emerged in the last century: dynamic programming (DP) and Pontryagin's minimum principle (PMP). The former characterizes the solution by shaping the dynamics in a closed loop ( a priori unknown) via the selection of a feedback input, at the price, however, of the solution to (typically daunting) partial differential equations. The latter, instead, provides (extended) dynamics that must be satisfied by the optimal process, for which boundary conditions ( a priori unknown) should be determined. The results discussed in this article combine the two approaches by matching the corresponding trajectories, i.e., combining the underlying sources of information: knowledge of the complete initial condition from DP and of the optimal dynamics from PMP. The proposed approach provides insights for linear as well as nonlinear systems. In the case of linear systems, the derived conditions lead to matrix algebraic equations, similar to the classic algebraic Riccati equations (AREs), although with coefficients defined as polynomial functions of the input gain matrix, with the property that the coefficient of the quadratic term of such equation is sign definite , even if the corresponding coefficient of the original ARE is sign indefinite , as it is typically the case in theH∞control problem. This feature is particularly appealing from the computational point of view, since it permits the use of standard minimization techniques for convex functions, such as the gradient algorithm. In the presence of nonlinear dynamics, the strategy leads to algebraic equations that allow us to (locally) construct the optimal feedback by considering the behavior of the closed-loop dynamics at a single point in the state space.

We study the infinite-horizon optimal control problem for nonlinear, multi-input, input-quadratic systems. It is shown that optimality of the input-quadratic closed-loop system is intimately related to the property that an auxiliary input-affine system possesses a L 2 -gain smaller than one. Such equivalence is established, or approximated, by relying on (a combination of) three alternative sets of technical conditions based (i) on the inclusion of the gradient of the underlying storage function in a certain co-distribution, (ii) on verifying specific algebraic inequalities, (iii) or achieved dynamically by considering the immersion of the original nonlinear plant into a system defined on an augmented state-space.

A model matching and passivating control architecture for multi-input/multi-output linear systems, comprising dynamic feedback and feedforward, is proposed. The approach-essentially without any restriction on the relative degree and the zeros of the underlying system and by relying only on input/output measurements-provides a closed-loop system, the transfer matrix of which matches any desired matrix of rational functions. An alternative implementation of the above design allows to achieve an arbitrary approximation accuracy of a desired transfer matrix while also preserving structural properties-in particular observability-of the overall interconnected system. Such a construction can be then specialized to provide input/output decoupling or a system that is passive from a novel control input to a modified output. The result is achieved by arbitrarily assigning the relative degree and location of the poles and zeros on the complex plane of the interconnected system in a systematic way. It is also shown that similar ideas can be employed to enforce a desired, arbitrarily small, L 2 -gain from an unknown disturbance input to a modified output, while preserving the corresponding gain from the control input to the same output. The article is concluded with applications and further discussions on the results.