Search

A class of nonlinear finite-horizon optimal control problems is studied. We propose a solution based on an iterative strategy that relies on the linearization of the nonlinear dynamics and on the construction of the corresponding time-varying Hamiltonian dynamics. Differently from existing methods that hinge upon similar tools, the proposed strategy hinges upon the solution to a linear initial value problem and does not require at each step the (numerical) solution of a two-point boundary value problem or of a time-varying Riccati equation. The result is achieved by exploiting a time-varying change of coordinates with the objective of obtaining a constant optimal costate in the transformed coordinates.

The infinite-horizon optimal control problem for nonlinear systems is studied. In the context of model-based, iterative learning strategies we propose an alternative definition and construction of the temporal difference error arising in policy iteration strategies. In such architectures, the error is computed via the evolution of the Hamiltonian function (or, possibly, of its integral) along the trajectories of the closed-loop system. Herein the temporal difference error is instead obtained via two subsequent steps: first the dynamics of the underlying costate variable in the Hamiltonian system is steered by means of a (virtual) control input in such a way that the stable invariant manifold becomes externally attractive. Then, the distance-from-invariance of the manifold, induced by approximate solutions, yields a natural candidate measure for the policy evaluation step. The policy improvement phase is then performed by means of standard gradient descent methods that allows us to correctly update the weights of the underlying functional approximator. The above-mentioned architecture then yields an iterative (episodic) learning scheme based on a scalar, constant reward at each iteration, the value of which is insensitive to the length of the episode, as in the original spirit of reinforcement learning strategies for discrete-time systems. Finally, the theory is validated by means of a numerical simulation involving an automatic flight control problem.

Open-loop Nash equilibrium strategies that admit a feedback synthesis in Linear–Quadratic (LQ) games are studied. A characterization alternative to the classic system of coupled (asymmetric) Riccati equations – one for each player – is provided by relying on a fixed-point argument based on the composition of flows of the underlying state/costate dynamics. As a result, it is shown that in competitive games, namely games in which the players influence the shared state via linearly independent input channels, the characterization of Nash equilibrium strategies hinges upon the solution to a single (regardless of the number of players), sign-definite Riccati equation, with coefficients described by polynomial functions of the feedback gains. The structure of the latter equation is computationally appealing since it naturally allows for gradient-descent algorithms on matrix manifolds, thus ensuring (local) guaranteed convergence to the equilibrium strategy. In the case of antagonistic games, namely games in which the players may share linearly dependent input directions, the fixed-point condition above is combined with a geometric requirement involving the largest invariant subspace contained in the kernel of an auxiliary output matrix. Finally, by building on the latter characterization it is shown that closed-form expressions for the equilibrium strategy for a class of dynamic games can be given.

A fixed-point characterization of the optimal costate in finite-horizon optimal control problems for nonlinear systems is presented. It is shown that the optimal initial condition of the costate variable must be a fixed-point, for any time, of the composition of the forward and backward flows of the underlying Hamiltonian dynamics. Such an abstract property is then translated into a constructive condition by relying on a sequence of repeated Lie brackets involving the Hamiltonian dynamics and evaluated at a single point in the state space. This leads to a system of algebraic equations in the unknown initial optimal costate that allows achieving a desired degree of accuracy of the approximation while always consisting of a number of equations equal to the dimension of the state of the underlying system, regardless of the achieved accuracy. A dual characterization of the optimal terminal value of the state is also discussed, together with a few computational aspects of the proposed strategy.

To study optimal control and disturbance attenuation problems, two prominent—and somewhat alternative—strategies have emerged in the last century: dynamic programming (DP) and Pontryagin's minimum principle (PMP). The former characterizes the solution by shaping the dynamics in a closed loop ( a priori unknown) via the selection of a feedback input, at the price, however, of the solution to (typically daunting) partial differential equations. The latter, instead, provides (extended) dynamics that must be satisfied by the optimal process, for which boundary conditions ( a priori unknown) should be determined. The results discussed in this article combine the two approaches by matching the corresponding trajectories, i.e., combining the underlying sources of information: knowledge of the complete initial condition from DP and of the optimal dynamics from PMP. The proposed approach provides insights for linear as well as nonlinear systems. In the case of linear systems, the derived conditions lead to matrix algebraic equations, similar to the classic algebraic Riccati equations (AREs), although with coefficients defined as polynomial functions of the input gain matrix, with the property that the coefficient of the quadratic term of such equation is sign definite , even if the corresponding coefficient of the original ARE is sign indefinite , as it is typically the case in theH∞control problem. This feature is particularly appealing from the computational point of view, since it permits the use of standard minimization techniques for convex functions, such as the gradient algorithm. In the presence of nonlinear dynamics, the strategy leads to algebraic equations that allow us to (locally) construct the optimal feedback by considering the behavior of the closed-loop dynamics at a single point in the state space.

We study the infinite-horizon optimal control problem for nonlinear, multi-input, input-quadratic systems. It is shown that optimality of the input-quadratic closed-loop system is intimately related to the property that an auxiliary input-affine system possesses a L 2 -gain smaller than one. Such equivalence is established, or approximated, by relying on (a combination of) three alternative sets of technical conditions based (i) on the inclusion of the gradient of the underlying storage function in a certain co-distribution, (ii) on verifying specific algebraic inequalities, (iii) or achieved dynamically by considering the immersion of the original nonlinear plant into a system defined on an augmented state-space.

A model matching and passivating control architecture for multi-input/multi-output linear systems, comprising dynamic feedback and feedforward, is proposed. The approach-essentially without any restriction on the relative degree and the zeros of the underlying system and by relying only on input/output measurements-provides a closed-loop system, the transfer matrix of which matches any desired matrix of rational functions. An alternative implementation of the above design allows to achieve an arbitrary approximation accuracy of a desired transfer matrix while also preserving structural properties-in particular observability-of the overall interconnected system. Such a construction can be then specialized to provide input/output decoupling or a system that is passive from a novel control input to a modified output. The result is achieved by arbitrarily assigning the relative degree and location of the poles and zeros on the complex plane of the interconnected system in a systematic way. It is also shown that similar ideas can be employed to enforce a desired, arbitrarily small, L 2 -gain from an unknown disturbance input to a modified output, while preserving the corresponding gain from the control input to the same output. The article is concluded with applications and further discussions on the results.

The infinite-horizon optimal control problem with stability in the presence of single-input, input-quadratic nonlinear systems is addressed and tackled in this article. In addition, it is shown that similar ideas can be extended to study the property of passivity of the underlying input-quadratic system from a given output. The constructive design of the optimal solution revolves around the interesting fact that the property of optimality of the closed-loop underlying system is shown to be locally equivalent to the property that an input-affine system possesses an L 2 -gain less than one from a virtual disturbance signal. The global version of the statement requires a technical condition on the graph of the storage function of the latter auxiliary plant, and hence leads to the new notion of graphical storage function. Finally, the theory is corroborated by the application to the optimal control of the movable plane positioning in micromechanical systems actuators.

Dynamic optimisation, with a particular focus on optimal control and nonzero-sum differential games, is considered. For nonlinear systems solutions sought via the dynamic programming strategy are inevitably characterised by partial differential equations (PDEs) which are often difficult to solve. A detailed overview of a control design framework which enables the systematic construction of approximate solutions for optimal control problems and differential gameswithoutrequiring theexplicitsolution of any PDE is provided along with a novel design of a nonlinear control gain aimed at improving the ‘level of approximation’ achieved. Multi-agent systems are considered as a possible application of the theory.

A separation principle for a class of nonlinear systems inspired by the techniques of feedback linearizationand observer design with linear error dynamics is discussed. The output feedback construction combines strategies for approximate feedback linearization and observer design, which are of interest per se, yielding a dynamic control law that ensures a linear, spectrally assignable, behavior from the certainty equivalence input mismatch to the extended state of the system and the observer. The first ingredient, namely the approximate feedback linearization strategy, can be applied, under mild conditions, also to nonlinear systems that are linearly uncontrollable - or that do not possess a well-defined relative degree in the case of a given output function -yet providing a chain of integrators of length equal to the dimension of the state in the transformed coordinates. Interestingly, a systematically designed nonlinear inner loop enables use of linear design techniques, e.g. pole placement. The observer design, on the other hand, employs an additional dynamic extension that allows to assign the local dynamic behavior of the error dynamics independently from its zeros, differently from the classic high-gain observer design. The paper is concluded by presenting several numerical simulations, including an output tracking control problem for the Ball and Beam model that does not possess a well-defined relative degree.