Search

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

In this paper, we develop an energy-based dynamical system model driven by a Markov input process to present a unified framework for stochastic thermodynamics predicated on a stochastic dynamical systems formalism. Specifically, using a stochastic dissipativity, losslessness and accumulativity theory, we develop a nonlinear stochastic port-Hamiltonian system model characterized by energy conservation and entropy non-conservation laws that are consistent with statistical thermodynamic principles. In particular, we show that the difference between the average stored system energy and the average supplied system energy for our stochastic thermodynamic model is a martingale with respect to the system filtration, whereas the difference between average system entropy production and the average system entropy consumption is a submartingale with respect to the system filtration. This article is part of the theme issue 'Thermodynamics 2.0: Bridging the natural and social sciences (Part 2)'. [ABSTRACT FROM AUTHOR]

In this paper, we derive stability margins for optimal and inverse optimal stochastic feedback regulators. Specifically, gain, sector, and disk margin guarantees are obtained for discrete-time nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal controllers that minimise a nonlinear-nonquadratic performance criterion. Furthermore, using the newly developed notion of stochastic dissipativity we derive a return difference inequality to provide connections between stochastic dissipativity and optimality of nonlinear controllers for discrete-time stochastic dynamical systems. In particular, using extended Kalman-Yakubovich-Popov conditions characterising stochastic dissipativity we show that our optimal feedback control law satisfies a return difference inequality predicated on a difference operator of a controlled Markov dispersion process and is stochastically dissipative with respect to a specific quadratic supply rate. [ABSTRACT FROM AUTHOR]

In this paper, we develop stochastic dissipativity theory for discrete‐time nonlinear dynamical systems expressed by Itô‐type difference equations using basic state properties. Specifically, a stochastic version of dissipativity and losslessness using a state dissipation inequality in expectation for controlled Markov chains is presented. The results are then used to derive extended Kalman–Yakubovich–Popov conditions for characterizing stochastic dissipativity and losslessness in terms of the nonlinear system functions using continuous storage functions. Finally, feedback interconnection stability in probability results are developed generalizing the small gain and positivity theorems to discrete‐time stochastic systems. [ABSTRACT FROM AUTHOR]

In this paper, we develop a covariance control problem to address a tradeoff between H 2 performance and H ∞ disturbance attenuation. In particular, we formulate a mixed-norm H 2 / H ∞ and entropy covariance control problem that guarantees that the state covariance of an uncertain dynamical system driven by white noise is upper bounded in the sense of the cone of nonnegative definite matrices by a given threshold matrix via state feedback as well as output feedback control. This is accomplished by combining H 2 covariance control theory and mixed norm H 2 / H ∞ control theory. By using suitable transformations involving dynamic weighting on the complimentary sensitivity system transfer function, the proposed formulation is applicable to robustness problems involving nominal performance subject to a robust stability requirement. The proposed formulation allows for solutions via semidefinite programming. Finally, two illustrative numerical examples are provided to show the efficacy of the proposed approach. [ABSTRACT FROM AUTHOR]

In this article, we investigate the role of Lyapunov functions in evaluating nonlinear–nonquadratic cost functionals for Itô‐type nonlinear stochastic difference equations. Specifically, it is shown that the cost functional can be evaluated in closed‐form as long as the cost functional is related in a specific way to an underlying Lyapunov function that guarantees asymptotic stability in probability. This result is then used to analyze discrete‐time linear as well as nonlinear stochastic dynamical systems with polynomial and multilinear cost functionals. Furthermore, a stochastic optimal control framework is developed by exploiting connections between stochastic Lyapunov theory and stochastic Bellman theory. In particular, we show that asymptotic and geometric stability in probability of the closed‐loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic stability and optimality. [ABSTRACT FROM AUTHOR]

Dissipative dynamical systems provide fundamental connections between physics, dynamical systems theory, and control science and engineering. In the deterministic setting, dissipativity theory has been extensively developed in the literature to provide a general framework for the analysis and design of control systems using an input-state-output system description based on generalized system energy considerations that uses a state–space formalism to link engineering systems with memory to well known physical phenomena. Recently, several results have appeared in the literature extending dissipativity notions to the stochastic setting in order to develop an analogous theory for stochastic dynamical systems. However, unlike the deterministic theory, which can involve either an energy balance or a power balance state dissipation inequality for characterizing system dissipativity, in the stochastic case this equivalence is far more nuanced. In this paper, we develop a general theory for stochastic dissipativity and present general conditions on the system drift and diffusion functions as well as the system energy storage and supply rates to provide an equivalence between the sample path dependent energetic (i.e., supermartingale) and the power balance (i.e., algebraic) forms for characterizing stochastic dissipativity. Specifically, for a given class of admissible system inputs guaranteeing the existence and uniqueness of a solution to an open input-state-output stochastic dynamical system driven by a Brownian motion process we provide sufficient regularity conditions for establishing the equivalence between the energetic and algebraic power balance definitions of stochastic dissipativity involving an inequality between the infinitesimal generator of the system storage function and the system supply rate. [ABSTRACT FROM AUTHOR]

Quadcopters are increasingly used for applications ranging from hobby to industrial products and services. This paper serves as a tutorial on the design, simulation, implementation, and experimental outdoor testing of digital quadcopter flight controllers, including Explicit Model Predictive Control, Linear Quadratic Regulator, and Proportional Integral Derivative. A quadcopter was flown in an outdoor testing facility and made to track an inclined, circular path at different tangential velocities under ambient wind conditions. Controller performance was evaluated via multiple metrics, such as position tracking error, velocity tracking error, and onboard computation time. Challenges related to the use of computationally limited embedded hardware and flight in an outdoor environment are addressed with proposed solutions. [ABSTRACT FROM AUTHOR]