Search

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

In this paper, we develop an energy-based dynamical system model driven by a Markov input process to present a unified framework for stochastic thermodynamics predicated on a stochastic dynamical systems formalism. Specifically, using a stochastic dissipativity, losslessness and accumulativity theory, we develop a nonlinear stochastic port-Hamiltonian system model characterized by energy conservation and entropy non-conservation laws that are consistent with statistical thermodynamic principles. In particular, we show that the difference between the average stored system energy and the average supplied system energy for our stochastic thermodynamic model is a martingale with respect to the system filtration, whereas the difference between average system entropy production and the average system entropy consumption is a submartingale with respect to the system filtration. This article is part of the theme issue 'Thermodynamics 2.0: Bridging the natural and social sciences (Part 2)'. [ABSTRACT FROM AUTHOR]

In this paper, we derive stability margins for optimal and inverse optimal stochastic feedback regulators. Specifically, gain, sector, and disk margin guarantees are obtained for discrete-time nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal controllers that minimise a nonlinear-nonquadratic performance criterion. Furthermore, using the newly developed notion of stochastic dissipativity we derive a return difference inequality to provide connections between stochastic dissipativity and optimality of nonlinear controllers for discrete-time stochastic dynamical systems. In particular, using extended Kalman-Yakubovich-Popov conditions characterising stochastic dissipativity we show that our optimal feedback control law satisfies a return difference inequality predicated on a difference operator of a controlled Markov dispersion process and is stochastically dissipative with respect to a specific quadratic supply rate. [ABSTRACT FROM AUTHOR]

In this paper, we develop stochastic dissipativity theory for discrete‐time nonlinear dynamical systems expressed by Itô‐type difference equations using basic state properties. Specifically, a stochastic version of dissipativity and losslessness using a state dissipation inequality in expectation for controlled Markov chains is presented. The results are then used to derive extended Kalman–Yakubovich–Popov conditions for characterizing stochastic dissipativity and losslessness in terms of the nonlinear system functions using continuous storage functions. Finally, feedback interconnection stability in probability results are developed generalizing the small gain and positivity theorems to discrete‐time stochastic systems. [ABSTRACT FROM AUTHOR]

In this paper, we develop a covariance control problem to address a tradeoff between H 2 performance and H ∞ disturbance attenuation. In particular, we formulate a mixed-norm H 2 / H ∞ and entropy covariance control problem that guarantees that the state covariance of an uncertain dynamical system driven by white noise is upper bounded in the sense of the cone of nonnegative definite matrices by a given threshold matrix via state feedback as well as output feedback control. This is accomplished by combining H 2 covariance control theory and mixed norm H 2 / H ∞ control theory. By using suitable transformations involving dynamic weighting on the complimentary sensitivity system transfer function, the proposed formulation is applicable to robustness problems involving nominal performance subject to a robust stability requirement. The proposed formulation allows for solutions via semidefinite programming. Finally, two illustrative numerical examples are provided to show the efficacy of the proposed approach. [ABSTRACT FROM AUTHOR]

In this article, we investigate the role of Lyapunov functions in evaluating nonlinear–nonquadratic cost functionals for Itô‐type nonlinear stochastic difference equations. Specifically, it is shown that the cost functional can be evaluated in closed‐form as long as the cost functional is related in a specific way to an underlying Lyapunov function that guarantees asymptotic stability in probability. This result is then used to analyze discrete‐time linear as well as nonlinear stochastic dynamical systems with polynomial and multilinear cost functionals. Furthermore, a stochastic optimal control framework is developed by exploiting connections between stochastic Lyapunov theory and stochastic Bellman theory. In particular, we show that asymptotic and geometric stability in probability of the closed‐loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic stability and optimality. [ABSTRACT FROM AUTHOR]