Search

In this paper, exponential stability of discrete-time Markov jump linear systems (MJLSs) with the Markov chain on a Borel space $(\Theta, \mathcal{B}(\Theta))$ is studied, and bounded real lemmas (BRLs) are given. The work generalizes the results from the previous literature that considered only the Markov chain taking values in a countable set to the scenario of an uncountable set and provides unified approaches for describing exponential stability and $H_{\infty}$ performance of MJLSs. This paper covers two kinds of exponential stabilities: one is exponential mean-square stability with conditioning (EMSSy-C), and the other is exponential mean-square stability (EMSSy). First, based on the infinite-dimensional operator theory, the equivalent conditions for determining these two kinds of stabilities are shown respectively by the exponentially stable evolutions generated by the corresponding bounded linear operators on different Banach spaces, which turn out to present the spectral criteria of EMSSy-C and EMSSy. Furthermore, the relationship between these two kinds of stabilities is discussed. Moreover, some easier-to-check criteria are established for EMSSy-C of MJLSs in terms of the existence of uniformly positive definite solutions of Lyapunov-type equations or inequalities. In addition, BRLs are given separately in terms of the existence of solutions of the $\Theta$-coupled difference Riccati equation for the finite horizon case and algebraic Riccati equation for the infinite horizon case, which facilitates the $H_{\infty}$ analysis of MJLSs with the Markov chain on a Borel space.

The stochastic $H_{\infty}$ control is studied for a linear stochastic It\^o system with an unknown system model. The linear stochastic $H_{\infty}$ control issue is known to be transformable into the problem of solving a so-called generalized algebraic Riccati equation (GARE), which is a nonlinear equation that is typically difficult to solve analytically. Worse, model-based techniques cannot be utilized to approximately solve a GARE when an accurate system model is unavailable or prohibitively expensive to construct in reality. To address these issues, an off-policy reinforcement learning (RL) approach is presented to learn the solution of a GARE from real system data rather than a system model; its convergence is demonstrated, and the robustness of RL to errors in the learning process is investigated. In the off-policy RL approach, the system data may be created with behavior policies rather than the target policies, which is highly significant and promising for use in actual systems. Finally, the proposed off-policy RL approach is validated on a stochastic linear F-16 aircraft system.

This paper discusses the stabilizability, weak stabilizability, exact observability and robust quadratic stabilizability of linear stochastic control systems. By means of the spectrum technique of the generalized Lyapunov operator, a necessary and sufficient condition is given for stabilizability and weak stabilizability of stochastic systems, respectively. Some new concepts called unremovable spectrums, strong solutions, and weakly feedback stabilizing solutions are introduced. An unremovable spectrum theorem is given, which generalizes the corresponding theorem of deterministic systems to stochastic systems. A stochastic Popov-Belevith-Hautus (PBH) criterion for exact observability is obtained. For applications, we give a comparison theorem for generalized algebraic Riccati equations (GAREs), and two results on Lyapunov-type equations are obtained, which improve the previous works. Finally, we also discuss robust quadratic stabilization of uncertain stochastic systems, and a necessary and sufficient condition is given for quadratic stabilization via a linear matrix inequality (LMI).

This paper studies finite-time stability and instability theorems in probability sense for stochastic nonlinear systems. Firstly, a new sufficient condition is proposed to guarantee that the considered system has a global solution. Secondly, we propose improved finite-time stability and instability criteria that relax the constraints on $\mathcal {L}V$ (the infinitesimal operator of Lyapunov function $V$) by the uniformly asymptotically stable function(UASF). The improved finite-time stability theorems allow $\mathcal {L}V$ to be indefinite (negative or positive) rather than just only allow $\mathcal {L}V$ to be negative. Most existing finite-time stability and instability results can be viewed as special cases of the obtained theorems. Finally, some simulation examples verify the validity of the theoretical results.