Search

This paper investigates the stochastic stability analysis and stabilization problems for discrete-time Takagi-Sugeno fuzzy semi-Markov jump systems with upper-bounded sojourn time. The fuzzy rules can be different for different system modes. Consequently, the membership functions for fuzzy rules are dependent on the system modes. Allowing for the fact that semi-Markov kernel (SMK) are difficult to fully obtain in practice, the elements in the SMK of the underlying systems are deemed to be partly known, which is more general than both semi-Markov jump systems with completely available semi-Markov kernel and Markov jump systems with unknown transition probabilities. Afterwards, the stability and stabilization conditions are established by part of the known SMK information and then by all the known SMK information. In the end, the validity and the superiority of our proposed theoretical results are exemplified via a single-link robot arm and a truck-trailer model.

This paper concentrates on the $l_{2}-l_{\infty }$ filtering problem of discrete-time fuzzy semi-Markov jump nonlinear system. The random jumps among the studied system are governed by the discrete-time semi-Markov process. Therefore, the storage characteristics of the transition probability between systems are fully considered. To analyze $\sigma$ -mean-square stability of the filtering error system, a filter, which is mode-dependent, based on the discrete-time fuzzy semi-Markov jump model is constructed to estimate the state of the system. Thereafter, based on the Lyapunov stability theory and the discrete-time semi-Markov kernel concept, a set of criteria to ensure the are proposed. In addition, by using Takagi-Sugeno fuzzy model, the nonlinear problem is effectively solved. Ultimately, an illustrative example of a tunnel diode circuit and a numerical example effectively and practicability reveal the developed filtering strategy.

This article is concerned with the problem of the static output-feedback control for a class of discrete-time linear semi-Markov jump systems (SMJSs). Through a mode-dependent resilient control scheme and an invertible linear transformation, a resulting equivalent closed-loop system can be obtained. The embedded Markov chain (EMC) is piecewise homogeneous, which leads to incomplete semi-Markov kernel is variable in the finite interval. A novel class of multivariate dependent Lyapunov function is constructed, which is mode-dependent, elapsed-time-dependent, and variation-dependent. Numerically testable stabilization criteria are established for discrete-time linear SMJSs via abovementioned Lyapunov function. Under bound sojourn time, a desired stabilizing controller is designed such that the closed-loop system is mean-square stable. Finally, the theoretical results are applied to a practical RLC circuit system to show the effectiveness and applicability of the proposed control strategy.

We consider integer‐valued processes with a linear or nonlinear generalized autoregressive conditional heteroscedastic models structure, where the count variables given the past follow a Poisson distribution. We show that a contraction condition imposed on the intensity function yields a contraction property of the Markov kernel of the process. This allows almost effortless proofs of the existence and uniqueness of a stationary distribution as well as of absolute regularity of the count process. As our main result, we construct a coupling of the original process and a model‐based bootstrap counterpart. Using a contraction property of the Markov kernel of the coupled process we obtain bootstrap consistency for different types of statistics.

In this paper, the $\mathcal {H}_{\infty }$ stabilization problem is studied for discrete-time semi-Markov jump singularly perturbed systems (SMJSPSs) with repeated scalar nonlinearities. As the exact statistical information of the sojourn time or the mode transition is difficult to obtain, the case with only partial semi-Markov kernel information available is considered. Furthermore, introducing an external disturbance or nonlinearity into the analysis of discrete-time semi-Markov jump systems (DTSMJSs) meets critical obstacles, since the relation between the system state vectors at two nonadjacent instants is difficult to determine. To address this issue, the variation trend of the Lyapunov function for a semi-Markov jump sequence is analyzed in detail. Subsequently, criteria of mean-square exponential stability (MSES) for DTSMJSs are established for the first time based on the Lyapunov stability theory. By virtue of the criteria obtained and the cone complementary linearization algorithm, a controller ensuring MSES and $\mathcal {H}_{\infty }$ performance for discrete-time nonlinear SMJSPSs is constructed. Finally, the effectiveness and applicability of the proposed method are validated by simulation examples including an inverted pendulum model.

We show how to speed up sequential Monte Carlo (SMC) for Bayesian inference in large data problems by data subsampling. SMC sequentially updates a cloud of particles through a sequence of distributions, beginning with a distribution that is easy to sample from such as the prior and ending with the posterior distribution. Each update of the particle cloud consists of three steps: reweighting, resampling, and moving. In the move step, each particle is moved using a Markov kernel; this is typically the most computationally expensive part, particularly when the dataset is large. It is crucial to have an efficient move step to ensure particle diversity. Our article makes two important contributions. First, in order to speed up the SMC computation, we use an approximately unbiased and efficient annealed likelihood estimator based on data subsampling. The subsampling approach is more memory efficient than the corresponding full data SMC, which is an advantage for parallel computation. Second, we use a Metropolis within Gibbs kernel with two conditional updates. A Hamiltonian Monte Carlo update makes distant moves for the model parameters, and a block pseudo-marginal proposal is used for the particles corresponding to the auxiliary variables for the data subsampling. We demonstrate both the usefulness and limitations of the methodology for estimating four generalized linear models and a generalized additive model with large datasets.

This work suggests a solution for joint input-state estimation for nonlinear systems. The task is to recover the internal states of a nonlinear oscillator, the displacement and velocity of the system, and the unmeasured external forces applied. To do this, a Gaussian process latent force model is developed for nonlinear systems. The model places a Gaussian process prior over the unknown input forces for the system, converts this into a state-space form and then augments the nonlinear system with these additional hidden states. To perform inference over this nonlinear state-space model a particle Gibbs approach is used combining a &ldquo, Particle Gibbs with Ancestor Sampling&rdquo, Markov kernel for the states and a Metropolis-Hastings update for the hyperparameters of the Gaussian process. This approach is shown to be effective in a numerical case study on a Duffing oscillator where the internal states and the unknown forcing are recovered, each with a normalised mean-squared error less than 0.5%. It is also shown how this Bayesian approach allows uncertainty quantification of the estimates of the states and inputs which can be invaluable in further engineering analyses.

Perron–Frobenius theory developed for irreducible non-negative kernels deals with so-called R-positive recurrent kernels. If the kernel M is R-positive recurrent, then the main result determines the limit of the scaled kernel iterations $R^nM^n$ as $n\to\infty$ . In Nummelin (1984) this important result is proven using a regeneration method whose major focus is on M having an atom. In the special case when $M=P$ is a stochastic kernel with an atom, the regeneration method has an elegant explanation in terms of an associated split chain. In this paper we give a new probabilistic interpretation of the general regeneration method in terms of multi-type Galton–Watson processes producing clusters of particles. Treating clusters as macro-individuals, we arrive at a single-type Crump–Mode–Jagers process with a naturally embedded renewal structure.

Aiming at the simplest possible representation of maxmin copulas in terms of two generators Kosir and Omladic recently introduced the class of so-called reflected maxmin (RMM) copulas in [10] . All results for RMM copulas can easily be translated to results on maxmin copulas. Building upon the afore-mentioned article we provide alternative simplified (mainly Markov kernel based) proofs for some of their theorems and, more importantly, derive numerous new results. In particular, we show that the family of all RMM copulas is a compact subset of the metric space of all copulas (with the standard uniform metric), characterize absolutely continuous RMM copulas, and show that the limit of absolutely continuous RMM copulas is absolutely continuous too. Finally, we determine Kendall's τ and Spearman's ρ of RMM copulas, prove some first inequalities for τ and ρ, and state a conjecture on sharp inequalities. Some examples illustrating the main results round off the paper.

We present a solution to the terminal-hitting stochastic reach-avoid problem for a Markov control process. This solution takes advantage of a nonparametric representation of the stochastic kernel as a conditional distribution embedding within a reproducing kernel Hilbert space (RKHS). Because the disturbance is modeled as a data-driven stochastic process, this representation avoids intractable integrals in the dynamic recursion of the reach-avoid problem since the expectations can be calculated as an inner product within the RKHS. We demonstrate this approach on a high-dimensional chain of integrators and on Clohessy-Wiltshire-Hill dynamics.

This paper establishes several results for uniform convergence of nonparametric kernel density and regression estimates for the case where the time series regressors concerned are nonstationary null–recurrent Markov chains. Under suitable conditions, certain rates of convergence are also established for these estimates. Our results can be viewed as an extension of some well–known uniform consistency results for the stationary time series to the nonstationary time series case.

We study notions of conditional probability and stochastic dependence/independence in an upgraded probability model in which the space of events is modeled by a full Łukasiewicz tribe of all measurable functions from some measurable space into [0, 1]. Our study is based on properties of joint experiments and the notion of stochastic channel, a construct equivalent to the notion of Markov kernel between two measurable spaces. Using the notion of a degenerated stochastic channel, a channel transmitting no stochastic information between two spaces, we define an asymmetrical independence of random experiments. Finally, we define the notion of conditional probability on full Łukasiewicz tribes.

This paper investigates the stabilization issue for a class of discrete-time stochastic switching systems. The switching behavior is dominated by a semi-Markov process with finite sojourn time. Allowing for the fact that it is often difficult to get complete semi-Markov kernel (SMK) in practice, the elements in SMK of the model under study are considered to be partly accessible, which is more general than both semi-Markov model with complete SMK and Markov model with unknown transition probabilities. Sufficient stability condition is derived for the underlying system without any a priori knowledge, based on which a stabilization criterion is presented such that the closed-loop stochastic switching systems can be mean-square stable. In the end, the validity of the theoretical results is testified by a numerical example.

The debate over standard β, σ, and club convergence is empirically re-examined in India with respect to per capita net state domestic product (PCNSDP) across 22 states over time (viz. 1980–1981 to 2015–2016). We go beyond the σ convergence, and the distribution-sensitive inequality measure such as generalized entropy is used to exploring the pattern of distribution of PCNSDP among states over time. We find σ divergence, rising trend of inequality of PCNSDP with a higher sensitivity to the right tail of the distribution. The results show that there exists absolute β divergence but conditional β convergence; the conditioning variables which appear to be significant are per capita bank deposit (PCBD) and composite physical infrastructure index. The kernel density estimation and Markov’s transitional probability matrix (TPM) show that there exists club convergence among the states; we find higher stability, but lower mobility indices during different time intervals confirm the existence of club convergence in the short run; however, the long-run TPM gives the opposite results. The long-run TPM shows that very few states are stable to their respective clubs, but most of the states have been experiencing transitional movement from higher to lower income group over time; thus, our results do not support the earlier findings of ‘missing middle’.

The objective of this study is to investigate the presence of ‘club convergence’ in respect of income among 15 major states in India during 1982–2014 using Markov chain along with stochastic kernel. The distributional dynamics observed among the major states support the process of ‘club convergence’. The empirical findings prove the hypotheses that economies that are similar in their structural characteristics and initial per capita income levels will converge with each other in per capita terms in the long run. The present study empirically re-establishes the hypotheses of ‘low-level equilibrium trap’ caused by low human capital investment. JEL: C14, H51, H52, O49, O53

Commutative POVMs are smearing of spectral measures with the smearing realized by a Markov kernel. In the case of POVMs defined on the Borel σ-algebra of the reals, the existence of a universal Markov kernel has been established. Here we show that every weak Markov kernel is functionally subordinated to the universal Markov kernel. Then, we analyze the implications of that result to the characterization of a Markov kernel as a Choquet’s integral on the space of deterministic Markov kernels.

We propose a new technique for deriving the differential privacy parameters in federated learning (FL). We consider the setting where a machine learning model is iteratively trained using stochastic gradient descent (SGD) and only the last update is publicly released. In this approach, we interpret each training iteration as a Markov kernel. We then quantify the impact of the kernel on privacy parameters via the contraction coefficient of the $E_{\gamma}$ -divergence that underlies differential privacy. To do so, we generalize the well-known Dobrushin's ergodicity coefficient, originally defined in terms of total variation distance, to a family of $f$ -divergences. We then analyze the convergence rate of SGD under the proposed private FL framework.

We present a method to compute the stochastic reachability safety probabilities for high-dimensional stochastic dynamical systems. Our approach takes advantage of a nonparametric learning technique known as conditional distribution embeddings to model the stochastic kernel using a data-driven approach. By embedding the dynamics and uncertainty within a reproducing kernel Hilbert space, it becomes possible to compute the safety probabilities for stochastic reachability problems as simple matrix operations and inner products. We employ a convergent approximation technique, random Fourier features, in order to alleviate the increased computational requirements for high-dimensional systems. This technique avoids the curse of dimensionality, and enables the computation of safety probabilities for high-dimensional systems without prior knowledge of the structure of the dynamics or uncertainty. We validate this approach on a double integrator system, and demonstrate its capabilities on a million-dimensional, nonlinear, non-Gaussian, repeated planar quadrotor system.

The problem of private data disclosure is studied from an information theoretic perspective. Considering a pair of dependent random variables $(X,Y)$, where $X$ and $Y$ denote the private and useful data, respectively, the following problem is addressed: What is the maximum information that can be revealed about $Y$ (measured by mutual information $I(Y;U)$, in which $U$ is the revealed data), while disclosing no information about $X$ (captured by the condition of statistical independence, i.e., $X\independent U$, and henceforth, called \textit{perfect privacy})? We analyze the supremization of \textit{utility}, i.e., $I(Y;U)$ under the condition of perfect privacy for two scenarios: \textit{output perturbation} and \textit{full data observation} models, which correspond to the cases where a Markov kernel, called \textit{privacy-preserving mapping}, applies to $Y$ and the pair $(X,Y)$, respectively. When both $X$ and $Y$ have a finite alphabet, the linear algebraic analysis involved in the solution provides some interesting results, such as upper/lower bounds on the size of the released alphabet and the maximum utility. Afterwards, it is shown that for the jointly Gaussian $(X,Y)$, perfect privacy is not possible in the output perturbation model in contrast to the full data observation model. Finally, an asymptotic analysis is provided to obtain the rate of released information when a sufficiently small leakage is allowed. In particular, in the context of output perturbation model, it is shown that this rate is always finite when perfect privacy is not feasible, and two lower bounds are provided for it; When perfect privacy is feasible, it is shown that under mild conditions, this rate becomes unbounded., The longer version of the journal paper accepted for publication in IEEE Journal of Selected Areas in Information Theory (JSAIT)

Motivated by the increasing popularity and the seemingly broad applicability of pair-copula constructions underlined by numerous publications in the last decade, in this contribution we tackle the unavoidable question on how flexible and simplifying the commonly used `simplifying assumption' is from an analytic perspective and provide answers to two related open questions posed by Nagler and Czado in 2016. Aiming at a simplest possible setup for deriving the main results we first focus on the three-dimensional setting. We prove that the family of simplified copulas is flexible in the sense that it is dense in the set of all three-dimensional co\-pulas with respect to the uniform metric $d_\infty$ - considering stronger notions of convergence like the one induced by the metric $D_1$, by weak conditional convergence, by total variation, or by Kullback-Leibler divergence, however, the family even turn out to be nowhere dense and hence insufficient for any kind of flexible approximation. Furthermore, returning to $d_\infty$ we show that the partial vine copula is never the optimal simplified copula approximation of a given, non-simplified copula $C$, and derive examples illustrating that the corresponding approximation error can be strikingly large and extend to more than 28\% of the diameter of the metric space. Moreover, the mapping $\psi$ assigning each three-dimensional copula its unique partial vine copula turns out to be discontinuous with respect to $d_\infty$ (but continuous with respect to $D_1$ and to weak conditional convergence), implying a surprising sensitivity of partial vine copula approximations. The afore-mentioned main results concerning $d_\infty$ are then extended to the general multivariate setting., Comment: 36 pages, 3 figures

We study the relationship between functional inequalities for a Markov kernel on a metric space $X$ and inequalities of transportation distances on the space of probability measures $\mathcal{P}(X)$. Extending results of Luise and Savar\'e on Hellinger--Kantorovich contraction inequalities for the particular case of the heat semigroup on an $RCD(K,\infty)$ metric space, we show that more generally, such contraction inequalities are equivalent to reverse Poincar\'e inequalities. We also adapt the "dynamic dual" formulation of the Hellinger--Kantorovich distance to define a new family of divergences on $\mathcal{P}(X)$ which generalize the R\'enyi divergence, and we show that contraction inequalities for these divergences are equivalent to the reverse logarithmic Sobolev and Wang Harnack inequalities. We discuss applications including results on the convergence of Markov processes to equilibrium, and on quasi-invariance of heat kernel measures in finite and infinite-dimensional groups., Comment: 38 pages. To appear in Electronic Journal of Probability

Hamiltonian Monte Carlo (HMC) is currently one of the most popular Markov Chain Monte Carlo algorithms to sample smooth distributions over continuous state space. This paper discusses the irreducibility and geometric ergodicity of the HMC algorithm. We consider cases where the number of steps of the Störmer–Verlet integrator is either fixed or random. Under mild conditions on the potential $U$ associated with target distribution $\pi$, we first show that the Markov kernel associated to the HMC algorithm is irreducible and positive recurrent. Under more stringent conditions, we then establish that the Markov kernel is Harris recurrent. We provide verifiable conditions on $U$ under which the HMC sampler is geometrically ergodic. Finally, we illustrate our results on several examples.

In this paper we consider the field of local times of a discrete-time Markov chain on a general state space, and obtain uniform (in time) upper bounds on the total variation distance between this field and the one of a sequence of $n$ i.i.d. random variables with law given by the invariant measure of that Markov chain. The proof of this result uses a refinement of the soft local time method of [11]., 42 pages, 2 figures, coupling construction of Section 4 corrected, results unchanged

In this paper we prove results on logarithmic convexity of fixed points of stochastic kernel operators. These results are expected to play a key role in the economic application to strategic market games.

The positive stochastic stabilization issue is investigated for Markov jump nonlinear systems. The nonlinear systems in the absence and presence of uncertainties are considered and modeled into Markov jump fuzzy systems with known and partially known membership functions, respectively. For the former, the parallel distributed compensator (PDC) controllers are designed to realize the positive stochastic stabilization. Besides, an additional criterion is derived to reduce the number of linear inequalities. For the latter, namely, some elements of membership functions are unavailable, and the PDC strategy is infeasible. Then, a switching controller, whose structure is parallel to that of fuzzy model, is constructed to achieve the corresponding positivity and stability. All the conditions are transformed to standard form of linear programming problem, which is solved with the help of linprog toolbox. Finally, the feasibility of the obtained results is illustrated by a pest's structured population model.

We study two relevant characterizations of a commutative positive operator valued measure (POVM) F. The first one is a Choquet type of an integral representation. It introduces a measure ν on the space of the projection valued measures (PVMs) and describes F as an integral over this space. The second one represents a commutative POVM F as the randomization of a single PVM E by means of a Markov kernel μ. We show that one can be derived from the other. We also elaborate upon some previous results on Choquet’s representation of Markov kernels and find a functional relationship between ν and μ. Finally, we analyze some relevant particular cases and provide some physically relevant examples which include the unsharp position observables.

Certain aspects of convergence rate in a non-homogeneous Markov system (NHMS) using fuzzy set theory and fuzzy reasoning are presented in this paper. More specifically, a fuzzy non-homogeneous Markov system is considered where fuzzy inference systems are used to estimate the transition probabilities, the input and loss probabilities and the population structure of the system. It is proved that under some conditions easily met in practice, the rate of convergence of the sequence of the relative population structures in such a system is geometric, i.e. it converges to its limit geometrically fast.

Virtually every biological model utilising a random number generator is a Markov stochastic process. Numerical simulations of such processes are performed using stochastic or intensity matrices or kernels. Biologists, however, define stochastic processes in a slightly different way to how mathematicians typically do. A discrete-time discrete-value stochastic process may be defined by a function p : X0 × X → {f : Υ → [0, 1]}, where X is a set of states, X0 is a bounded subset of X, Υ is a subset of integers (here associated with discrete time), where the function p satisfies 0 < p(x, y)(t) < 1 and EY p(x, y)(t) = 1. This definition generalizes a stochastic matrix. Although X0 is bounded, X may include every possible state and is often infinite. By interrupting the process whenever the state transitions into the X −X0 set, Markov stochastic processes defined this way may have non-quadratic stochastic matrices. Similar principle applies to intensity matrices, stochastic and intensity kernels resulting from considering many biological models as Markov stochastic processes. Class of such processes has important properties when considered from a point of view of theoretical mathematics. In particular, every process from this class may be simulated (hence they all exist in a physical sense) and has a well-defined probabilistic space associated with it.

Purpose The main purpose of this paper is to explore the transitional dynamics of housing affordability indicators of major cities in three developed countries: the USA, Canada and Australia, in the period after the global financial crisis. As the global housing markets are more interconnected today, it is essential to investigate the demographic movement pattern and their impacts on housing market dynamics. Design/methodology/approach Based on the Markov transition matrix approach and the stochastic kernel technique, a newly established framework named the mobility probability plot (MPP) is adopted to investigate the city-level trends of housing affordability in the three countries during the period 2008-2015. Findings The results suggest that the transitional dynamics of the USA’s housing affordability trend saliently differs from those of Canada and Australia: in the USA, MPP results reveal that when the price-to-income (P/I) ratio is higher than 3.5 times, it has a high tendency of moving downward in the next period. In Australia, housing affordability tends to continue deteriorating when the P/I ratios are in the range from 8.0 to 8.6. In Canada, the MPP analysis indicates that the P/I ratios tend to increase further when the ratios are between 5.7 and 7.0, and within the range of 8.3-9.5. Originality/value This paper adopts an innovative approach to explore the city-level trends of housing affordability in the three developed countries during the period 2008-2015. The distribution dynamics approach has several virtues: first, this approach does not merely focus on the issue of housing affordability but also includes an analysis of the underlying housing affordability distribution. Second, it can clearly show the mobility of the city-level units in terms of the P/I change. Third, it can predict the proportion of the entities in different P/I ratio bands in a number of years ahead and even in the long run.

In this paper, we present examples of a class of Markov chains that occur frequently, but whose associated matrices are a challenge to construct efficiently. These are Markov chains that arise as a result of several identical Markov chains running in parallel. Specifically for the cases considered, both the infinitesimal generator matrix for the continuous case, and more so the transition probability matrix for the discrete equivalent, are complex to construct effectively and efficiently. We summarize the algorithms for constructing the associated matrices and present examples of applications, ranging from special queueing problems to reliability issues and order statistics. MATLAB subroutines are provided in an online supplement for the implementation of the algorithms. The online supplement is available at https://doi.org/10.1287/ijoc.2017.0759 .

We consider a method of obtaining non-closed solutions of the first and second Kolmogorov equations for the exponential (double) generating function of transition probabilities for quadratic death-processes of one, two and three dimensions. We obtain a representation for the generating function of transition probabilities in the form of a Fourier series, using generalized hypergeometric functions and Jacobi polynomials.

We derive a sufficient condition for a $k$-th order homogeneous Markov chain $\mathbf{Z}$ with finite alphabet $\mathcal{Z}$ to have a unique invariant distribution on $\mathcal{Z}^k$. Specifically, let $\mathbf{X}$ be a first-order, stationary Markov chain with finite alphabet $\mathcal{X}$ and a single recurrent class, let $g{:}\ \mathcal{X}\to\mathcal{Z}$ be non-injective, and define the (possibly non-Markovian) process $\mathbf{Y}:=g(\mathbf{X})$ (where $g$ is applied coordinate-wise). If $\mathbf{Z}$ is the $k$-th order Markov approximation of $\mathbf{Y}$, its invariant distribution is unique. We generalize this to non-Markovian processes $\mathbf{X}$., Comment: 11 pages, 1 figure

This paper proposes a class of models that jointly model returns and ex-post variance measures under a Markov switching framework. Both univariate and multivariate return versions of the model are introduced. Bayesian estimation can be conducted under a fixed dimension state space or an infinite one. The proposed models can be seen as nonlinear common factor models subject to Markov switching and are able to exploit the information content in both returns and ex-post volatility measures. Applications to U.S. equity returns and foreign exchange rates compare the proposed models to existing alternatives. The empirical results show that the joint models improve density forecasts for returns and point predictions of return variance. The joint Markov switching models can increase the precision of parameter estimates and sharpen the inference of the latent state variable.

The continuous time Markov process is commonly used to model state transitions in a multi-state system. However, it is hard to verify the Markov assumption by traditional goodness-of-fit techniques due to the multiple states. In this article, a goodness-of-fit test conditioning on a sufficient statistic is developed to test the Markov assumption. We propose a test statistic whose conditional distribution given the sufficient statistic is free of any unknown parameters. Therefore, the conditional distribution of the test statistic together with the corresponding percentiles could be obtained by Monte Carlo simulation. This conditional test outperforms two conventional goodness-of-fit techniques for the Markov assumption. Further, we consider a multi-state degradation system in which the operating conditions are available. An acceleration relation is often used to model the effects of the operating conditions on the degradation system. In this scenario, a goodness-of-fit test based on the Greenwood statistic is developed to test both the Markov assumption and the acceleration relation. The performance of the proposed goodness-of-fit tests in multi-state systems is demonstrated by extensive simulations and a numerical example.

We study a class of Markov-modulated compound Poisson processes whose arrival rates and the compound random variables are both modulated by a stationary finite-state Markov process. The compound random variables are i.i.d. in each state of the Markov process, while having a distribution depending on the state of the Markov process. We prove a functional central limit theorem and local limit theorems under appropriate scalings of the arrival process, compound random variables and underlying Markov process.

This paper develops vector autoregressive models with infinite hidden Markov structures, motivated by the recent empirical success of hierarchical Dirichlet process mixture models in financial and macroeconomic applications. We begin by developing a new Markov chain Monte Carlo (MCMC) method that is built upon precision-based algorithms, in order to improve the computational efficiency. We then investigate the forecast performances of these infinite hidden Markov switching models. Our forecasting results suggest that (1) models with separate infinite hidden Markov processes for the VAR coefficients and the volatilities generally forecast better than other specifications of infinite hidden Markov switching models; (2) using a single infinite hidden Markov process to govern all model parameters tends to result in poor forecasts; (3) most of the gains obtained when forecasting the inflation rate and GDP growth seem to come from allowing for time-variation in the volatilities rather than in the conditional mean coefficients. In contrast, when forecasting the short-term interest rate it is important to allow time-variation in all model parameters.

In this paper, we prove that the Foster–Lyapunov drift condition is necessary and sufficient for recurrence of a Markov chain on a general state space. And then an answer in the affirmative for the question presented in the monograph of Meyn and Tweedie (Markov chains and stochastic stability. Springer, Berlin, 1993, P175) is obtained.

Motivated by various applications in queueing theory, this article is devoted to the stochastic monotonicity and comparability of Markov chains with block-monotone transition matrices. First, we introduce the notion of block-increasing convex order for probability vectors, and characterize the block-monotone matrices in the sense of the block-increasing order and block-increasing convex order. Second, we characterize the Markov chain with general transition matrix by martingale and provide a stochastic comparison of two block-monotone Markov chains under the two block-monotone orders. Third, the stochastic comparison results for the Markov chains corresponding to the discrete-time GI/G/1 queue with different service distributions under the two block-monotone orders are given, and the lower bound and upper bound of the Markov chain corresponding to the discrete-time GI/G/1 queue in the sense of the block-increasing convex order are found.

In this article, we study the optimal control of a partially observed Markov chain for which a mean squared cost functional is minimised. Both the terminal cost and the running cost are considered. Minimum principles are established. In both cases, if the optimal control is Markov feedback, more explicit forms for the stochastic integrands and adjoint processes are obtained. Refereed/Peer-reviewed

An investigation has been made into the sliding mode control (SMC) problem for singular stochastic Markov systems (SSMSs) in this paper. By using replacement of matrix variables, a new mean square admissibility criterion of SSMSs is proposed first. Based on this admissibility criterion, the desired state-feedback controller is designed to guarantee the closed-loop system to be mean-square admissible. Then, the obtained results given in the form of strict linear matrix inequalities are applied to solve SMC problem of SSMSs. In order to illustrate the workability and applicability of the theoretic results developed, numerical examples are provided.

This paper considers a two-person zero-sum continuous-time Markov pure jump game in Borel state and action spaces over a fixed finite horizon. The main assumption on the model is the existence of a drift function, which bounds the reward rate. Under some regularity conditions, we show that the game has a value, and both of the players have their optimal policies.

We propose some backward–forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional Central limit theorem for reversible Markov chains with asymptotically linear variance of partial sums. We also provide a proof of the equivalence between asymptotic linearity of the variance and convergence of the integral of 1 / ( 1 − t ) with respect to the associated spectral measure ρ . We show a result on uniform integrability of the supremum of the average sum of squares of martingale differences. We also study the asymptotic behavior of linear processes having as innovations mean zero square integrable functions of stationary reversible Markov chains. We include in our study the long-range dependence case. We apply this study to several cases of reversible stationary Markov chains that arise in regression estimation.

We give a method to associate a right Markov process to a resistance form, that is different from the one based on the regularity assumption for the associated Dirichlet form. We use tools from potential theory associated to resolvents of kernels and cones of potentials (H-cones). The induced capacity and processes obtained by time change with the inverses of continuous additive functionals also occur.

This paper deals with the H2 optimal control problem of discrete-time Markov jump linear systems (MJLS) considering the case in which the Markov chain takes values in a general Borel space S. It is assumed that the controller has access only to an output variable and to the jump parameter. The goal, in this case, is to design a dynamic Markov jump controller such that the H2-norm of the closed-loop system is minimised. It is shown that the H2-norm can be written as the sum of two H2-norms, such that one of them does not depend on the control, and the other one is obtained from the optimal filter for an infinite-horizon filtering problem. This result can be seen as a separation principle for MJLS with Markov chain in a Borel space S considering the infinite time horizon case.

The paper deals with parsimonious models of integer valued time series. Such models are special cases of high-order Markov chain with a small number of parameters. Two new parsimonious models are presented. The first is Markov chain of order s with r partial connections, and the second model is called Markov chain of conditional order. Theoretical results on probabilistic properties and statistical inferences for these models are given.

The matrices of non-homogeneous Markov processes consist of time-dependent functions whose values at time form typical intensity matrices. For solvingsome problems they must be changed into stochastic matrices. A stochas-tic matrix for non-homogeneous Markov process consists of time-dependent functions, whose values are probabilities and it depend on assumed time pe- riod. In this paper formulas for these functions are derived. Although the formula is not simple, it allows proving some theorems for Markov stochastic processes, well known for homogeneous processes, but for non-homogeneous ones the proofs of them turned out shorter.

A Markov chain with memory is no different from the conventional Markov chain on the product state space. Such a Markovianization, however, increases the dimensionality exponentially. Instead, Markov chain with memory can naturally be represented as a tensor, whence the transitions of the state distribution and the memory distribution can be characterized by specially defined tensor products. In this context, the progression of a Markov chain can be interpreted as variants of power-like iterations moving toward the limiting probability distributions. What is not clear is the makeup of the second dominant eigenvalue that affects the convergence rate of the iteration, if the method converges at all. Casting the power method as a fixed-point iteration, this paper examines the local behavior of the nonlinear map and identifies the cause of convergence or divergence. As an application, it is found that there exists an open set of irreducible and aperiodic transition probability tensors where the Z-eigenvector type power iteration fails to converge.

In this paper, we are going to study the strong laws of large numbers for countable nonhomogeneous hidden Markov models. Firstly, we introduce the nation of countable nonhomogeneous hidden Markov models. Then, we obtain some properties for those Markov models. Finally, we establish two strong laws of large numbers for countable nonhomogeneous hidden Markov models. As corollaries, we obtain some known results of strong laws of large numbers for finite nonhomogeneous Markov chians.

This paper is concerned with Schur-constant survival models for discrete random variables. Our main purpose is to prove that the associated partial sum process is a non-homogeneous Markov chain. This is shown in three different situations where the random variables considered take values in the sets 0, {0,1} or {0,…,m}, m ≥ 2. The property of Schur-constancy is also compared for these three cases.