Search

Deriving the time-dependent expected reward function associated with a continuous-time Markov chain involves the computation of its transient deviation matrix. In this paper we focus on the special case of a finite quasi-birth-and-death (QBD) process, motivated by the desire to compute the expected revenue lost in a MAP/PH/1/C queue. We use two different approaches in this context. The first is based on the solution of a finite system of matrix difference equations; it provides an expression for the blocks of the expected reward vector, the deviation matrix, and the mean first passage time matrix. The second approach, based on some results in the perturbation theory of Markov chains, leads to a recursive method to compute the full deviation matrix of a finite QBD process. We compare the two approaches using some numerical examples.

We consider perturbations of positive recurrent Markov modulated fluid models. In addition to the infinitesimal generator of the phases, we also perturb the rate matrix, and analyze the effect of those perturbations on the matrix of first return probabilities to the initial level. Our main contribution is the construction of a substitute for the matrix of first return probabilities, which enables us to analyze the effect of the perturbation under consideration.

In this paper we study the distribution of the level at time $\theta$ of Markovian fluid queues and Markovian continuous time random walks, the maximum (and minimum) level over $[0,\theta]$, and their joint distributions. We approximate $\theta$ by a random variable $T$ with Erlang distribution and we use an alternative way, with respect to the usual Laplace transform approach, to compute the distributions. We present probabilistic interpretation of the equations and provide a numerical illustration.

We consider Poisson's equation for quasi-birth-and-death processes (QBDs) and we exploit the special transition structure of QBDs to obtain its solutions in two different forms. One is based on a decomposition through first passage times to lower levels, the other is based on a recursive expression for the deviation matrix. We revisit the link between a solution of Poisson's equation and perturbation analysis and we show that it applies to QBDs. We conclude with the PH/M/1 queue as an illustrative example, and we measure the sensitivity of the expected queue size to the initial value.