This article was downloaded by: [UQ Library]

On: 24 April 2015, At: 10:20

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer
House, 37-41 Mortimer Street, London W1T 3JH, UK

Communications in Statistics. Stochastic Models

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/lstm19

Calculating the equilibrium distribution in level

dependent quasi-birth-and-death processes
a

L. Bright & P.G. Taylor

Department of Applied Mathematics , The University of Adelaide , South Australia,

Australia, 5005 E-mail:
b

Department of Applied Mathematics , The University of Adelaide , South Australia,

Australia, 5005 E-mail:
Published online: 21 Mar 2007.

To cite this article: L. Bright & P.G. Taylor (1995) Calculating the equilibrium distribution in level dependent quasi-birthand-death processes, Communications in Statistics. Stochastic Models, 11:3, 497-525, DOI: 10.1080/15326349508807357
To link to this article: http://dx.doi.org/10.1080/15326349508807357

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of
the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied
upon and should be independently verified with primary sources of information. Taylor and Francis shall
not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other
liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions

COMMUN. STATIST.-STOCHASTIC MODELS, 11(3), 497-525 (1995)

Calculating the Equilibrium Distribution in Level

Dependent Quasi-Birt h-and-Deat h Processes
L. BRIGHT and P.G. TAYLOR
Department of Applied h/Iathematics
The University of Adelaide, South Austra!ia 5005, Australia

Downloaded by [UQ Library] at 10:20 24 April 2015

email: lbright@maths.adelaide.edu.au,ptaylor@maths.adelaide.edu.au

Abstract Quasi-Birth-and-Death (QBD) processes have been analysed in

detail by many authors. This paper considers the calculation of equilibrium
distributions in level dependent QBD processes which are an extension of
the classical QBD process. In addition to the general case, we consider
a number of special cases of level dependent QBD processes, presenting
algorithms for computing the equilibrium distribution.

Keywords: level dependent Quasi-Birth-and-Death process, dominating

process, level, phase.

1. INTRODUCTION
Consider a regular continuous time two-dimensional Markov chain X ( t )
on the state space {(k, j) ; k 2 0 , l 5 j 5 M k ) . In the state (k, j ) , k is
referred to as the level of the state and j is referred to as the phase of the
state. Suppose the generator matrix for this Marltov chain is of the block
partitioned form

Copyright O 1995 by Marcel Dekker, Inc

BRIGHT AND TAYLOR

>

where Qik),k 1, Q?), k 2 0, Q r ) , k 2 0 are matrices of order Mk x Mk-1,

Mk x Mk, and Mk x Mk+1respectively. A Markov chain with this structure
is referred to as a level dependent quasi-birth-and-death process (hereby
abbreviated to LDQBD). We will assume that the process is irreducible.
I f w e p u t M k = M ~ k Z ~ a n d Q f ) = Qv o~ > o , Q ~ ) =VQk L~1 ,

Downloaded by [UQ Library] at 10:20 24 April 2015

QP) = Q1 and Q f ) = Qz Vk 2 1 then X ( t ) becomes a level independent

QBD process. Level independent QBD's are a powerful modelling tool.

They can be used to model, for example, queues in a random environment,
high speed communication systems, database systems and multiprogramming systems (see [6], [lo], [ll],[19], [20] and [9]). Since LDQBD's are a
generalisation of level independent processes, they enable us to model a
wider class of problems. For example, the P H I M I 1 queue (i.e the single
server queue with a phase-type arrival process) can be analysed using a
level independent QBD whereas the PH/M/oo can be analysed using a
LDQBD. Similarly, LDQBD's enable us to model infinite server queues in
a random environment rather than just single server queues. LDQBD's
also enable us to model more complex communication systems and multiprogramming systems.
Level independent QBD's have been dealt with widely in the literature.
Neuts [12] has developed methods for determining positive recurrence of
X ( t ) and for calculating the equilibrium distribution. Neuts also considered level independent QBD7s where there is some non-homogeneous
boundary behaviour but the process is eventually level independent. Recently Latouche and Ramaswami [7] have developed a logarithmic reduction algorithm that can be used to calculate the equilibrium distribution
in the level independent case. The computational complexity of this algorithm is significantly less than that for previously existing algorithms. The
methods we present for LDQBD's involve an extension of this logarithmic
reduction algorithm to the level dependent case.
The special case where X ( t ) has a finite state space has also received significant attention in the literature. This occurs when for some I?, QiK) = 0
and Q f ) = Q!') = Q r ) = 0 V k 2 l?
1. Hajeli [4] initially considered
level independent processes on a finite state space and Gaver, Jacobs and
Latouche [3] developed a general method for the level dependent case on

LEVEL DEPENDENT QBD PROCESSES

499

a finite state space. Ye and Li [18] have developed a computationally efficient algorithm for the case where the generator is piecewise homogeneous.
In the level dependent case on an infinite state space, as far as the authors
know, there is no general method for determining positive recurrence and
the problem of calculating the equilibrium distribution has not been addressed.
In order to calculate the ecpilibrium distribution for a LDQBD, it is
necessary to evaluate the family of matrices { R k k, 2 0) which are the
minimal non-negative solutions to the system of equations

The matrices { R k k, 2 0 ) have the following physical interpretation. (Rk)i,j

is the expected sojourn time in the state (k + 1 , j ) per unit sojourn in the
state ( k , i) before returning to level k , given the process started in state
( k , i).

Downloaded by [UQ Library] at 10:20 24 April 2015

(1)
Define mo to be a positive left eigenvector of the matrix Q(P) RoQz
and define m k by m k = mo l$;j Re. Then it is easily seen that m =
(mo,
m l , . . .) satisfies mQ = 0. From Theorem 4.5 in Anderson [ I ] , X ( t )
is positive recurrent if and only if m e < oo where e is a column vector of
ones. In this case, the equilibrium distribution x = ( x o X, I , . . .) of X ( t ) is
given by xk = m k / m e .

Thus to show that a particular process is positive recurrent we must

show that the equation

[QjO)

+ R ~ Q ~= )o ]

has a positive solution xo such that

and then x is given by

where xo satisfies (1.3) and

We can only calculate the infinite sum in ( l . G ) if we know { R k k, 2 0).

This is not often the case since generally only numerical solutions can be
found for the Rk matrices. Hence we must calculate a truncated sum. If
we define ( x ~ ( I < * 0) )5~ ,k 5 Ii* to be the stationary probability that

BRIGHT AND TAYLOR

500

<

X(4) is in the state ( k ,j ) cgnditional on X ( t ) being in the set {(i,j ) l O

i 5 k*,1 5 j 5 M i ) then it i s clear that x k ( I < * ) 0, 5 k 5 I(* is given by

where xo(I<*)satisfies (1.3) and

Note that we are not truncating X ( t ) to a finite state space consisting

of all states in and below level I<*, rather we are calculating an invariant
measure for the equilibrium distribution for all states in and below level
Ii* and then we are normalising this invariant measure over these states.
It is obvious that V I<* 2 0 , x k ( I < * )is an upper bound for xk. So we
have
and

xk = lim x k ( I < * ) V k 2 0
Downloaded by [UQ Library] at 10:20 24 April 2015

IC'iec

where the inequality and the limit hold elementwise. Provided we take I<*
large enough, we hope to have xk x x k ( K * ) . The question of finding a
sufficiently large value of Ii*will be considered in a later section.
The rest of the paper is as follows. In Section 2 we present a n algorithm
for evaluating x k ( I < * )given a value of I<*. In Section 3 we discuss how to
efficiently compute the matrices Rk and in Section 4 we give methods for
choosing a value for K * . Section 5 presents some numerical examples and
we summarise our results in Section 6.
2. CALCULATING xlc(I<*)FOR A GIVEN VALUE OF K*

From equation (1.7), in order to calculate x k ( I < * ) ,0 5 k 5 I(*, we need

to know R k , 0 5 k 5 IiV*- 1. It is possible to obtain a closed from
expression for R k by developing a continuous time analogue of Theorem
2.2 in Ramaswami and Taylor [14]. To do this it is necessary to use the
jump chain of X ( t ) . The jump chain of X ( t ) is a discrete time LDQBD
obtained by observing X ( t ) at time points immediately after a transition
has occurred.

Lemma 1
If X ( t ) is positive recurrent then Rk is given by

LEVEL DEPENDENT QBD PROCESSES

501

where Uf and D! are Adk-l x Mk-1+2tand Mk-1 x Mk-l-2c matrices respectively and are defined recursively by

Proof

Downloaded by [UQ Library] at 10:20 24 April 2015

In Theorem 2.2 of [14], Ramaswami and Taylor considered discrete time

LDQBD's. In discrete time LDQBD's the generator matrix Q in (1.1) is
replaced'by a stochastic matrix P which has the same block-partitioned
form as Q. Suppose for the jump chain of X ( t ) this matrix P consists
of the sub matrices ALk), AIk) and AF). In discrete time LDQBD7s it is
necessary to calculate the family of matrices {Rk, 0 3 k) which are the
minimal non-negative solutions of the equations

By Theorem 2.2 of [14] an explicit expression for

jump chain has an invariant measure is

for the case where the

uf

where
and D: are defined recursively by a set of equations similar to
equations (2.2) to (2.5). Since S ( t ) is positive recurrent, from results
in [I] we know that the jump chain must be recurrent and so have an
invariant measure. Ramaswami and Taylor showed that Rk can be written
as Rk = Qk~kJQ;l,where RkJ is the matrix Rk for the jump chain and Qk is
defined by Qk = -diag[(Q(,k))ii]. Using this expression together with (2.7j,
it is a simple matter to derive equation (2.1).
In order to use Lemma 1 to calculate Rk, we need to truncate the infinite
sum in (2.1). A simple way to do this is to define Rk(n) as the sum of
the first n 1 terms in (2.1) and then to take Rk = Rk(L) such that
(Rk(L) - R k ( L - I)),,,, < 6 for some tolerance E where (M),,, is the
maximum entry of the matrix Ad.

To calculate {Rk,0 5 k 5 K * ) ,it is not necessary to use equation (2.1)

repeatedly. By re-arranging equation (1.2), we have

<

provided the inverse exists. So a possible method for calculating {Rk,0

k 5 I<* - 1) is to use equation (2.1) to calculate Rrc*-l and then use

BRIGHT AND TAYLOR

502

equation (2.8) to calculate RIp-2,RIi--3,..., RO.In order for the recursion

to be stable we require that the inverse contain non-negative elements. We
prove this is the case and that the inverse does exist in the appendix. Note
that the truncation rule given above does not guarantee that ( R k ( n )Rk(n- I)),,,
< E holds V n > L which is what we want. An algorithm for
calculating {xn.(I<*),
0 5 k 5 I<*}for a given value of I(* is as follows.

Algorithm 1 : Calculating {xk(Ir'*),

05k

< I{*) given I<*

Calculate R I ~ - 1
Recursively calculate Rip-2, RI(*-3,
..., RO

Solve x o ( ~ < * ) ( ~R$~) Q ! )=

) o s.t. x o ( K * ) e= I

Downloaded by [UQ Library] at 10:20 24 April 2015

for k = 1 to I<* do

xk(I<*)
=x~-~(I<*)R~-~
Normalise xo(I<*),
xl(I<*),
...,xk(I<*)s.t.

c&=
x,(IC*)e
~

=1

3. EFFICIENTLY COMPUTING Rk

In Section 2 we defined Rk(n)to be the sum of the first n 1 terms in

the sum (2.1) and we tool; Rk = R k ( L )where L was chosen such that
( R k ( L) Rk(L- I)),,,
< E . An efficient method for determining the value
of L is expressed in the following algorithm.

Algorithm 2 : Computing Rk

Downloaded by [UQ Library] at 10:20 24 April 2015

LEVEL DEPENDENT QBD PROCESSES

In Algorithm 2 we calculate U: and D:+zol at the same step. This is

because both quantities involve the same inverse and so it is sensible to
compute them together to avoid calculating the same inverse twice. To see
this we restate equations (2.2) to (2.5) for U: and D:+~(+~.

Note that we calculate

in Algorithm 2 but do not use it. However,
it is no extra effort to compute this quantity because we already have the
necessary inverse from calculating U:
We refer to U i and D:+2t+l as the UD-pair UD(C, k). At the end of Algorithm 2 we have calculated the UD-pairs UD(e, k) for for C = 0, l , . . . ,L
and fixed k. To do this we need to calculate other UD-pairs UD(n, m) for
n < e. We consider a way to compute these quantities so that UD-pairs are
not computed more than once. By storing various UD-pairs it is possible
to develop an efficient method for calculating UD(e, k), l = 0 , . . . , L. To
determine which UD-pairs to calculate and which to store, it is useful to
represent the recursive relationships expressed by equations (3.3) and (3.4)
by a diagram. Figure 1 shows that to calculate UD(e, k), we need to know
the UD-pairs UD(C- 1, k), UD(e- 1, k+2e-1), and UD(C- 1, k +2.2e-1).

BRIGHT AND TAYLOR

1-1.
k

k+2e-1

I~+2.2~-l

Downloaded by [UQ Library] at 10:20 24 April 2015

Figure 1: Recursive relationship between UD-pairs.

By using Figure 1 repeatedly, we see that to calculate UD(2, k) we

need first to calculate UD(1, k + m ) for in = 0 , 2 , 4 and UD(0, k + m) for
m = 0 , 1 , . . . , 6 . Similarly, to calculate UD(3, k) we need first to calculate
UD(2, k + m) for m = 0,4,8, UD(1, k + rn) for m = 0 , 2 , 4 , . . . ,12 and
UD(0, k + m) for in = 0 , 1 , 2 , . . . ,14. Figure 2 represents the last two
statements. To calculate UD(1, k) we need to calculate all the UD-pairs
in the second smallest triangle in Figure 2. To calculate UD(2, k) we need
to calculate all the UD-pairs in the third smallest triangle and to calculate
UD(3, k) we need to calculate all the UD-pairs in the largest triangle.
Figure 2 can be extended to any arbitrary value of !. From Figure 2 we
see that if we calculate UD(l, k ) for e = 0 , 1 , 2 , 3 by directly implementing
equations (3.3) and (3.4) then we will be computing some UD-pairs over
and over again. However, it is possible to calculate UD-pairs in an order
that will result in not computing any UD-pair more than once. The UDpairs needed for Algorithm 2 can be calculated (and used) in the following
order.
A l g o r i t h m 3 : Evaluating UD-pairs for A l g o r i t h m 2

compute UD(0, k), use it in Algorithm 2 and store it

do
for i = 0 to !
for j = k
j =k

+ (2e-i+1- 1 ) 2 ~to

+ 2(2e-i+1- 1)2i in steps of size 2i

compute UD(i, j) and store it

LEVEL DEPENDENT QBD PROCESSES

Downloaded by [UQ Library] at 10:20 24 April 2015

Figure 2: Dependencies between UD-pairs

compute U D ( t , k), use it in Algorithm 2 and store it

remove all UD-pairs from storage except the pairs
UD(j, k

+ (2e-j - 1)2j+l),j

= 0,1,. . .,

until (Algorithm 2 stops)

When Algorithm 2 stops, we have calculated the UD-pairs UD(-t, k)
for -t = 0 , 1 , . . . , L and fised I;. If these UD-pairs are computed according
to Algorithm 3 then a total of 4.2L - L - 3 inverses must be calculated.
If the UD-pairs were calculated by directly implementing the recursive
equations (3.3) and (3.4) then a total of approximately i.5L inverses must
be calculated. Note that with Algorithm 3 the number of inverses still
increases exponentially with L. However, for many practical examples
only a small value of L (say four or five) is needed. To see this, we give a
physical interpretation of equation (2.1).
Recall that (Rk)i,jis interpreted as the expected sojourn time in the
state (k + 1,j) per unit sojourn in the state (k, i) before returning to level
k, given the process started in state (k, i). In order to calculate Rk,we
need to consider all sample paths that start and finish a t level k and don't
visit any levels below level k 1 in between. For each of these sample
paths we have to calculate the expected sojourn times in each of the states

BRIGHT AND TAYLOR

506

in level k 1. Adding together the sojourn times for all the sample paths
will give us Rk. Ramaswami and Taylor [14] observed that the 1th term
in (2.1) calculates the expected sojourn times in level k 1 for sample
paths that go at least as high as level k - 1 2e and no higher than level
k - 1 2'+'. So when we truncate the sum in (2.1) to L 1 terms, we only
consider the sample paths that remain below level k - 1 2L+2. Thus even
for relatively small L (say four or five) we consider paths which move up 64
or 128 levels before returning. For most physical processes this accounts
for most of the probability mass.

+
+

Storage is an issue in Algorithm 3. In its present form the Algorithm

requires the storage of many UD-pairs. However, it is not difficult to alter
the algorithm to decrease the amount of storage space required. In fact it
is possible to alter Algorithm 3 so that at any time only 1 1 UD-pairs
need to be in storage.

Downloaded by [UQ Library] at 10:20 24 April 2015

4. CHOOSING A VALUE FOR I<*

When calculating {xk(I<*)),0 5 k 5 I<*)

we want to find a level I(* such
that the equilibrium probability of being in a state in or above level K* is
approximately zero. In some special cases it is possible to calculate K* a
priori. In this section we firstly discuss a general method for calculating
I<* and then we look at some special cases.

4.1 A General Method

For a LDQBD we must study the particular process and devise some
way of showing that the process is or is not positive recurrent. If X ( t ) is
positive recurrent then xt(I<*), k 2 0 is an upper bound for the equilibrium distribution and from equation (1.8) it can be seen that by choosing
I<* very large, xk(I<*)will give a close approximation to the equilibrium
distribution. However, as I<*increases, so does the computational effort
required to calculate xk(K1). To see this consider Algorithm 1. The main
computational overhead is in computing the inverses in the first two steps.
To calculate RIc.-l with equation (2.1) requires the calculation of approximately 4.2L inverses and to calculate each of Rk for I<*- 1 > k 2 0 requires
one inverse.
In some applications L is large for small values of k and smaller for
large values of k. Suppose that for some value of k we need a very large
value of L. In this case, the more efficient method would be to calculate
Rk for a larger value of k (one which requires only L = 4 or 5 say) and then
calculate the Rkmatrices for lower values of k using the recursion in (2.8).
For example, suppose we required L = 7 to calculate Rk,, for some value

507

LEVEL DEPENDENT QBD PROCESSES

kl. This requires approximately 4.27 = 512 inverses. Now suppose in order
to calculate
we required only L = 4. This requires approximately
4.24 = 64 inverses. If we use the recursion in (2.8) to calculate Rk,,
we
need to calculate a further 40 inverses giving a total of 104 inverses. The
second method is obviously the more efficient.

A simple approach to finding a value of I<* consists of trying various

values of I<* until we obtain one that is sufficient. This method is not
guaranteed to work. It is certainly possible to do better for some LDQBD's
(discussed in Sections 4.2 and 4.3 below). However, for most processes of
physical interest, the following algorithm will halt at a suitable value of
I<*.
Algorithm 4 : A general method for calculating

{xk(I< *),O2 k 5 I<*}

Downloaded by [UQ Library] at 10:20 24 April 2015

Set I<& = 0

do
Choose I<;,, such that I<:,,

> I&

Calculate 5 terms in the expression (2.1) for

RIpw-l

if ( the 5 terms are sufficient ) then continue

else repeat above two steps
Recursively calculate

RIp,-2,

RI<;,,-3, . . . ,R I ~ , ~

Solve X ~ ( I L ~ , , , , ) ( Q \ ~R) ~ Q ~
= )0 )subject to xo(K~,,)e= 1.

for k = I<:/,

+ 1 t o I<;,,

do

x k ( I i i e w )= Z ~ - ~ ( I < ~ , , ) R ~ - ~
Normalise xo(I<iew),x l ( K i e W )., . . , x k ( I < i e w )
s.t.

~ k xm(I<iew)e
= ~ =1

BRIGHT AND TAYLOR

until ( x ~ ; , , ( I ( ~ ~<~ E) )e
Set I<* =
We hope that by choosing I<*
in this fashion that we will find a value
of I<*such that xk x o V k > K*. It is certainly possible to construct
LDQBD's for which this is not the case. However, for most LDQBD's
that arise in practice, the matrices Q f ) ,
and Q r ) can be expressed
in some parametric form. Indeed this is necessary if a process is to be
analysed using a computer. It would be very difficult, if not impossible,
even to input into a com uter the parameters of a process involving a large
number of unrelated Qlif matrices. By considering the parametric form of
found by
the matrices QIk),it should be possible to tell if the value of I<*
the above method is accurate. For the above reasons, careful consideration
must be given to the structure of the LDQBD when using this method to
find I(*.

~(1")

Downloaded by [UQ Library] at 10:20 24 April 2015

4.2 L D Q B D ' s with a L a r g e Number of B o u n d a r y S t a t e s

If Q r ) = Q2, Q ( ~=~ Q1
) and Q f ) = Qo, V k 2 K, then the process is
essentially a level independent QBD where there is some level dependent
behaviour in the levels 0 5 k. .<? I We refer to the states in these levels
as the boundary states.
Such processes have also been considered in Neuts 1121. The approach
there also uses xk = X ~ ( R ~ ) ' " -to" find xk, k > I? which is what we
obtain from equation (1.5). To find xk, 0 5 k 5 I?, Neuts solved

This does not highlight the matris analytic structure of xk for k 5 I?. A
nice property of our method is that it shows that xa has a matrix analytic
form for all 0 5 k 5 I?.
In this particular case, it is possible to determine if X ( t ) is positive
recurrent and to determine the equilibrium distribution exactly. Neuts
[12] has shown that X ( t ) is positive recurrent if and only if x Q o e < x Q z e
where Q = Q1+ Q o + Q 2 and rr is the solution to rrQ = o such that x e = 1.
Because of the physical interpretation of R k , it is clear that Rk = RI;.for
k E . If X ( t ) is positive recurrent then

>

LEVEL DEPENDENT QBD PROCESSES

and the equilibrium distribution can be calculated exactly using equation

(1.5). This is one of the few cases where this is possible. Since X ( t ) is
a level independent QBD for levels I? and above, when we calculate R K ,
we are in fact dealing with a level independent QBD. In this case, the
expression in (2.1) simplifies to the case in Latouche and Ramaswami [7].
An algorithm for this case is given below.
Algorithm 5 : Calculating the equilibrium distribution
for a large number of boundary states

Downloaded by [UQ Library] at 10:20 24 April 2015

Calculate R R
Recursively calculate RRWl,RRv2,...,RO

Solve X ~ ( Q ( ~R~ ~) Q ~=) o) s.t.

xo [E!=~

n s R,, + ni=oR m ( I- RE)-']

Calculate xl: k

e=1

1 0 according to equat,ion (1.5)

4.3 A Dominating Process Exists

Suppose X ( t ) is such that the following condition holds.

Condition 1 : V k, i 3 j s.t (

>

~ f ) ) i , ~
0.

If condition 1 holcls, then at any phase at any level, X ( t ) moves down to

a state in the level below according to some positive rate. Note that this
class of processes is a non-trivial class. The P H / M / c o queue modelled
as a LDQBD has this property. When condition 1 holds, it is possible to
construct a second LDQBD that stochastically dominates the first process.
This second process also has the property that its marginal distribution
over the levels is very easy to calculate. It is then possible to calculate a
value of I{*using this marginal distribution. Before we present our results,
we define what it means for a process X'(t) on a partially ordered state
space S , to stochastically dominate S ( t ) .

BRIGHT AND TAYLOR

510

Definition : Let y and ij be probability density functions on a partially

ordered set S. Then TJ stocl~asticallydominates y (written y 5 , jj) if for
all monotone functions f (.) : S + R

Definition : The process ~ ( twith

) state space S and density function
~ ( t stochastically
)
dominates the process X ( t ) with state space S and
density function x ( t ) if
~ ( 0 5s
) a:(O)
2 ( t ) for all t .

+ x ( t ) 5,

(4.2)

If X ( t ) stochastically dominates X ( t ) then equation (4.2) holds V t 2 0

so we can take the limit as t -+ cm of both sides of the inequality and
obtain
x <$ a:
(4.3)

Downloaded by [UQ Library] at 10:20 24 April 2015

provided the equilibrium distributions x and Z exist.

The following theorem describes how to construct a process X ( t ) that
stochastically dominates X ( t ) . The proof of the theorem is left until the
appendix.

Theorem 1
Let X ( t ) be a continuous time LDQBD which satisfies condition 1.
Moreover, let X'(t) be a continuous time LDQBD on the same state space
as X ( t ) and with generator

where

LEVEL DEPENDENT QBD PROCESSES

511

Q
= ( Q Y ) ) ~ , j~ ,# i , k 2 o
(4.9)
and (v),~,and (v),,, are the minimum and maximum entries of the vector
v respectively. Then ~ ( tstochastically
)
dominates X ( t ) .
We define the dominating process of X ( t ) (when it exists) to be the
process X ( t ) given by the above theorem. X ( t ) can be given a physical
interpretation as follows. The vectors Qf-l'e consist of the sums of the
rates from a state in level k - 1 to a,ll the states in level k and the vectors
&f-l)e consist of the sums of the rates from a state in level k - 1 to all
the states in level k - 2. Call these rates the total up and down rates from
level k - 1 respectively. The rate at which X ( t ) moves from phase i in level
k 2 1 to phase j in level k 1 is given by the greater of the maximum
total up rate from level k - 1 divided by the number of states in level k 1
and the rate at which X ( t ) moves from state ( k ,i ) to state ( k 1, j ) . ~ ( t )
moves from states in level 0 to states in level 1 in the same way that X ( t )
moves from states in level 0 to states in level 1. The rate at which X ( t )
moves from phase i in level k 2 2 to phase j in level k - 1 is given by the
lesser of the minimum total down rate from level k divided by the number
of states in level k - 1 and the rat,e at which X ( t ) moves from state (k, i )
to state ( k - 1,j ) . X ' ( t ) cannot move from level 1 to level 0. X ( t ) between
states on level k in the same way that X ( t ) moves between states on level
k. It is easily seen that , y ( t ) is an irreducible Markov chain on the state
space { ( k ,j ) : k 2 1 , l 5 j 5 Adk}.

Downloaded by [UQ Library] at 10:20 24 April 2015

>

Let {en,n 2 0 ) and {&, n

1 ) be the marginal distributions over
the levels for X ( t ) and - y ( t ) respectively under their stationary distributions. Define m as an invariant measure for X ( t ) and partition m as
r7z = (ml,f i t z ,ms,.. .). If we define Ln = m , e then provided G-I =
Cr=l Ln < cc then an equilibrium distribution for X ( t ) exists and jn
is then given by .& = GI,,. Now in the case where for all k we have
j~r-l'e),~~~
~i:-~)e)~,~~
5 ( Q $ ~ ) )W,
~ ,j ~then {l,,n 2 1 )
2
i , j ~ 2 i . and
i l . f ~+
t 1
is the equilibrium distribution of t,he standard birth-and-death process on
state space { i 2 1 ) with transition rates q ( i ,j ) given by

(QP))

r1(0,1) = 0 ,

q ( i ,i 1) = ( ~ [ - ' ) e ) , ~i >
~ 0,
g(1,O) = 0.

q(i,i - 1 )
So {j,,, n

= (Q;-"e),,,,,,

> 0.

> 1 ) is given by
+

Note that a sufficient condition for G-' < cm is that ~ ( ii , l ) / i j ( i 1, i) 5

< 1 V i 2 I for some I since then the terms in the infinite sum will be

CY

BRIGHT AND TAYLOR

512

>

bounded above by a geometric series. If { I n ,n

1 ) exists, 5 must exist
and therefore x must exist since ~ ( tstochastically
)
dominates X ( t ) .
Now we want to find I<* such that CTZIc.en < E . It can be shown from
equation (4.3) that Cjm=,C j 5 C&, Fj V n 1 SO it is sufficient to find K*
such that
03
C jn<~.
(4.11)

>

n=K*

Often it is the case that we have an analytic form for the matrices Qi(k)
and hence an analytic
form for the rates q ( i ,j ) . In this case it is possible
to compute C:&. Cn analytically, using (4.10) and hence we can find K*
from (4.11).
When the rates q ( i , j ) are not l i n o ~ l analytically,
l
we can still compute
the quantity { & ( N ) ,1 n 5 N ) given by

<

>

Downloaded by [UQ Library] at 10:20 24 April 2015

>

where j 1 ( N ) is found using c:=~ j n ( N ) = 1. It is clear that for any N 1,

we have & ( N )
& V 1 n N and &(m) = jn. Hence we have
f n ( ~ 2
) Cn V l 5 n 5 N , N 21.

< <

We can then choose Ii* such that TIi.(lr'*) < E. The reasoning behind
this choice of Ii* is as follows. Firstly, &.(I<*) < E tells us that lip < E so
the equilibrium probability that S ( t ) is in level I<' is small (according to
some tolerance E ) . We want I<* such that the equilibrium probability that
X ( t ) is in level I<* or above is small. So when we choose I<* as above, we
assume that if the equilibrium probability that X ( t ) is in level I<* is small
then the equilibrium probability that S ( t )is in level I<* or above is also
small. We hope that this will be the case but there will be situations where
this will fail to hold. If it can be shown that q ( k , I."+ l ) / q ( k + 1, k ) cr < 1
holds V k 2 I<* then we have

<

and hence
So in this case we have an upper bound on
- en and provided cr is
small, the bound will be small. Here we can say xk(I<*)is approximately
equal to the equilibrium distribution of being in level k. In other cases more
detailed analysis of X ( t ) may be necessary to determine if this choice of

LEVEL DEPENDENT QBD PROCESSES

513

I(* is sufficiently large for xk(I<*)to approximate xk. At the very least,
xk(I<*)can always be interpreted as an upper bound for the equilibrium
distribution.
We now give an algorithm for determining I<* when condition 1 holds.
A l g o r i t h m 6 : Calculating { x k ( K * )0, 5 k 5 I<*)
w h e n c o n d i t i o n 1 holds
Set I<* = 1
Set ~ ( I C =
* )1
while ( j ( K * ) > E ) d o

I<* =I<*

+1

Downloaded by [UQ Library] at 10:20 24 April 2015

Calculate jI;. (I<-*)

Proceed with Algorithm 1 using the current value of I<*
As mentioned before, if condition 1 holds, then from any state, X ( t )
moves down to a state in the next highest level according to some positive
rate. In many cases condition 1 will not hold for all states but if it holds
for all states in and above level I?- then it is still possible to construct a
dominating process. This weaker form of condition 1 is as follows.
Condition 2 : V k 2

I? and V i 3 j

s.t ( & P 1 ) i j

> 0.

In this case define the two-dimensional continuous time Markov chain ~ ( t )

with generator matrix Q of the same form as Q in (4.4) with
~ ( 1 ~ )
and
given by

QP)

QF),

BRIGHT AND TAYLOR

5 14

Using a proof similar to that of Theorem 1 it can be shown that ~ ( t )

stochastically dominates X ( t ) provided condition 2 holds for X ( t ) . Note
that ~ ( tis) irreducible only on the levels I(,I?
1 , k 2 , . . .. In the
same way that we defined in(^) we can define { j , ( ~ )I?, 5 n N) and
this will have a similar form to & defined in (4.12). Then we choose I(*
such that ircw(I<*)
< E . SO when condition 2 holds, we have the following
algorithm.

<

5 k 5 I<*)
A l g o r i t h m 7 : Calculating {xk(I<*),O
w h e n condition 2 holds
Set I<* = I?
Set j ( K 3 ) = 1

Downloaded by [UQ Library] at 10:20 24 April 2015

while ( $I<*) > E ) do

Ir" = I<* + 1
Calculate

i~p
(I<*)

Proceed with Algorithm 1 using the current value of I(*

5. SOME NUMERICAL RESULTS

To illustrate how the algorithm works we present two simple examples

where there are only two phases at each level. We then consider a case
where there are 20 phases at each level. Consider an M / M / o o queue in
a random environment. This is an infinite server queue whose behaviour
is determined by the state of a in-state irreducible Marltov process Q. If
Q is in state i and there are k customers in the queue then the arrival
and service rates for the queue are A i and kpi respectively. If Q changes
from state i to state j , the arrival and service rates change to Xj and k p j
instantaneously. Such a queue can be modelled as a LDQBD where the
state (k, j ) represents k customers in the queue and Q being in state j.
The matrices Q f ) , ~ ( l k )and
are given by

QP)

LEVEL DEPENDENT QBD PROCESSES

515

where D ( v ) is a diagonal matrix with diagonal elements equal to the

elements of the vector v.

Downloaded by [UQ Library] at 10:20 24 April 2015

Condition 1 holds provided pl,p2 > 0 so we can use Algorithm 6 to

compute the equilibrium distribution. It is easy to show that the dominating process is positive recurrent for all values of X and p so the original
process is always positive recurrent. Consider the case where Q is a 2-stage
Markov process and Q is given by

and the parameter values are pl = 2, ~ 1 =

2 1, X1 = 40 and X2 = 5. For this
and the other examples below, we took Rk = Rk(n) s.t ( R k ( n )- Rk(nI)),,, < 10-12. In Algorithm 6 we took 6 = 10-lo which gave a value of 88
for I<*.In Table 1 we give the marginal probabilities l k .In order to see if
the value I<*
= 88 was sufficiently large, we used Algorithm 1 to calculate
the equilibrium distribution with I<*= 188. We found that the absolute
value of the differences of the two results was less than 10-l4 in all cases.
Next we altered the above example by setting p:, = 0. This corresponds to
a situation where the servers are all idle when Q is in state 2. Here both
conditions 1 and 2 fail to hold so we must, use Algorithm 4 to compute the
equilibrium distribution.
In order to malie an initial guess for Ii*in Algorithm 4, we used the
results for the previous example. Since we have set pz = 0, we expect
that we will require a value of I<*at least as high as in the previous
example. For this reason we guessed an initial value of I<* = 88. We also
= 88 proved to be
set E = 10-lo and we found that our initial guess of I<*
a sufficiently large choice. The marginal probabilities lk are presented in
Table 2. For this example we haven't proved that the process is actually
positive recurrent but the values in Table 2 certainly suggest that it is.
Obviously when analysing processes it is desirable to be able to determine
positive recurrence before we attempt to compute any of the Rk matrices.
How this is done in general is a topic for further research.
Now consider the PHIMloo queue. This is the infinite server queue
with a Marliovian service time distribution and a phase-type interarrival
time distribution. For more details on phase-type distributions, see 112,
Chapter 21. We considered the phase-type distribution
0.05841196Elo(1.168239125) 0.94158804Elo(18.83176088) which is the
mixture of 2 ten stage Erlang distributions considered by Ramaswami and
Latouche [13]. This queue can also be modelled as a LDQBD where the
state (k, j ) represents k customers and the interarrival process being in
phase j . Since we have a mixture of 2 ten stage Erlang distributions, there
are 20 phases at each level. If we let each customer be served a t rate p then

BRIGHT AND TAYLOR

Downloaded by [UQ Library] at 10:20 24 April 2015

Table 1

QP)

we have
= k p I where I is the identity matrix. The matrices ~ fand
)
are obtained by considering the phase-type arrival process. Condition
1 holds for this process so we used Algorithm 6 (with the same value of E
as before) to calculate the equilibrium distribution. We set p = 0.5 and
found I(* = 84. The marginal probabilities Ct are presented in Table 3.

QY)

In all three of the above examples the estimated values of IC* are larger
than necessary. That is, the algorithms tend to over estimate the value of
I<*. For example, for the PH/A4/oo queue the algorithm gave IC* = 84
yet from Table 3, the equilibrium probability of being in any state above
level 18 is negligible.
Latouche and Ramaswami [7] have observed in the level independent
case that it is possible to construct processes such that the algorithm re-

Downloaded by [UQ Library] at 10:20 24 April 2015

LEVEL DEPENDENT QBD PROCESSES

Table 2

BRIGHT AND TAYLOR

Downloaded by [UQ Library] at 10:20 24 April 2015

Table 3

quires an excessive number of iterations. In the LDQBD case this problem

is of course also relevant. However, given the physical interpretation discussed in Section 3, most physically meaningful processes should only require a relatively small number of iterations. Another potential problem is
the risk of overflow or underflow errors when recursively calculating the Rk
matrices. This problem was also observed in Gaver, Jacobs and Latouche's
[3] computation of the matrices ck.These problems are more likely to arise
when either Qf) or Qf) are very large or very small compared to the other
matrices in Q. For some cases where this problem arises, it is possible that
it could be eliminated via a numerical scaling tcchnique.

6. CONCLUSIONS
The algorithms presented in this paper allow the computation of the equilibrium distribution for LDQBD's. The basic form of the algorithms is to
choose a level I<*,calculate an invariant measure on the levels 0 2 k 2 I<*,
and then normalise this invariant measure. To calculate the equilibrium
distribution we must first calculate the matrices R k , k 2 0. The method
used to calculate these matrices has the advantage of having a nice physical
interpretation. The critical step in the algorithms is finding a sufficiently
large value of I<*. Currently more work is being done on more elaborate
methods for determining I{*as well as methods for determining positive
recurrence.

LEVEL DEPENDENT QBD PROCESSES

APPENDIX
Proof that ( - Q ~ " + ~ ) - R ~ + ~ Q ~exists
+ ~ ) ) -and
' contains non-negative
elements

Downloaded by [UQ Library] at 10:20 24 April 2015

To prove the result we consider the jump chain of X ( t ) which is a discrete

time LDQBD. The jump chain is obtained from X ( t ) by observing X ( t )
only at time points when there is a change of state. The jump chain has
transition matrix

where Qk = -ding[(Q1(k) )ii].To analyse the jump chain, the matrices of

interest are {R;, k 2 0) which are the minimal non-negative solutions to
the equations

~ f +'R:A~+') + R: [ R L , A $ ~ + ~=) ]R:

k 2 0.

The (i,j ) entry of R; is interpreted as the expected number of times the

process visits the state (k + 1,j ) before the process first returns to level k
given the process started in state (k,i). Define P[{!);(~+~,~)
as the taboo
probability that given that the jump chain starts in state (k, i), it is in state
(k 1,j) at time n and has not visited any state in level k in between.
Given the interpretation of R i , a simple probabilistic argument gives

u/,:+') = (A,( k + l ) )i,j + ( R ; + ~ A ~ +i ,~j.) )Using the above expression

for R:, u/,:+') is interpreted as the taboo probability that, starting from
state (k + 1,i) at time t = 0, the jump chain visits the state (k + 1,j) at
Let

some time t

> 1, without moving down to level k and without visiting any

BRIGHT AND TAYLOR

520

state at level k + 1 in between. Since the jump chain is irreducible, the

is strictly s ~ ~ b s t o c h a ~ tNOW
i c . define an absorbing Markov
matrix u(~+')
chain by considering the following. Observe the jump chain at all time
points where it is in level 1; 1 and before it has visited level k for the
first time. Then take the jump chain visiting level k for the first time as
absorption occuring. Since the jump chain is irreducible, we know that
there is a non-zero probability of absorption. Given the interpretation of
U(k+l),we have an absorbing Markov chain with transition matrix

where o is an rn x 1 vector of zeroes. By appealing to theorem 3.2.1

in Kemeny and Snell [5], it can be shown that the inverse (I - u(~+'))-'
exists. Furthermore, [5] also gives us that (I - U("+'))-' contains only
non-negative elements. From Ramaswami and Taylor [14] we have the
relationship Rk =
and by using this and substituting in the
values of A f ) , A\" and d f ) ,into ( I - u("+'))-',it is seen that the inverse
(-Q(:+') - R ~ + ~ Q & ' "exists
+ ~ ) )and
- ~ contains only non-negative elements.

@k~i@iil

Downloaded by [UQ Library] at 10:20 24 April 2015

P r o o f of Theorem 1
Before we prove Theorem 1 we present some theoretical results. For
more details on these results, see, for example, [17], [15] and [16].
Let
indicate a partial order and 3 indicate a quasi-order. For a set
S, 5 is a partial order on S if
(i) s 5 s Vs E S;
(ii) s 5 t and t

5 s imply s = t ;
(iii) s 5 t and t 5 u imply s 5 u,

and 4 is a quasi-order on S if
(i) s -: s is false Vs E S;

(ii) s 4 t and t 4 u imply s 4 u.

Ross [15] states that given a quasi-order 4 , a partial order 5 can be defined
by
x 5 y if and only if x 4 y or x = y.
(0.2)
If S is the state space of a stochastic process endowed with a partial order,
then for any subset r of S, define

LEVEL DEPENDENT QBD PROCESSES

52 1

= {y]x 5 y for some x E l?}.

I' is then said to be an k r e a s i n g set if I? = I'T. The following is a result

due to Massey [S].

Lemma A
Let X l ( t ) and Xz(t) be uniform Markov processes on the state space
S with transition rates ql(i,j ) and qz(i,j) respectively (i.e for m = 1,2,
C j qm(i,j) is bounded V j ) . If 5 is a partial order on S then Xl(t) stochastically dominates X2(t) if and only if for all x 5 y in S and all increasing
sets r, the following hold

Downloaded by [UQ Library] at 10:20 24 April 2015

and

Brandt and Last [2] extended Lemma A by removing the requirement that
Xl(t) and &(t) be uniform. We now use this extended form of Lemma
A to prove Theorem 1. First define the relation 4 by (i, j ) 4 (k, 1) if and
only if i < k. It is easy to show that 4 is a quasi-order. Define the relation
4 according to (0.2).
Now according to this partial order, the increasing sets are of the form

where I {1,2,. . .Mi-l} is some non-empty set of indices. That is, the
increasing sets contain all the states in levels i and above and a t least one
state in level i - 1. Let @(i)= UI Q i S I and define O by

Thus @(i)is the set of all increasing sets r such that (e, j) E r V e 2 i , 1 5
j 5 M!, and (i - 1,j) E r for at least one 1 I j 5 Mi-1 and O is the set
= q5 V i # j and that the
of all increasing sets. Note that @(i)n @(j)
cardinality of O is infinite but the cardinality of @(i) is finite V i 2 1.
Tn order to prove Theorem 1, we need to show that conditions (i) and
(ii) hold in Lemma A with ql(x, z ) given by the matrices Q f ) ,
and
and q2(9,2 ) given by the matrices ~ f )Q\')
, and
of equations
(4.6)-(4.9).

~r)

QP)

QY)

BRIGHT AND TAYLOR

522

First look at the case x 4 y E r . Take r E @ ( i ) ,i

y = (i,k ) . We have

> 1 and x

= ( i - 1, j ) ,

and

so it is clear that (i) holds. Note that with I? E @ ( i ) , i > 1, (i) holds
trivially for all other x 4 y E I?. Consider r E @ ( I ) and x = (0, j ) ,
y = (1,k). We have

Downloaded by [UQ Library] at 10:20 24 April 2015

and

so (i) holds. Again for all other x 4 y E

r , (i) holds trivially.

Now consider x = y E I'. For r E Q ( i ) i, > 1 the only non-trivial cases

of x that we must consider are x = ( i ,j ) and x = ( i - 1,j ) . For x = ( i ,j )
we have
~ l ( x2 ), = C ( ~ f ) ) j , t
f$r
ts I
(i-l,t)$r

and

from (4.7). For x = ( i - 1,j) we have

and

LEVEL DEPENDENT QBD PROCESSES

523

from (4.9) and (4.7). For I' E @ ( I )we need only consider x = (1,j) and
x = (0, j ) . For x = (1, j) we have

and

and for x = ( 0 , j ) we have

Downloaded by [UQ Library] at 10:20 24 April 2015

and

so (i) holds in both cases.

The proof that (ii) holds for all x 5 y E S and all increasing sets
similar to the above. This completes the proof.

I? is
0

Acknowledgments
The authors would like to thank Guy Latouche, Nigel Bean and the
anonymous referees for suggestions that helped in the revision of this
manuscript. The first author aclmowledges The Australian Telecommunications and Electronics Research Board for supporting this research.

References
[I] Anderson, W. J., Continuous-Time Markov Chains, An ApplicationsOriented Approach, Springer-Verlag (1991).
[2] Brandt, A. and Last, G., On the pathwise comparison of jump processes driven by stochastic intensities, preprint, Humbolt-Universitat
zu Berlin (1993).
[3] Gaver, D. P., Jacobs, P. A, and Latouche, G., Finite birth-anddeath models in randomly changing environments, Adv. App. Prob.

BRIGHT AND TAYLOR

524

[4] Hajek, B., Birth-death processes on integers with phases and general
boundaries, J. App. Prob. 19 (1982), 488-499.
[5] Kemeny, J. G., and Snell, J. L. Finite Markov Chqins, Springer-Verlag
(1976).
[6] Latouche, G., Algorithmic analysis of a multip~ogramming-multiprocessor computer system, J. Assoc. Comp. Mach,. 28 (1981), 662679.
[7] Latouche, G. and Ramaswami, V., A logarithmic reduction algorithm
for quasi-birth-and-death processes, J . App. Puob. 30 (1993), 650-674.
[8] Massey, W. A., Stochastic orderings for Marlcov processes on partially
ordered spaces, Mathematics of Operations Research 12 (1987), 350367.

Downloaded by [UQ Library] at 10:20 24 April 2015

[9] Nelson, R. D., A performance evaluation of a general parallel processing model, Performance Evaluation Review 18 (1990), 13-26.
[lo] Neuts, M. F., The M/M/l queue with randomly varying arrival and
service rates, Opsearch 15 (1987), 139-157.
[ll]Neuts, M. F., Further results on the M/M/1 queue with varying rates,

Opsearch 15 (1987), 158-168.

[12] Neuts, M. F., Matrix Geometric Solutions in Stochastic Models, Johns
Hopkins (1981).
[13] Ramaswami, V. and batouche, G., An experimental evaluation of the
matrix-geometric method for the GI/PH/l queue, Commun. Statist.
- Stochastic Models 5(4) (1989), 629-667.
[14] Ramaswami, V. and "paylor, P. G., Some properties of the rate matrices in level dependeqt quasi-birth-and-death processes with a countable number of phases, submitted for publication.
[15] Ross, I(. A . and Wright, C. R. B., Discrete Mathematics, Prentice
Hall, 2 edition (19881,
[16] Smeitink, E., Stochastic Models for Repairable Systems, PhD thesis,
Vrije University (1992).
and Other Stochastic
1171 Stoyan, D., Comparison Methods for Qz~ez~es
Models, Wiley series in probability and mathematical statistics (1983).
[18] Ye, J., and Li, S., Folding algorithm: A computational method for
finite QpP processes with level-dependent transitions, IEEE Trans.
Commun. 42 (1994), 625-639,

LEVEL DEPENDENT QBD PROCESSES

525

[19] Zulierman, M., Applications of matrix-geometric solutions for queueing performance evaluation of a hybrid switching system, J. Aust.
Math. Soc. Ser. B 31 (1989), 219-239.

[20] Zukerman, M. and Kirton, P., Applications of matrix-geometric solutions to the analysis of the bursty data queue in a B-ISDN switching
system, Proc. GLOBECOM '88 (1988), 1635-1639.
Received:
Revised:
Accepted:

8/21/1994
4/4/1995
41411995

Downloaded by [UQ Library] at 10:20 24 April 2015

Recommended by Marcel F. Neuts, Editor