Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, noncommercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=apt.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a notforprofit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
Applied Probability Trust is collaborating with JSTOR to digitize, preserve and extend access to Advances in
Applied Probability.
http://www.jstor.org
Adv.Appl. Prob. 36, 455470 (2004)
Printed in NorthernIreland
@AppliedProbabilityTrust2004
JEANBERNARDGRAVEREAUX* AND
JAMESLEDOUX,*** INSAand IRMAR,Rennes
Abstract
Inthispaper,we considera failurepointprocessrelatedto theMarkovian arrivalprocess
definedby Neuts.Weshowthatit convergesin distribution to a homogeneous Poisson
process.Thisconvergence takesplacein thecontextof rareoccurrences of failures.We
alsoprovidea convergencerateof theconvergence in totalvariationof thispointprocess
usinganapproach developedby Kabanov, LiptserandShiryaevforthedoublystochastic
Poissonprocessdrivenby a finiteMarkovprocess.
Keywords: Compensator;softwarereliability;
Markovianarrival process;doublystochas
tic process
2000Mathematics SubjectClassification:
Primary 60G55;60J25
Secondary 60J75;90B25;93E11
1. Introduction
This work originates in Littlewood's papers [12] and [13] on a Markovtypemodel for
reliability assessment of modularsoftware. Basically, for a piece of software with a finite
numberof modules:
1. the structureof the software is representedby a finite continuoustimeMarkovchain
(CTMC) (Xt)t, where Xt is the active module at time t;
2. when modulei is active,failurestimes arepartof a homogeneousPoisson process (HPP)
with intensity
/(i);
3. when controlswitches frommodulei to module j, a failuremay happenwith probability
g (i, j);
4. when anyfailureappears,it does not affectthe softwarebecausethe executionis assumed
to be restartedinstantaneously.Such an event is referredas to a secondaryfailurein [9].
An extension of such a model was considered in [9], taking into account the influence of
failures on the execution dynamic of the software and dealing with the delays in recovering
an operationalstate. Transientanalysis was providedby means of resultsfrom [10]. Roughly
speaking, the failurepoint process was a Markovarrivalprocess (MAP) as definedby Neuts
(see e.g. [14]). It is well knownthatwe then obtainas particularinstancesof ourfailureprocess
a phasetyperenewalprocess, a doubly stochasticPoisson process with a stochasticintensity
drivenby a CTMC(also called a MarkovmodulatedPoisson process (MMPP)in the queueing
literature),etc.
455
456 J.B.GRAVEREAUX
ANDJ. LEDOUX
X= Q(i, j)(i, j) +
r(i)•
(i)'
i distributionand Q the generatorof the CTMC (Xt)
where 7r is the stationary
(assumed to be irreducible).
This statementis well known in the softwarereliabilitycommunityand has been widely used
to justify the hierarchicalapproachto modelling modularsoftware (see e.g. [6] for details).
However,to the best of the authors'knowledge,no proof of this fact is reportedin the applied
probabilityliterature.The aim of this note is to provideprecise statementsand proofs for the
asymptoticof the generalfailurepoint process definedin [9] for which Littlewood'smodel is
a particularcase. Specifically,we show that the countingprocess correspondingto this point
process convergesin distributionto the countingprocess of an HPP when failure parameters
tend to zero but at a specific timescale. Roughly speaking,we introducea small parameter8
into the failureparametersand the convergencetakes place at timescale t/e (in other words,
when failure parametersare small and on a large horizon time). Proving this result is easy
using the criterionof convergencein distributiongiven in [8] for instance. It is based on the
convergencein probabilityof compensatorscorrespondingto the variouscountingprocesses.
In fact, the countingprocess convergesin variationto the countingprocess of an HPP and the
convergencerate will be shown using a methoddevelopedin [8] for the MMPP.
Note that the class of MMPPs is widely used to model trafficstreamsfor communication
systems. It is easily seen that dealing with the presentissue is equivalentto consideringan
asymptoticMMPP with a fast modulatingMarkovchain. Thus, we retrievethe more or less
known fact that, when jittering takes place, the arrivalprocess tends to Poissonian (see [15,
p. 116] for a partialdiscussion).
The articleis organizedas follows. Section 2 recalls some backgroundon the point process
studiedhere. In addition,the compensatorof the point process is derivedin a straightforward
manner.In Section 3, we reportresults aboutconvergencein distributionof the point process
to an HPP. Connectionto the problemof fast modulationin the case of an MMPP is briefly
addressedin Subsection3.3. The rate of convergencein total variationof the point process is
given in Section 4. AppendixA recalls an estimate of the convergencerate of the singularly
perturbatedgeneratorprovided in [18]. The derivationof an inequality used in the text is
reportedin AppendixB.
componentat time t for a failurefreesystem, that is, X is the execution process. The vector
a = (a(i))isA denotesthe distributionof randomvariableX0. The new parametersX(i, j) and
X(i) (i, j E AM)are definedto have the same meaningas tg(i, j) and g (i). But, when failure
of this type happensin module i or duringa transitionfrom module i, there is a probability
p(i, k) that execution restartsin module k. So, for each i E NM,(p(i, k))kE is a probability
distribution.Such an eventwas referredas to a primaryfailurein [9]. Simultaneousoccurrence
of primaryand secondaryevents is neglected. For simplicity,we do not considerthe delay in
recoveringan operationalstate as in [9].
Then, taking into account failure occurrences, the random variable Xt* gives the active
componentat time t. The randomvariableNt counts the numberof (primaryand secondary)
failuresin the interval]0, t] (No = 0). Thus, N = (Nt)t is the countingprocess of the failure
point process. Underthe variousassumptionsin [9], the bivariateprocess Z := (Nt, XT)t can
be consideredas a CTMCover the statespace S = N x . Its infinitesimalgenerator,denoted
by G, has the special structure
Do DI 0
G 0 Do DI
using a lexicographicorderon the state space S. The matricesDo and D1 are definedby
+1 k)p(ij)+ j) if j,
[(i) Q(i,
k)X(i, j)[1X(i,
Q(i, j)]pt(i,
+ Q(i,k)X(i,k) p(i, i) + [(i) if i = j.
W.(i)k/i
Note thatmax IG (x, x)I < +oo. The structureof the generatorG shows thatN is the counting
processof a MAP. Finally,X* is a CTMCwith statespaceA, initialdistributiona andgenerator
Q* := Do + D1.
The CTMC X* is supposed to be right continuous with left limits (cadlig). If the failure
parametersare assumedto be such that X(i, j) < 1 for any (i, j), then X* is irreduciblesince
X is. This assumptionis not very stringent.
Nt= 1{Zs_=x,Zs=y}
(x,y)ET O<s<t
At(x, y) j s (x, y) ds
is the ?(z)compensator (7F(z) dual predictableprojection)of N(x, y). Then it is easily seen
that M = (Mt)t with Mt = Nt  At and
At
= j At(x,y)
(x y) ~ T
Poissonapproximation 459
t := l{z,_=x)G(x,y)
(x,y)eT
= X X DI(i, j)
1{(Nt_,X*_)=(n,i)}
n>O0ieM jeM
= = D(X*,
lxt=i)[ Dl(i, j)] j).
In the case of Example 2.1, the intensity of the countingprocess correspondingto Little
wood's model is
AM(X_)+ Q(Xt, j)(Xt,
jAxt j).
In the case of Example2.2, we retrievethe wellknownexpressionfor the $(N,X)intensity
for an MMPP (Xt).
Q=D(1 1 (1) 0
= 1 )' 0D1 (2)
reD
t
1T
=
[cosh
2
sinh t
( ete((1)(1)+(2)(2))et
NE) = Nt/e,
whereNt countsthe numberof failuresin the interval]0, t] for the reliabilitymodel of Section2
with the system (3.1) of perturbatedfailure parameters. The matrices D(0e, D E) are those
associatedwith the model N. Note that
A() D (X*_,j) ds
=
f B(X*_,j) ds + 82 L(Xj, j) ds (from(3.2))
jEeM jEiM
Ae) +
t r(i) 1 B(i, j) = Xt almost surely as E  0,
iEM j]EM
with Xas in (1). In particular,this implies thatA) convergesin probabilityto Xt. We recognize
the compensatorof an HPP with intensityX. It follows fromTheorem 1 of [7] that
as e + oo,
N()  P
where P = (Pt)t is the counting process of an HPP with parameterX. We have shown that
Theorem 3.1 below holds for an MMPP and Littlewood's reliability model (Examples 2.1
and 2.2).
Since convergencein the L2normimplies convergencein probability,the following lemma
will give the convergencein probabilityof compensator to Xt as e tendsto 0 for the general
reliabilitymodel of Section 2. Ate)
limE e] r(i))ds = 0.
e+0 0O
(1{xs=i}
Proof The perturbatedgeneratorQ* = D) + Di) of X* can be decomposedas follows:
Q* = Q + Re, (3.4)
where
Ix(i) + k)X(,
k)(i p(i j)  Q(i, j)X(i, j) if j i,
Q(i,
R(i,j)= ki
X(i) + E Q(i, k)X(i, k) p(i, i)  X(i) if j = i.
I k~i
The change of variablesu = se gives that
A(8) A=
 (Xs) ds
This fact is known and has been investigatedfor various applications. The closest context
to reliability theory is [1], where the underlyingPoisson process is either homogeneous or
nonhomogeneous.Proofsarebasedon asymptoticexpansionof the transitionsemigroupof the
bivariateMarkovprocess Z(e). Introductionof timedependentintensityis not relevantin our
context. Indeed, the decrease of failureparameters,or the reliabilitygrowth,is alreadytaken
into accountby the small parametere.
The case where the modulatingprocess is a finite nonhomogeneousMarkovprocess X is
addressedin [4]. The asymptoticprocess is a nonhomogeneousPoisson process with suitable
ergodicityassumptionson X.
From a strictlymathematicalpoint of view, the issue addressedin this paperis equivalent
both to a fast modulatingMarkovprocess X* (with generatorQ*/E) and to the introduction
of a small scalare only in the failureparametersA(., .) correspondingto a transitionbetween
states. Speeding up the modulatingMarkovprocess X* at rate 1/E implies a speeding up of
the numberof transitionsbetween states of X*. Therefore,we have to compensatefor this
'explosion'by introducinga small factorE in /(., .).
4. Convergence rate
We provide in this section an estimate of the convergencerate of the finitedimensional
distributionsof N(E)to thatof an HPP with intensityas in Theorem3.1. The countingprocess
of the HPP is denotedby P = (Pt)t. Note that Xin (3.5) is the scalarproduct(.r, B 1T). We
can write the limit compensatoras
At = (7r, B 1T)t. (4.1)
From (3.3), the F(N('), X*,E)compensatorof N(e) is
A(E) 
i (B + EL) 1T(i) dO
ds
'l{Xs*_=i}
 L(B + EL)
1T)(i)
] ds
(setting u =
sE)
l{x.•=i}
IP{N) = k}P{PT= kI
kENn
(see [17]). For a locally boundedvariationfunctiont f (t), the total variationin the interval
[0, T] is
n
where P ([0, T]) is the set of all finite subdivisionsof the interval[0, T].
In this section, we show that the finitedimensionaldistributionsof N(E) convergesin total
variation to those of an HPP with intensity X at rate e. The proof is borrowed from [8,
Theorem 6.1], where a similar result is given for an MMPP. This is heavily based on the
following estimate of the total variationbetween finitedimensionaldistributionsof N(e) and
P [8, Theorem3.1]:
<  A),
dTV(G.(NE)),Z(PT)) Evar[o,T](A(e)
where A(e) is the F(N(*))compensatorof N(e). The term A(e) is obtainedfrom (4.2) and
with
I(P{X'e = i N(e)))iEA
(see [11, Theorem 18.3]). Hence, the F(N(8))intensityof N(e) is
dTv((NNE)), X
?(P)) E [,
var[0,T]( (s(e)  B 1T) + e (s(), L 1T) ds
= E I((e)  r, B 1 ) + L 1 )Ids
e(Ys_),
<E  r, B 1T) Ids + eE I(s), L 1T) ds
I(_(e)
< E
0IT
I  r, B1T)I ds + eCT
(Ys(e) (.5
(4.5)
464 J.B.GRAVEREAUX
ANDJ. LEDOUX
(since Y(E)is bounded). Since and 7r are stochasticvectors, it follows from (B.1) below
that
is
7 B  *,r(4.6)
1(Ys  r, 1T)[ < 2 11
where 3 := max(B 1T(i))  min(B 1T(i)) and II IIIis the linorm. Hence, it remains to
estimatein (4.5) the convergencerate of 1Iis• Fr Ii to 0 when e  0. The firststep consists
in writinga filteringequationfor the vector Y0
4.1. Filtering equation for y(E)
)
Recall that =
(l{X*,e.=i})iE(. Note that each componentY (i) of Y( is a bounded
randomvariable. We now basically follow ChapterIV of [2].
YtE)
Lemma 4.1. Define Y E[Y(e) Let a be the probabilitydistributionof
Then,for all t > 0, ()]. Xof*.
It
jd8) =a + i(t
Y) 1 Tf Q*ds +
v3)*
Vs(dN  X<ds)(4.7)

ds),
(4.7)
o ?(dN4s)
where (s) is the of N(') given in (4.4) and
F•(N(E))intensity
Proof Recall thatthe Markovprocess X*,' has the generatorQ*Q'= Q*/E. It follows from
the Dynkin formulathat
^ *
1 rt
s ) Q ds +M,
0. =Y (6. +
Yt f0
I
where M = (Mt) is an F(N('E)martingale. Now, Theorem 17 of [2, ChapterIII] gives us the
following representationof the F(NE))EmartingaleM:
Gs(dN
(e) 
)ds),
where •(E) is the 3F(N() intensity of N(W)and G = (Gt) is an .F(N E)predictable process
called the innovationsprocess. We also know from Theorem2 of [2, ChapterIV] that Gt
 + G3,t, where the ith entryof the vectors G1,s and G3,s must be computedfrom
Gi•t )
EJ CsY (i)•• ds 
E Y
(i)X0•• s
f(00T
Cs
= E CsY(s)(i)(Y E, (B + eL) )ds (from(4.3))
ft
= E CsY()(i)((B + eL) 1T)(i) ds
where Nt() (x, y) is the cumulativenumberof transitionsof the bivariateprocess (N(e), X*,e)
up to time t. Since AMs(i) = A Ys) (i),
CsYs(e)>(i)AN(e)
 CsYs(e)(i) AN e)((n, j), (n + 1, k)).
O<s<t O<s<t n=0 jEAt kEAM
(i)jAN(e)
=j 3 Y2(j)dN~E)((n, j), (n + 1, i))
CsAMs 0 Cs
O<s<t n=0 jEA
 CsjYs(
C2(i) dNe).
466 J.B.GRAVEREAUX
ANDJ. LEDOUX
Since the processes C and y(e) are F(N('),x*,E)predictable, it follows from the definitionof
the compensator,(2.2) and (4.3) that
E L CsAMs(i)ANe)
O<s<t
= EE Cs
Y i) + eL(j, i))  Y (B + eL) 1T) dsds
c[ C ((j)(B(j, i)
Y(j)(B(j, eL(j, i)) Ys (i (B +Li
(i)(Y5s_),
EEJ
= s (B (j, i) + EL(j, i)) Y (i) (B (i, j) + eL (i, j)) ds
Ys(j)
jeM
je:
=E Cs j Ei Yf2(j)(B(j, i) + eL(j, i))  Y i
i) j (B(i, j)) ds
jii Y [
(j)(B(j, i) + eL(j, i))  f(s0) j]i + eL(i, j))]ds
(i)L:(B(i,ij)
=sEE
t t
l
= a + 1t  (UsVs)Q*ds (dN ds).
+
Second,notethat(177) UrVt)t is a solutionof thehomogeneouslineardifferentialequation
y(t)=  y(s)Q* ds
X
= (r, B 1T) = 7r(i)[I(i)+ Q(i,
X(i)+ E j)Q(Q(i, j) + L(i, j)),
where 7r is the probability distributionsuch that r Q = 0. For any T > 0, there exists a
constant CT such that
dTv(?(N(e)), ?(PT))
< Cre.
Proof Let us recall that
E
t
dTv(?(NTe)), ?(P)) 11j

7rillldt+ C1,e
(see (4.5) and(4.6)). Wejust haveto controlthe firsttermon therighthandside of the inequality.
Since v(e) 1T = 0, using (4.14) we can write

t(e) r = aexp(Q*t/e)  + (e)[exp(Q*(t  s)/e)  1
r](dNs()e)
ds).
t 
+E v(e)[exp(Q*(t s)/e)

1r r](dNs()  (e)ds) dt.
(4.15)
E   1T
v(E)[exp(Q*(t s)/e) r](dN(e)  (e) ds)
h(E)ds)
<E Iv(')[exp(Q*(t  s)/e)  1 r]IIl(dN(E)+
since j1) is the (N''E))intensity of N(E) and v(E is F(NE))predictable. We recall that
IxMII < 1lxlM for anyvectorx andmatrixM, where IMliM
lIIMII := maxi()j IM(i,j)1)
is the matrix 1norm.Now,
E  s)/e) _1Tr]l1k) ds
IIvs_[exp(Q*(t
S(C5+ EC6) f( + exp((t  s)p/e)) ds
We deduce from the previous estimate (and Fubini's theorem) that the second term on the
righthandside of (4.15) is such that
(0
(exp(Q*(t  s)/e)
E  ) dt
fT 1T r)(dN)  ds)
_
< (Cat+ C9)edt = C3,TE (fore < 1).
Remark 4.1. Withrespectto ourTheorem3.1, note thatTheorem1 of [7] would, in fact, give
convergencein distributionof the countingprocess N(W)to the Poisson process P in the space
of all counting processes equippedwith the Skorokhodtopology. Moreover,convergencein
variationalso takes place in this space. Indeed,the distancein total variationover the interval
[0, T] betweendistributionsof N(E)and P is also boundedfrom aboveby E var[o,r](A(E) A)
(see [8, Theorem4.1]). Thus, it follows from Theorem4.1 thatthe rate of convergenceis e.
Remark 4.2. The orderof the convergenceratein Theorem4.1 cannotbe improvedin general.
This follows from [4, Section 5, Example 1], where the authorsreporta lower bound for the
distancein variationthathas order1 in e for a Poissonprocessmodulatedby a twostateMarkov
process.
Poissonapproximation 469
IIexp(Q(")t)  1T
K(e + exp(pt/e)),
7rill (A.1)

Ilatexp(Q(e)t) xlr1 < K(e + exp(pt/e)) (A.2)
Ilil 
ot2
Il = (otl  U2)+ 1T +(otl  o2) 1T = 2(otl  a2)+ 1T = 2(aot a2) 1T,
where II illiis the 11norm.We may then write the vectoral  as (ul  u2)IIcll U2111/2,
o2
where u i, u2 are the stochasticvectors
  2)
(atl (otl
Ul o2)+ 1T'
a2)+ U2
(at  (l  2) 1T'
 2 8
 = 10li 2111I(Ul  (  (B.1)
I(Oti U2, 1V)I 2 u2, V1)1 2 Illil 22111.
References
[1] ANISIMOV, V. V. ANDSITYUK, V. N. (1979). Asymptoticbehaviorof inhomogeneousPoisson flow with conductor
function,dependenton a small parameter,controlledby a Markovchain. Cybernetics15, 526532.
[2] BREMAUD,P. (1981). Point Processes and Queues. Springer,New York.
[3] CINLAR, E. (1975). Introductionto StochasticProcesses. PrenticeHall,EnglewoodCliffs, NJ.
[4] DI MASI,G. B. ANDKABANOV, Y. (1993). The strongconvergenceof twoscale stochasticsystems and singular
perturbationsof filteringequations.J. Math.SystemsEstimationControl3, 207224.
[5] Etov, I. I. ANDSKOROKHOD, A. V. (1969). Markovprocesses with homogeneoussecond component.II. Theory
Prob.Appl. 14, 652667.
[6] GOSEVAPOPSTOJANOVA, K. ANDTRIVEDI, K. S. (2001). Architecturebasedapproachto reliabilityassessment
of softwaresystems. PerformanceEvaluation45, 179204.
[7] KABANOV, Y. M., LIPTSER,R. S. ANDSHIRYAEV, A. N. (1980). Some limit theoremsfor simple point processes
(martingaleapproach).Stochastics3, 203216.
[8] KABANOV, Y. M., LIPTSER,R. S. ANDSHIRYAEV, A. N. (1983). Weakand strongconvergenceof the distributions
of countingprocesses. TheoryProb.Appl. 28, 303336.
[9] LEDOUX, J. (1999). Availabilitymodeling of modularsoftware.IEEE Trans.Reliab. 48, 159168.
[10] LEDOUX, J. ANDRUBINo,G. (1997). Simple formulaefor countingprocesses in reliabilitymodels. Adv.Appl.
Prob. 29, 10181038.
[11] LIPTSER,R. S. ANDSHIRYAEV, A. N. (2001). StatisticsofRandom Processes, Vol. II, 2nd edn. Springer,Berlin.
470 J.B. GRAVEREAUX AND J. LEDOUX
[12] LITTLEWOOD, B. (1975). A reliabilitymodel for systemswith Markovstructure.J. R. Statist.Soc. C 24, 172177.
[13] LITTLEWOOD, B. (1979). Softwarereliability model for modularprogramstructure.IEEE Trans.Reliab. 28,
241246.
[14] NEUTS,M. F. (1989). StructuredStochasticMatrices of M/G/1 Typeand TheirApplications.Marcel Dekker,
New York.
[15] NEUTS,M. F., LIU, D. ANDNARAYANA, S. (1992). Local Poissonificationof the Markovianarrivalprocess.
Commun.Statist.Stoch.Models 8, 87129.
[16] PACHECO, A. ANDPRABHU, N. U. (1995). Markovadditiveprocesses of arrivals.In Advancesin Queueing,ed.
J. Dshalalow,CRC Press, Boca Raton,FL, pp. 167194.
[17] SERFOZO, R. F. (1990). Pointprocesses. In StochasticModels (HandbookOperat.Res. ManagementSci. 2), ed.
D. P. Heymanand M. J. Sobel, NorthHolland,Amsterdam,pp. 193.
[18] SETHI, S. P. AND ZHANG, Q. (1994). HierarchicalDecision Making in Stochastic ManufacturingSystems.
Boston, MA.
Birkhdiuser,