Microelectronics and Reliability, Vol. 15, pp. 633 to 636. Pergamon Press, 1976.

Printed in Great Britain

MARKOV RENEWAL PROCESSES WITH SOME

NON-REGENERATION POINTS AND THEIR
APPLICATIONS TO RELIABILITY THEORY
TOSrlIO NAKAGAWA
Department of Mathematics, Meijo University, Nagoya 468, Japan
SHUNJI OSAKI
Department of Industrial Engineering, Hiroshima University,
Hiroshima 730, Japan

Abstract--The conventional Markov renewal processes are based on the regeneration points at which
the process enters each of the states. However, it is of great interest to discuss the Markov renewal
processes with some non-regeneration points. This paper discusses such Markov renewal processes
and gives their stochastic behavior. A few examples are presented as applications of redundant repairable systems in reliability theory.

I. I N T R O D U C T I O N

cations of such M.R.P.'s to analysis of redundant

repairable systems including some non-regeneration
points. We define new mass functions and derive (i)
the first-passage time distributions, (ii) the transition
probabilities, and (iii) the renewal functions, with the
aid of renewal-theoretic arguments.

Semi-Markov Processes (S.M.P.) were first introduced by L6vy and Smith, independently, in 1954 and
1955. Pyke I-8,9] defined Markov Renewal Processes
(M.R.P.) which are equivalent to the family of
S.M.P.'s. Further, he gave the matrix forms for the
transition probabilities, the first-passage time distributions, and the renewal functions, and discussed
2. D E F I N I T I O N S A N D N O T A T I O N S
their limits for an M.R.P. with finitely many states.
Many redundant repairable systems in reliability
We define a stochastic process {Z(t), t > 0}, where
theory can be solved by using the techniques of Z(t) = i denotes that the process is in state i at time
S.M.P.'s or M.R.P.'s. For instance, Takfics 1-10] ana- t. The state is denoted by integers 0, 1, 2 . . . . , n ~ S
lyzed an n-unit paralleled redundant system by solv- and the state space is finite. Let N~(t) denote the
ing a telephone trunking problem that generates an number of times that the process is in state i, not
S.M.P. Barlow and Proschan [1, p. 151] showed the including the initial state, in the interval (0, t).
We define
possibilities of analysis by stochastic processes associated with repairman problems. Osaki [7] discussed
Po(t) = Pr{Z(t) = jlZ(0) = i},
(1)
some redundant repairable systems by investigating
the relationship between M.R.P.'s and signal-flow
which denotes the probability that the process is in
graphs. Thus, an S.M.P. or an M.R.P. is one of the
state j at time t if the process starts in state i at time 0.
most powerful mathematical techniques for analyzing
Further, we define
redundant repairable systems. However, it is very
complicated to investigate some states of systems havGo(t ) - Pr{N~(t) > 01Z(0) = i},
(2)
ing non-regeneration points, i.e. some epochs at which
the process enters some states which are not regeneMo(t) =- E { N ~(t)IZ(O) = i}.
(3)
ration points. For example, if we are interested in
the availability for a two-unit standby redundant
Here, Gij(t) is the first-passage time distribution from
system, we have to consider the epoch at which the moment the process enters state i at time 0 to
system failure occurs, which is no longer a regenera- state j entered at time t, and M o ( t ) is the mean
tion point under some conditions (see Barlow and number of visits to state j during (0, t) if the process
Proschan [2], p. 203). Then we show how to analyze starts in state i at time 0, which is referred to as the
such situations in detail. Of course, such a two-unit "renewal function."
standby redundant system can be analyzed as shown
Throughout this paper we assume that an M.R.P.
in the sequel.
under consideration has only one (positive) recurrent
This paper proposes unique modifications of the class, because we restrict ourselves to applications
M.R.P.'s discussed by Pyke [9] and considers appli- to reliability theory. Assume also that each Gjl(t ) is
633

634

TOSHIO NAKAGAWAand SHUNJIOSAKI

theoretic arguments, the first-passage time distributions are
Gol(t) = Qol(t),

(5)

Gofl) = Q~o~'2 ..... J- l~(t),

j = 2, 3 . . . . .

n,

Goo(t) = Q~j2 ...... )(t)

(6)
(7)

The renewal functions are

Mol(t) = Qol(t) + Qolo'2 ...... '(t)*Mol(t)

= Qol(t) + Goo(t)*Mol(t),
Fig. 1. The state-transition diagram for the Type 1-M.R.P.,
where represents a state with regeneration point and
[] a state with non-regeneration point.
non-lattice for any state j. It follows by Pyke [9] that
1ii =-

fo

Moi(t) = Q~olj2 ..... J-l~(t) + Goo(t)*Mo,(t),

j = 2,3 . . . . . n,
Moo(t) = Goo(t) + Goo(t)*Moo(t).

(10)

Pol(t) = Q o l ( t ) - Q~ol)2(t) + Q~o16:...... )(t)*Pol(t)

and

= Qol(t) - Q{~(t) + Goo(t)*Pol(t),

(11)

:M~ -- lim Mu{t)/t ,

Poj(t) =

which are independent of the initial state i.

Consider a case where epochs at which the process
enters some states are not regeneration points. Then,
we partition the state space S into S = S*voS +
(S* c~ S + = 0), where S* is the portion of the state
space such that the epoch entering state i(i ~ S*) is
not a regeneration point, and S + is such that the
epoch entering state i(i ~ S +) is a regeneration point,
where we assume that S* is not the empty set.
Define the mass function Qo{t) from state i(i ~ S +)
to state j ( j ~ S) by the probability that after entering
state i, the process makes a transtion into state j,
in an amount of time less than or equal to time t.
However, it is impossible to define mass functions
Qu(t) if i ~ S*, since the epoch entering state i is not
a regeneration point. Define new mass functions
Q!~,,k ...... k~)(t) which is the probability that after
entering state i(i ~ S+), the process next makes transitions into states kl, k2 . . . . ,km (k~, k2 . . . . , km~S*) and
finally enters state j ( j ~ S), in an amount of time less
than or equal to time t. Moreover, we define
Hi(t) - ~ Qu{t),

i~ S+,

(4)

which is the unconditional distribution of the time

elapsed from state i to the next state entered (possibly
i itself).

n1,2 .....
~oi

J- 1)m
,.

I-M.R.P.

Consider an M.R.P. with n + 1 states, wl~ch consists of S + = {0} and S* = {1, 2 . . . . . n}(see Fig. 1).
The process starts in state 0, makes transitions into
states 1, 2 ..... n, and comes back to state 0. Suppose
that Z ( 0 ) = 0. Then, from straightforward renewal-

O.~o~,'j~+'i'J~(t)

+ Goo(t)*Poa(t),

) ' - 2 , 3 . . . . . n,

Poo(t) = 1 - Qol(t) + Goo(t)*Poo(t),

(12)
(13)

where ~o,,+1
/'1{1,2 . . . . . n)lf
u~~ = Q~o~2...... ~(t) and the asterisk
denotes the pairwise Stieltjes convolution. Taking the
Laplace-Stieltjes transforms on both sides of (8)-(13)
yields:
tool(S) = qol(S)/[1 - Yoo(S)],

(14)

mo j(S) = q~o} "2'''' i - 1)(s)/[1 _ (]oo(S)],

j = 2,3 . . . . ,n,

(15)

moo(S) = 9oo(S)/[l - Ooo(S)],

(16)

pol(S) - [qol(S) - @~(s)]/[l - 0oo(S)],

(17)

poj{S) = [q~o~' 2 , ' j

l~(s) -- q~o',j2"i"J'(s)]/

[1 - goo(S)],

j = 2,3 . . . . . n ,

Poo(S) = [1 - qol(S)]/[1 - goo(S)],

(18)

(19)

where, throughout this paper, we denote the LaplaceStieltjes transform of the function by the corresponding
lower case letter, e.g.,
moo(S) =

3, T Y P E

(9)

The transition probabilities are

t dGji(t )

is finite, and that there exist both the limits

Pj -= lim Pu(t)

(8)

foXe

~tdMoo(t).

It is evident that
~ poj(S} = 1.
j=O

F r o m (14)-(19), the renewal functions and the transition

probabilities are obtained explicitly upon inversion.

Markov Renewal Processes

However, it is not easy except in the simplest
cases.
Example 1. Consider an (n + 1)-unit parallel redundant system. If at least one of the n + 1 units is operating, then the system is operating. If all the units
are down simultaneously, then the system fails and
will begin to operate again immediately by replacing
all the failed units by the new ones. All units operate
independently and each unit has an identical failure
time distribution F(t) which is non-lattice and has a
finite mean 1/2. We denote the states by the total
number of the failed units. Suppose that all the units
begin to operate at time 0. Then the mass functions
are
Qol(t) -- 1 - [F(t)] n+l,
n+l

Q~,2 ..... ~-l)(t ) = ~

(.+1)[F(t)]k[F(t)].+,-k,

k-j

where if(t) - 1 - F(t). Thus, substituting the above

equations into (14)-(19), we can obtain the renewal
functions and the transition probabilities.

4. TYPE 2-M.ILP.

Consider an M.R.P. consisting of S + = {0} and

S* = {1, 2 ..... n} (see Fig. 2). The process starts in
state 0 and is permitted only to make a transition
into one state j ( j s S * ) . Then the process returns to
state 0, and so on. For instance, the process repeats
among 0, Jl, 0, J2 . . . . . where Jl, J2 . . . . E S*. The
Laplace-Stieltjes transforms of the first-passage time
distributions, the renewal functions, and the transition
probabilities are

['r

Fig. 2. The state-transition diagram for the Type 2-M.R.P.

and Osaki [4]. Assume that the failure time distribution for an operating unit is F(t) and the repair time
distribution for a failed unit is G(t). If an operating
unit fails and the other unit is in standby, the failed
unit undergoes repair immediately and the unit in
standby takes over the operation. However, if an
operating unit fails while the other unit is under
repair, the failed unit must wait for repair until a
repairman is free (i.e. we assume a single repair facility). For more detailed assumptions, refer to Nakagawa and Osaki [4].
Define the states 0 Cone unit is operating and the
other unit is under repair), 1 (one unit is operating
and the other unit is in standby), and 2 (one unit
is under repair and the other unit waits for repair).
Then, the system generates an M.R.P. with the state
space S = {0, 1, 2}, where S = {0} and S* = {1, 2}.
Then, the mass functions are

Qol(t) =

goo(S) = ~ qg~o(s),

635

F(t) dG(t),

Qo2(t) =

~ t ) dF(t),

(20)

i=O

9o ;(S) = qoj( S)/[1 - 9oo(S)],

j=l,2

Q~ld(t) =

f0'

G(t) dF(t),

Q~2d(t)=

fo'

F(t) dG(t).

(22)

moo(S) = 9oo(S)/[ l - goo(S)],

moj(S) = qoj(S)/[1 - #oo(S) ],

. . . . . n,
(21)

j=l,2

. . . . . n,

Thus, we can obtain the interesting quantities of the

system by using the results for type 2-M.R.P.'s.

(23)
5. CONCLUDING REMARKS

Poo(S) = [ i -

~ qoi(t)]/[1 -goo(S)],

(24)

i=1

Poj(S) = [qoj(S) - q~)o(S)]/[1 - goo(S)],

j = 1, 2 . . . . . n.

(25)

If n = 1 then the process corresponds to a special

one of Model 1. That is, it is the simplest state space
with a single non-regeneration point. The process
takes two alternate states 0 and 1. If the epoch entering state 1 is also a regeneration point, then the process becomes an alternating renewal process studied
by Cox [3].
Example 2. Consider a two-unit standby redundant
system of two identical units discussed by Nakagawa

We considered the unique modifications of the

regeneration point techniques in the M.R.P.'s with
some non-regeneration points. That is, the LaplaceStieltjes transforms of the first-passage time distributions, the renewal functions, and the transition probabilities are given in terms of the mass functions for
such M.R.P.'s.
We considered two applications of the processes
to redundant systems. In a similar fashion, many
redundant models can be also described by Type
1-M.R.P.'s or Type 2-M.R.P.'s, or mixtures and linkage of Type-l, Type-2, and the usual M.R.P.'s. For
instance, some two-unit standby redundant systems
with repair [5, 6] were discussed by combining Type

636

TOSHIO NAKAGAWAand SHUNJI OSAKI

Q(oaJ(t) =
Q~L3)(t) =
lira

Qo4(t) =

[A(u)* B(u)] dF(u),

f fo
;o'

[ f ( u ) - F(v)] d A ( v ) d B ( u - v),

A(uld f(u),

Q,o(tt = B(tt.
Fig. 3. The state-transition diagram for the spare part inventory model.

In particular, Laplace-Stieltjes transforms of the

transition probabilities are

Poo(S) = [1 - q o l ( s ) - qo4(s)]/[1 -hoo(S)],

1-M.R.P. and the usual M.R.P. Note that if either
the repair time distribution or the failure time distribution is not exponential, such systems have nonregeneration points.
Finally, we consider a model of spare parts inventory discussed by Wiggins [11]. A unit begins to operate at time 0. If the operating unit fails before time
T, the spare part is ordered immediately and is placed
into service as soon as it is delivered after a lead
time L. If the operating unit does not fail up to time
T, the spare part is ordered at time T a n d is delivered
after the lead time. In this case, the spare part is put
into service if the original unit has failed, or is put
into the spare part inventory if the original unit has
not failed.
Define the states 0 (the unit is operating and the
spare part is not delivered), 1 (the unit is operating
and the spare part is ordered), 2 (the unit is operating
and the spare part is in inventory), 3 (the operating
unit has failed during the order of the spare part),
and 4 (the spare part is ordered because the operating
unit has failed before T).
Assume that the failure time distribution for each
unit is F(t). Further assume that Pr{T_< t} = A(t) and
Pr{L _< t} = B(t), which might be the degenerate distributions placing unit masses at to and lo, respectively. Then, the mass functions of the above process
(see Fig. 3) are

Qox(t) =

F(u) dA(u),

O~o'l(t)= f~ F(u) d[A(u)* B(u)],

Q~oL2)(t) =

[A(u)*B(u)] dF(u),

pods) = [qo,(S)- q ~ I s ) - q~ol~(s)]/[1 -boo(S)],

po2(S) = [qCo~(S)- q(oLZ~(s)]/[l -- boo(S)],
Poa(S) = [qCo13)(s)- @d3)(s)]/[1 - hoo(S)],

po,(s) = qo,(S)[1 - q,,o(s)]/[l - hods)],

where

hoo(S) = q(old21(s)+ q~o~3~(s)+ qo4(s)q,o(s),

which is the recurrence time distribution for state 0.
In a similar way, we can obtain the first-passage time
distributions and the renewal functions.
We believe that the unique modifications of the
regeneration point techniques discussed in this paper
can be applied in other contexts, e.g. inventory theory
and queueing theory.

REFERENCES

l. R. E. Barlow and F. Proschan, Mathematical Theory

of Reliability. Wiley, New York (1965).
2. R, E. Barlow and F. Proschan, Statistical Theory q(
Reliability and Life Testing. Holt, Rinehart and Winston, New York (1975).
3. D. R. Cox, Renewal Theory. Methuen, London (1962).
4. T. Nakagawa and S. Osaki, INFOR 12, 66 (1974).
5. T. Nakagawa and S. Osaki, IEEE Trans. Reliab. R-23,
86 (1974).
6. T. Nakagawa and S. Osaki, IEEE Trans. Reliab. R-24,
143 (1975).
7. S, Osaki, J. Operations Res. Sot.. Japan 12, 127 (1970).
8. R. Pyke, Ann. math. Stat. 32, 1231 (1961).
9. R. Pyke, Ann. math. Stat. 32, t243 (1961).
10. L. Takfics, Introduction to the Theory o! Queues.
Oxford University Press, New York (1962).
11. A. D. Wiggins, Technometrics 9, 661 (1967).