MTH6141 Random Processes, Spring 2012, Exercise Sheet 5

Please drop your answers to Questions 1 and 2 in the red box on the second oor of the maths building by 13:00 on Tuesday 28th February 2012. You are strongly encouraged to attempt all questions. Please send comments and corrections to m.jerrum@qmul.ac.uk. 1. Consider the Markov chain on state trix 0 2 3 1 space S = {1, 2, 3} with transition ma 1 0 0 1 3 0 0

For each state i S calculate directly the distribution of Ri (the time of rst return to i) and E(Ri ). Use these values, together with Theorem 1.15, to write down the equilibrium distribution w = (w1 , w2 , w3 ) for the chain. Check your answer directly against the denition of equilibrium distribution. 2. Suppose that a and b are states in the same communicating class of a Markov chain. Prove that there exist constants u N and > 0 such that
(t+u) paa pbb , (t)

for all t N.

Hence deduce that null recurrence is a class property (that is, if a is a null recurrent state then b is also null recurrent). 3. Consider the Markov chain on state space S = {0, 1, 2, . . . } with transition probabilities i+1 i+2 , if j = i + 1; 1 pi,j = i+2 , if j = 0; 0, otherwise. Calculate f00 and f00 for this chain and deduce from this that the chain is recurrent. Calculate E(R0 ) and decide whether the chain is positive recurrent or null recurrent.
(t)

4. In lectures we considered an asymmetric random walk on the integers Z with and 1 ) for moving up and down. More generally, we specic probabilities ( 2 3 3 could consider the walk with transition probabilities if j = i + 1; p, pij = 1 p, if j = i 1; and 0, otherwise.
1 Show that state 0 is transient whenever 0 < p < 1 and p = 2 . 2 Hint. Repeat the analysis used in the lectures for the special case p = 3 , and then nish with a little calculus.

5 . Consider the random walk on N with a reecting barrier at 0, i.e., the Markov chain with transition probabilities 1 2 , if i = 0 and |j i| = 1; pij = 1, if i = 0 and j = 1; and 0, otherwise. Prove that state 0 is recurrent. Is state 999 recurrent? Hint. Try comparing the above random walk with the symmetric walk on Z that we analysed in lectures.

Other activities for reading week Try some of the exercises on the earlier sheets that you missed out rst time. Revise the properties of the Poisson and exponential distributions from MTH 5121 Probability Models.