THE CHINESE UNIVERSITY OF HONG KONG

Department of Statistics
STAT3007: Introduction to Stochastic Processes
Problem Sheet 1
The deadline for this Problem Sheet is 5.30pm on Thursday 6th October.
Please hand in your answers to Dong Fangyuan in LSB 126 or Liu Zhengyun
in G26 or alternatively to box 14. No late submissions will be accepted.
A late submission will receive a mark of zero. Students may discuss set
problems with others, but their final submissions must be their own work. Do
show your working - it helps us to give you marks.
Please answer the following problems. (All are based on questions that can
be found in An Introduction to Stochastic Modeling, 4th Ed. by Mark Pinsky
and Samuel Karlin, unless otherwise stated.)
1. (Problem 2.4.1 in Pinsky and Karlin) Suppose that the outcome X of a
certain chance mechanism depends on a parameter p according to Pr{X =
1} = p and Pr{X = 0} = 1 p, where 0 p 1. Suppose that p is chosen
at random, uniformly distributed over the unit interval [0, 1], and then,
that two independent outcomes X1 and X2 are observed. What is the
unconditional correlation coefficient between X1 and X2 ?
2. (Problem 2.4.4 in Pinsky and Karlin) Suppose X and Y are independent
random variables having the same Poisson distribution with parameter ,
but where is also random, being exponentially distributed with parameter . What is the conditional distribution for X given that X + Y = n?
3. (Problem 2.3.4 in Pinsky and Karlin) Suppose 1 , 2 , . . . are independent
and identically distributed random variables having mean and variance
2 . Form the random sum SN = 1 + + N .
(a) Derive the mean and variance of SN when N has a Poisson distribution with parameter .
(b) Determine the mean and variance of SN when N has a geometric
distribution with mean = (1 p)/p.
(c) Compare the behaviours in the above two questions as .
4. (Problem 2.4.6 in Pinsky and Karlin) :et X0 , X1 , X2 , . . . be independent
identically distributed nonnegative random variables having a continuous
distribution. Let N be the first index k for which Xk > X0 . That is,
N = 1 if X1 > X0 , N = 2 if X1 X0 and X2 > X0 , etc. Determine
the probability mass function for N and the mean E[N ]. (Interpretation:
X0 , X1 , . . . are successive offers or bids on a car that you are trying to sell.
Then N is the index of the first bid that is better than the initial bid.)

5. (Problem 3.1.1 in Pinksy and Karlin) A simplified model for the spread of
a disease goes this way: The total population size is N = 5, of which some
are diseased and the remainder are healthy. During any single period of
time, two people are selected are random from the population and assumed
to interact. The selection is such that an encounter between any pair of
individuals in the population is just as likely as between any other pair. If
one of these persons is diseased and the other not, with probability = 0.1
the disease is transmitted to the healthy person. Otherwise, no disease
transmission takes place. Let Xn denote the number of diseased persons
in the population at the end of the nth period. Specify the transition
probability matrix.
6. (Problem 3.1.4 in Pinsky and Karlin) The random variables 1 , 2 , . . .
are independent and with the common probability mass function P r( =
0) = 0.1,P r( = 1) = 0.3, P r( = 2) = 0.2 and P r( = 3) = 0.4. Set
X0 = 0, and let Xn = max{1 , . . . , n } be the largest observed to date.
Determine the transition probability matrix for the Markov chain {Xn }.
7. (Problem 3.2.1 in Pinsky and Karlin) Consider the Markov chain whose
transition probability matrix is given by

0.4 0.3 0.2 0.1

0.1 0.4 0.3 0.2

P=
0.3 0.2 0.1 0.4
0.2 0.1 0.4 0.3
Suppose that the initial distribution is pi = 0.25 for i = 0, 1, 2, 3. Show
that P r(Xn = k) = 0.25, k = 0, 1, 2, 3, for all n. Can you deduce a general
result from this example?)