18.

466 - Spring 2013 - Homework 1

Exercise 1
We have
P ois(x|) = e

x
x!

e
()
P ois(x|)(; , )
(|x; , ) = P
i=0 P ois(x|)(; , )
(; , ) =

We can simplify the expression of the posterior as follows:

(|x; , ) (e x )(1 e ) =

(1)

= e(+1) x+1

(2)

Where in (1) we dropped terms that do not depend on lambda. We recognize

in (2) a gamma distribution up to its normalization constant, therefore we
write
0

0 0 1 0

e
(0 )
0 = + x

(|x; 0 , 0 ) =

0 = + 1

Exercise 2
a) We first derive the expectation of the variance MLE. That is
"
#
N
 2 
1 X
E
MLE = E
(xi x
)2 =
N
i=1
"N
#
X
1
2
2
= E
xi + x
2
xxi
N
i=1
 
 2
1
= (N E x2
N E x
) =
N
| {z }
| {z }
Var[x]+E[x]2

Var[
x]/N +E[
x ]2

1
N 1
(N Var[x] Var[x]) =
Var[x]
N
N

18.466 - Spring 2013 - Homework 1

Then we note that

"
#


N
1 X
N
2
2
E
(xi x
) = E

=
N 1
N 1 MLE
i=1

 2 
N
E
MLE =
N 1
= Var[x]
=

b)

P
i) As we have just shown, Bias( N 11 N
)2 , 2 ) = 0. We
i=1 (xi x
P
N
)2 ].
now turn our attention to Var[ N 11 i=1 (xi x
1

Var[(N 1)

N
X
(xi x
)2 ] = Var[(N 1)1 2 2N 1 ] =
i=1

= (N 1)2 4 Var[2N 1 ] =
= 2(N 1)1 4

ii) Using the bias-variance

decomposition
and the fact that, as shown
i
h P
N
2
) = c(N 1) 2 , we proceed as
in part (a), E c i=1 (xi x
follows:
MSE(c

N
X

N
X
(xi x
) ) = Var[c
(xi x
)2 ] + (c(N 1) 2 2 )2 =
2

i=1

i=1

= 2(N 1)c2 4 + (c(N 1) 2 2 )2

Imposing first order optimality conditions we have


4c(N 1) 4 + 2(N 1) 2 c(N 1) 2 2 = 0
c=

1
N +1

We also have
MSE(c

PN

i=1 (xi

x
)2 )

= 2(N 1)2 4 + 4(N 1) 4

which is clearly always positive for all values of c, from which we

conclude that c = N 1+1 minimizes the MSE. 

18.466 - Spring 2013 - Homework 1

Exercise 3
a) We assume that X Binomial(n, 0 ) From this, we obtain MSE(
p)

and MSE() as follows:




X
E [
p] = E
= 0
n
"
2 #
 
X
X
MSE(
p) = E
=
0
= Var
n
n
=

Var[X]
n0 (1 0 )
0 (1 0 )
=
=
n2
n2
n

h i
E = (0.2)(0.1) + (0.8)E [
p] = 0.02 + 0.80
h i
0 ) = E 0 = 0.2( 1 0 )
Bias(,
10
= Var[0.02 + 0.8
Var[]
p] = 0.64 Var[
p] = 0.64

0 (1 0 )
n

= Bias2 (,
0 ) + Var[]
=
MSE()
(1 0 )0
= (0.02 0.20 )2 + 0.64
n
We proceed with a comparison of the estimators.

MSE()
<1
MSE(
p)
>0
MSE(
p) MSE()
0 (1 0 )
(1 0 )0
(0.02 0.20 )2 + 0.64
>0
n
n
(n + 9)02 (0.2n + 9)0 + 0.01n < 0
This quadratic inequality in 0 indicates that when n = 25, outperforms
p if 0.018707 < 0 < 0.393058 and when n = 100, outperforms p if
0.0407128 < 0 < 0.225342

18.466 - Spring 2013 - Homework 1

Exercise 4
a) The posterior pD|X (D|X) is
pD|X (D1 |X) =

pX|D (X|D1 )pD (D1 )

=
pX|D (X|D1 )pD (D1 ) + pX|D (X|D0 )pD (D0 )
1

0.06e 2 (x7)
1

0.06e 2 (x7) + 0.94e 2 (x4)

pD|X (D0 |X) = 1 pD|X (D1 |X)

e of performing or not performing a subb) The expectation of the cost C

sequent test can be computed as follows:
i
h
e 1 , D) = 1050pD|X (D0 |X) + 50pD|X (D1 |X)
E C(T
h
i
e 0 , D) = 1000050pD|X (D1 |X) + 50pD|X (D0 |X)
E C(T
To minimize the expected loss, we combine, rearrange and simplify the
above as follows:
pX|D (X|D1 ) D1 pD (D0 ) (1000050 50)
R
pX|D (X|D0 ) D0 pD (D1 ) (1050 50)
exp(3(

D1
11
47000
X)) R
2
3
D0
D1

X R
D0

11 1
47000
+ ln(
)
2
3
3

(3)
(4)
(5)

Where in case of equality the choice between testing again and not can
be arbitrary for the expected cost is the same.
If the prior probability of having D is 105 we can plug in the different
prior in (3) to see that the first test should produce a value greater
than (or, at least, equal to) 3843.14 in order for it to be optimal
to perform a second test. Given the distribution of the first tests
statistic, even in presence of the disease, the probability of such an
event is extremely close to 0 and thus the situation is most unlikely to
arise in practice.