Stat 245 Homework 3 Solution

Winter 2011

Note to graders: Please weight each of the 5 problems out of 20 points for a total of 100 points.

1. (t-test as likelihood ratio test)

The case when σ 2 is known is done in the book, and it is not equivalent to a t-test but to a z -test. We are
interested in the case when both µ and σ 2 are unknown. For that we use the generalized likelihood ratio
test (see page 339 (3rd ed.) or page 308 (2nd ed.) in Rice; the test statistic we use here is the reciprocal of
the one used in Rice, so our test rejects H0 when the statistic is large).
The (generalized) likelihood ratio statistic is given by:

max(µ,σ2 )∈ω1 [lik(µ, σ 2 )]

Λ= ,
max(µ,σ2 )∈ω0 [lik(µ, σ 2 )]

where ω1 = {(µ, σ 2 ) : µ ∈ R, σ 2 > 0} and ω0 = {(µ, σ 2 ) : µ = µ0 , σ 2 > 0}. By definition, the maximum in
the numerator is achieved when the MLEs are substituted for µ and σ 2 .
To obtain the denominator, we need to maximize the function
 Pn 2

1 1 i=1 (Xi − µ0 )
f (σ 2 ) = lik(µ0 , σ 2 ) = exp − .
(2π)n/2 σ n 2σ 2

Since logarithms are increasing functions, it is equivalent to maximize the log-likelihood:

n
1 X
log lik(σ) = C − n log(σ) − (Xi − µ0 )2 ,
2σ 2 i=1
where C is a constant. Take the derivative with respect to σ, set it equal to 0, and verify that the derivative
is positive to the left and negative to the right of the unique solution, to conclude that f is maximized at
n
1X
σ2 = (Xi − µ0 )2 .
n i=1

Therefore,
 n/2
 Pn 2

1 n i=1 (Xi − X̄) /2
Pn exp − Pn
(2π)n/2 i=1 (Xi − X̄)
2 2
i=1 (Xi − X̄) /n
Λ= n/2  Pn 2
 
1 n i=1 (Xi − µ0 ) /2
P n 2
exp − Pn 2
(2π)n/2 i=1 (Xi − µ0 ) i=1 (Xi − µ0 ) /n
 Pn n/2
(Xi − µ0 )2
= Pi=1n 2
.
i=1 (Xi − X̄)

This test rejects H0 if Λ is large; that is, if Λ > c for a cutoff point c ≤ 0 that satisfies

Prob[Λ > c|H0 ] ≤ α

where α is the specified significance level. We have to show that this test is equivalent to a t-test. Indeed,

1
  n
X

2
 (Xi − µ0 ) 
 i=1
 > c2/n

Λ > c ⇐⇒  n


X
(Xi − X̄)2
 
i=1
 n
X

2
 (Xi − X̄ + X̄ − µ0 ) 
 i=1
 > c2/n

⇐⇒ 

n 
X
(Xi − X̄)2
 
i=1
 n
X n
X

2 2
 (Xi − X̄) + (X̄ − µ0 ) 
 i=1 i=1  > c2/n

⇐⇒ 
 n 
X
(Xi − X̄)2
 
i=1
 

n(X̄ − µ0 )2 
 
 > c2/n

⇐⇒ 
1 + n 
X
(Xi − X̄)2
 
i=1
 
 
  r
 
 X̄ − µ0  > n − 1 (c2/n − 1)

⇐⇒  P
 s
n

2  n
i=1 (Xi − X̄) 


 n−1 

q
Taking c0 = n−1 2/n
n (c − 1) to be equal to tn−1,α/2 we have a two-sided t-test.

2. (Confidence interval for variance)

(a) We know that (n − 1)s2 /σ 2 has the chi-square distribution with n − 1 degrees of freedom. Let χ2n−1,α/2
and χ2n−1,1−α/2 be the α/2 and 1 − α/2 quantiles for the chi-square distribution with n − 1 degrees of
freedom. Recall that, in our notation, χ2m,α is the positive number that satisfies

Prob[X > χ2m,α ] = α for X ∼ χ2m .

Then
Prob[χ2n−1,1−α/2 < (n − 1)s2 /σ 2 < χ2n−1,α/2 ] = 1 − α,
so
" #
(n − 1)s2 (n − 1)s2
Prob 2 < σ2 < 2 = 1 − α.
χn−1,α/2 χn−1,1−α/2

(n − 1)s2 (n − 1)s2
Therefore, Lσ2 (α) = 2 and Uσ2 (α) = 2 determine a (1 − α)100% confidence interval
χn−1,α/2 χn−1,1−α/2
for σ 2 . And, clearly,
Prob[Lσ2 (α) > σ 2 ] = α/2 = Prob[Uσ2 (α) < σ 2 ].

2
(b) Since X̄ and s2 are independent, it is tempting to think that

?
Prob µ ∈ [Lµ (α), Uµ (α)], σ 2 ∈ [L2σ (α), Uσ2 (α)] = (1 − α)2 .

NO!! The random variables here are the interval endpoints, not µ and σ 2 , and since s2 appears in both
pairs of endpoints, there is no reason to think that Lµ (α), Uµ (α) and (L2σ (α), Uσ2 (α)) are independent.

The event that “µ ∈ [Lµ (α), Uµ (α)] and σ 2 ∈ [L2σ (α), Uσ2 (α)]” can be represented as a region in the
X̄ − s plane bounded by the following four lines:

q
L1 : s = σ 2 χ2n−1,α/2 /(n − 1)
q
L2 : s = σ 2 χ2n−1,1−α/2 /(n − 1)
tn−1,α/2
L3 : X̄ = µ + √ s
n
tn−1,α/2
L4 : X̄ = µ − √ s
n

This region is sketched in Figure 1. One way to obtain the exact probability of the event is to compute
the double integral of the joint density of X̄ and s (which is the product of the corresponding marginal
densities, using now the fact that X̄ and s are independent) over that region.

Figure 1: Sketch of the region on the X̄ − s plane that corresponds to the event in question 2(b)

At this point, though, a reasonable bound on probability of the event is enough. We will do that by
applying the Bonferroni inequality:

Prob(µ ∈ [Lµ (α),Uµ (α)], σ 2 ∈ [L2σ (α), Uσ2 (α)])

/ [Lµ (α), Uµ (α)] or σ 2 ∈
= 1 − (Prob(µ ∈ / [L2σ (α), Uσ2 (α)]))
/ [Lµ (α), Uµ (α)]) − Prob(σ 2 ∈
≥ 1 − Prob(µ ∈ / [L2σ (α), Uσ2 (α)])
≥ 1 − 2α.

(c) To obtain different CI’s, choose 0 < a < b such that

Prob[a < (n − 1)s2 /σ 2 < b] = 1 − α, (*)

3
since then, as above,
(n − 1)s2 (n − 1)s2
 
,
b a
1 1


is a 100(1 − α)% CI for σ 2 . The length of this interval is (n − 1)s2 a − b , so we need to find the
choice of a and b that minimizes a1 − 1b , subject to the constraint (*).

To do that, notice that the constraint (*) determines b as a function of a, so we can use implicit
differentiation:
Z c
d d
fn−1 (x) dx = (1 − α),
da a da

where fn−1 is the density function for the chi square distribution with n−1 degrees of freedom. Picking
an arbitrary c > 0, we obtain:
Z c Z c
d d
fn−1 (x) dx + fn−1 (x) dx = 0.
da a da b

By the Fundamental Theorem of Calculus, this means

db
−fn−1 (a) + fn−1 (b) = 0,
da
so
db fn−1 (a)
= .
da fn−1 (b)

Now:
 
d 1 1 1 1 fn−1 (a)
− =− 2
+ 2 ,
da a b a b fn−1 (b)

and this equals zero when

a2 fn−1 (a) = b2 fn−1 (b) (**)

To verify that we have actually found a minimum, a second implicit differentiation yields that
0
d2 b fn−1 (a) 0 fn−1 (a)
2
= − fn−1 (b) .
da fn−1 (b) fn−1 (b)

For the usual values of α, we will have that a must be small enough and b must be large enough so
d2 b
that fn−1 is increasing at a and decreasing at b, guaranteeing that da 2 > 0.

Consequently, an interval of minimum length will be found for any values of a, b that satisfy (*) and
(**). It does not seem possible to solve for a, b explicitly, but it is easy to find numerically the value
of ξ ∈ [0, α) such that makes the expression

(χ2n−1,ξ )2 fn−1 (χ2n−1,ξ ) − (χ2n−1,1−α+ξ )2 fn−1 (χ2n−1,1−α+ξ )

equal to zero. Then we can take a = χ2n−1,ξ and b = χ2n−1,1−α+ξ .

4
3. (Confidence intervals: pooled versus paired procedure)
(a) Proof: Suppose ρ = Corr[Xi , Yi ] = 1 for i = 1, . . . , n. Then, there exist ai , bi , i = 1, . . . , n such that
for each i,
Prob[Xi = ai + bi Yi ] = 1
(by Theorem B, p. 143 [3rd ed.] or p. 133 [2nd ed.] in Rice).

Now, for each i = 1, . . . , n σ 2 = Var(Xi ) = b2i Var(Yi ) = b2i σ 2 , so |bi | = 1. Furthermore, since the
correlation is positive, eachbi must be positive. Consequently, all the bi ’s equal 1, and from that we
can deduce that for each i

ai = E[ai ] = E[Xi − Yi ] = E[Xi ] − E[Yi ] = µ1 − µ2 .

Now, for all i,

Di = Xi − Yi = (µ1 − µ2 ) + Yi − Yi = µ1 − µ2 .

All the Di ’s being the same, we conclude that sD̄ = 0, and therefore the confidence interval
s
D̄ ± tn−1,α/2 √D̄
n

has length 0 around the point D̄ = µ1 − µ2 .

sD̄
(b) The length of the “paired interval” is 2tn−1,α/2 √ n
, and the length of the “pooled interval” is 2t2n−2,α/2 sX̂−Ŷ ,
where
1
s2X̂−Ŷ = (s2X + s2Y ).
n
We have that
n
X
(n − 1)s2D̄ = (Xi − Yi − D̄)2
i=1
n
X
= [Xi − X̄ − (Yi − Ȳ )]2
i=1
n
X n
X n
X
= (Xi − X̄)2 + (Yi − Ȳ )2 + (Xi − X̄)(Yi − Ȳ )
i=1 i=1 i=1
= (n − 1)s2X + (n − 1)s2Y ,

since the sample covariance equals 0. That is, s2D̄ = s2X + s2Y . Consequently, the ratio of the length of
the “paired interval” over that of the “pooled interval” is

tn−1 (α/2)
.
t2n−2 (α/2)

Since 2n − 2 ≥ n − 1, for n ≥ 1, we have that tn−1 (α/2) ≥ t2n−2 (α/2). Therefore, the “paired interval”
is longer.

5
4. (Data analysis: Two samples I)
(a) Here is the R code:
d<-read.table("calcium-c11.txt")
n<-118 # number of observations
x<-log(d[,1])
y<-log(d[,2])

# Pooled procedure
xbar<-mean(x)
ybar<-mean(y)
ssqpool <- (n-1)/(2*n-2)*(var(x)+var(y))
tpool <- (xbar - ybar)/sqrt(ssqpool*(2/n)) # pooled t-statistic
2*pt(-abs(tpool),2*n-2) # p-value

The t statistic equals 0.1817, and the corresponding p-value is 0.856. If this procedure were justied,
then the evidence against the null would be extremely weak.

(b) Here is the R code:

# Paired procedure
D <- x-y
Dbar <- mean(D)
ssqD <- var(D)
tpair <- Dbar/sqrt(ssqD/n) # paired t-statistic
2*pt(-abs(tpair),n-1) # p-value

The t-statistic in this case equals 4.0362, and the p-value is 9.73 ?10−5 . If this procedure is justied,
then the evidence is very strong against the null.

(c) The two variables are strongly correlated, as can be seen from the scatterplot (Figure 2) and the sample
correlation coe?cient of 0.998. To compute a 95% condence interval for the true correlation coe?cient
ρ, we use the Fisher Z transformation. Take
 
1 1+r
Z = log ,
2 1−r

where r represents possible values of the sample correlation coefficient. Then Z is approximately
1
normally distributed, regardless of the true value of ρ, with standard deviation √n−3 . For r∗ = 0.998 we
obtain Z ∗ = 3.453. The margin of error in the Z scale is given by 1.96 √115
1
= 0.1828, so the endpoints
of a 95% CI for the mean of Z are 3.27 and 3.64. Applying the inverse Fisher Z transformation
2Z
r = ee2Z −1
+1
, we obtain the endpoints of a 95% CI for ρ:

(0.997, 0.998).

6
 Figure 2: Scatterplot of “Flame method vs. “Oxalate method measurements (both log transformed).

(d) Clearly the paired procedure is more appropriate here given the strong correlation between the vari-
ables. The textbook suggests that there were 118 samples of feeds, and that each sample was measured
with both methods; so, the design of the study also suggests pairing the observations.

5. (Data analysis: Two samples II)

The experimental setup precludes the use of a paired procedure, since there is no natural way of pairing rats
in one group with rats in the other (the groups even have different numbers of individuals). Also, since the
standard deviation of the treated group (19.02) is almost twice as large as that of the control group (10.78),
a pooled procedure might also be questionable. We will use the R command t.test twice, once specifying
equal variances and once not. The difference is in the way the degrees of freedom are computed (using, or
not, the so-called Welch or Statterthwaite approximation).
Since there is no reason a priori to believe that the rats will gain (or lose) weight under the inuence of ozone,
we will select a two-sided alternative hypothesis.
Here are the R commands and the output:
> ozone <- scan("ratozone-c11.txt")
> control <- scan("ratcontrols-c11.txt")
> t.test(ozone,control) # All the defaults work in our case
# See ?t.test for more details

Welch Two Sample t-test

data: ozone and control
t = -2.4629, df = 32.918, p-value = 0.01918
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-20.848949 -1.985043
sample estimates:
mean of x mean of y
11.00909 22.42609

There is strong evidence against the null hypothesis of no effect; the effect is negative, as can be observed
by the sign of the t-value. The condence interval is not very tight, but conrms that breathing ozone makes
the rats lose weight.

7
Now we perform the same test assuming equal variances:
> t.test(ozone,control,var.equal=T)

Two Sample t-test

data: ozone and control
t = -2.4919, df = 43, p-value = 0.01664
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-20.656806 -2.177186
sample estimates:
mean of x mean of y
11.00909 22.42609

Notice that the t-statistics are very close (this is due to the fact that the sample sizes are similar; if they
were equal, then the t-statistic would not change at all), but the number of degrees of freedom is different.
This indicates that when the sizes of the two samples are similar then the pooled procedure may perform
reasonably well even if the variances are fairly different. With roughly equal sample sizes the only real
difference between the pooled and the Satterthwaite procedure is in the degrees of freedom which are chosen
more conservatively in the Satterthwaite procedure. Of course, if the sample size is large, then the t-
distribution starts to look like a normal N (0, 1) and little changes in degrees of freedom have no large effect.

A quick look a the histograms for each group reveals no strong failures of the assumption of normality, so
the inferences reached seem reasonably solid. See Figure 3 for boxplots of the two groups, obtained with the
command boxplot(control,ozone,names=c("Control","Ozone")).

Figure 3: Boxplot of weight increase for the two groups: control and treated with ozone (corresponds to
Problem 5).