Single Linear Hypothesis

Multiple Linear Hypothesis

Confidence Intervals and Regions

Hypothesis Testing and Confidence Intervals under

Normality
Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009

December 11, 2008

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Introduction
Classical linear model:
1

Linearity: Y = X + u.

Strict exogeneity: E(u|X) = 0

No Multicollinearity: (X) = K.

No heteroskedasticity/ serial correlation: V (u|X) = 2 X.

0 X)1 X 0 Y
OLS Estimator: = (XP
2
2
Estimator of : S = 2i=1 e2i /(n K).

Now we are interested in testing hypothesis about the unknown

vector of coefficients , or in constructing confidence intervals
We do not have enough information to perform this task!
Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Why? Consider the following example.

We are interested in testing H0 : j = 0 (significance).
j is not observed. But we can produce an estimator j
Classical approach: if H0 is correct, and j is good estimator
of j , then j ' 0. So we should reject H0 is j is sufficiently
different from 0.
What do we mean by j being sufficiently different from 0 ?
j is by construction a random variable, so it has to be a
probabilistic statement.
We need the distribution of j .
Can we get it based on the model as it is so far?

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Assumption 5: normality. u|X is normally distributed (it is a

vector, so this involves the multivariate normal).
Note that this together with the classical assumptions imply
u|X N (0, 2 In )
Remember that = + (X 0 X)1 X 0 u.Then
| X N , 2 (X 0 X)1

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Hypothesis about single coefficients

H0 : j = j0 vs. HA : j 6= j0
Let ais denote de (i, s) element of (X 0 X)1 .
Then, when H0 is true j j0 N (0, 2 ajj ) so
j j0
N (0, 1)
zj p
2 ajj
Note that 2 ajj = V (j ).
Special case: j0 = 0 significance hypothesis.
The distribution of zk does not depend on X.
If 2 is observed, reject if zj lies outside an acceptance region.

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

The problem is that 2 is not observed. Define:

j j0
tj p
S 2 ajj
which is zk with 2 replaced by its unbiased estimator S 2 .

Result: Under assumptions 1 to 5 and when H0 holds,

tj t(n K).

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Proof: First, two preliminary results:

If X N (0, 1) and Y 2 (m) and independent, then
X
p
tm
Y /m
If X N (0, Im ), then X 0 AX 2 ((A)).
Note that:
j j0
zj
zj
tj = p
=p
=q
e0 e/(nK)
S 2 ajj
S 2 / 2
2

We have already shown zj N (0, 1) so if we can show

e0 e/ 2 2 (n K) and independence, we are done.

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

2 numerator: Recall e = M u, so e0 e = u0 M M 0 u = u0 M u
since M is idempotent. Also, we have assumed
u|X N (0, 2 In ), then by our previous result
u0 M u/ 2 2 (n K) since (M ) = n K.
Independence: = + (X 0 X)1 X 0 u and e = M u are linear
functions of u, they are jointly normal given X. Now
e) = E[( )e0 ]
Cov(,
= E[(X 0 X)1 X 0 uu0 M ]
= (X 0 X)1 X 0 E(uu|X)M
= 2 (X 0 X)1 X 0 M = 0
using LIE and the spherical disturbances assumption.

Since zj depends solely on and the numerator only on e, then

the result follows.
Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Hypothesis about linear combinations of .

H0 : c0 r = 0 vs. HA : c0 r 6= 0, c <K , r <.

WLOG, supose K = 3 so,
Yi = 1 X1i + 2 X2i + 3 X3i + ui

i = 1, . . . , n

Consider the following hypotheses:

a) H0 : 2 = 3 , or H0 : 2 3 = 0. In this case
c = (0, 1, 1) and r = 0.
b) H0 : 2 + 3 = 1, so now c = (0, 1, 1) and r = 1.

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

To derive an appropiate test statistic note:

c0 r N (0, 2 c0 (X 0 X)c) N (0, 1)
So

c0 r
N (0, 1)
z=p
2 c0 (X 0 X)c)

And again, by the same argument as before, a feasible version is

c0 r
t= p
t(n K)
S 2 c0 (X 0 X)1 c)

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

As a simple exercise, the appropriate statistics for the cases

considered before are
a) c0 r = 2 3 and
d 1 , 2 ), so
2 c0 (X 0 X)1 c = V (2 ) + V (2 ) 2Cov(
t=

2 3
d 1 , 2 )
V (2 ) + V (2 ) 2Cov(

b) c0 r = 2 + 3 1, and
d 1 , 2 ), so
2 c0 (X 0 X)1 c = V (2 ) + V (2 ) + 2Cov(
t=

2 + 3 1
d 1 , 2 )
V (2 ) + V (2 ) + 2Cov(

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Multiple Linear Hypothesis

H0 : R r = 0, R is a q K matrix with (R) = q, and r <q
Example. In our previous case consider the multiple hypothesis
H0 : 2 = 0 : 3 = 0
These are actually two joint hypothesis about the coefficient vector
. In this case


 
0 1 0
0
R=
r=
0 0 1
0
with q = 2. r is the number of restrictions.
What is the full row rankrequirement, (R) = q, asking for?
Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Consider the following test statistic:


1
(R r)0 / q
(R r)0 R(X 0 X)1 R0
F =
S2
Result: under all assumptions and when H0 is true,
F F (q, n K).
= 2 R(X 0 X)1 R0 . Then,
Intuition: Note that V (R)
2
0
1
= S R(X X) R0 , so
V (R)
1

F = (R r)0 V (R|X)
(R r)0 / q

F is actually checking how large R r is.

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Proof: Again, let us start with two results

If X 2 (m) and Y 2 (n) and independent, then
(X/m)
F (m, n)
X/n
If X is an m random vector X N (0, ) with invertible,
then X 0 1 X 2 (m)
Reexpress the F statistic as follows:

1
(R r)0 R(X 0 X)1 R0
(R r) / q
W/r
=
F =
2
S
Q/(n K)

1
with W (R r)0 2 R(X 0 X)1 R0
(R r) and
0
2
Q = e e/ .
Now it is obvious what we have to do...
Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

We already know e0 e/ 2 2 (n K).

Now, under H0 : R r = 0 and our assumptions
R r | X N (0, 2 R(X 0 X)1 R0 )
It is easy to check that R(X 0 X)1 R0 is invertible (why? do it!),
hence
W | X 2 (r)
Now the desired result follows since W is a function of and Q a
function of e, which are independent.

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

An Alternative Formulation: Restricted Least Squares

Consider H0 : R r = 0, and the following estimator that arises
from the optimization problem
s.t. RR = r
R = argmin SSR()

R is the restricted least squares estimator. Lets adopt the

following notation
eR = Y X 0 R , SSRR = e0R eR
SSRU = e0 e

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Result:
F =

(SSRR SSRU ) / q
SSRU /(n K)

Intuition
SSRR SSRU 0. Why?
(SSRR SSRU ) is the loss in explanatory power by imposing
H0 as a restriction.
When H0 is true, the restriction should not matter so the loss
should be small.
Proof: see homework

Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis
Confidence Intervals and Regions

Confidence Intervals and Regions

Let t/2 be the 1 /2 quantile of a Students t RV with (n K)
df. By construction

j j
< t/2 = 1
P r t/2 < q

V (j )


q
q
P r j t/2 V (j ) < j < j + t/2 V (j ) = 1
Then
j t/2

V (j )

is an 1 confidence interval for j . Note that it contains all the

values of j for which H0 : j = 0 is accepted.
Walter Sosa-Escudero

Hypothesis Testing and Confidence Intervals under Normality