TESTS OF NORMALITY OF REGRESSION ERRORS Herman J.

Bierens Pennsylvania State University February 13, 2010

In this lecture note I will derive the Jarque-Bera (1980) and-Kiefer- Salmon (1983) tests of the normality of the regression errors. Some of the steps in the derivations follow from standard results, in particular Chapters 5 and 6 in Bierens (2004) and are therefore left as exercises, where the readers have to figure out for themselves which results in Bierens (2004) are applicable. Consider the linear regression with an intercept yj ' Txj % uj , xj ' (1 , x2,j , ... , xk,j) , j ' 1 ,.. ,n , where
T

(1)

ASSUMPTION 1: The vectors (uj,x2,j,...,xk,j)T are independent random drawings from a k-variate distribution, and E(uj,x2,j,...,xk,j) = 0 with probability 1.

We want to test the null hypothesis that (conditionally on the x variables), H0: uj*xj - N(0,2) . Under the null hypotheses we have: E(u j ) ' 0 , E(u j ) ' 34 ,
3 4

(2)

(3)

and it will be more feasible to test the latter implication of the null hypothesis than the null hypothesis (2) itself. If the errors uj would be observable and the variance 2 known, we could test the hypothesis (3) on the basis of the random vector

1 n
n j'1

3 j (u j / ) n j'1 4 j (uj / ) & 3

(4)

which under the null hypothesis converges in distribution to the bivariate normal distribution with zero mean vector and variance matrix E (u1 / )6 E (u1 / )7&3E (u1 / )3 ' 15 0 0 96 , (5)

E (u1 / )7&3E (u1 / )3 E (u1 / )8 & 6E (u1 / )4 % 9

where the equality involved follows from the fact that under the hypothesis (2), E (uj
2m&1 2 4 6 8

) ' ' '

0 2 34

for m ' 1 , 2 , 3,...

E (uj ) E (uj ) E (uj ) E (uj )

(6)

' 156 ' 1058

The results in (6) can be derived using the moment generating function of the standard normal distribution. (Exercise: Try it!) Thus under the null hypothesis (2) we have
3 j (uj / ) n j'1 4 j ( (uj / ) & 3 ) n j'1

1 n and consequently,

6 N2

0 0

15 0 0 96

in distr.

(7)

j (uj / )
n j'1

j ( (uj / ) & 3 )
n 4

15n

j'1

96n

6 2 in distr.

(8)

(Exercise: Why?) However, we do not observe the uj's and the variance 2 , so that this test is not feasible. The idea behind the Jarque-Bera (1980) and Kiefer-Salmon (1983) tests is to replace in (7) 2

the uj's by the least squares residuals T uj ' yj & xj ' uj & (&)Txj , where is the OLS estimator of , and 2 by the estimator 2 ' 1 2 j uj , n j'1
n

(9)

(10)

and to prove that the asymptotic normality result (7) goes through, although with a different variance matrix. Note that, due to the presence of an intercept, j uj ' 0 .
n j'1

(11)

(Exercise: Why?). Therefore, we can also write u j ' uj&u & (&)T(xj& ) , x
n n

(12)

where u ' (1/n)'j'1uj and x ' (1/n)'j'1xj . This will be convenient in proving Lemmas 2 and 3 below. Next assume:

ASSUMPTION 2: Q ' E (xjxj ) is nonsingular, and E (2xj24 ) # 4 ,

where 2x2 ' x Tx is the Euclidean norm. Then:

LEMMA 1: Under Assumptions 1-2 and the null hypothesis (2) we have plimn64 [ (1/ n)'j'1u j & (1/ n)'j'1uj ] ' 0 .
n 2 n 2

Proof: It follows from (9) that

2 2 T T 2 T T j u j ' j uj & (&) xj ' j u j & 2(&) j ujxj % (&) j xjxj (&) n n n n n j'1 j'1 n j'1 n j'1 j'1

' j uj & n(&)T

2 j'1

1 T j x jx j n j'1

(13) n(&) .

(Exercise: Prove the last step). Moreover, under the conditions of the lemma we have n(&) 6 N k(0 , 2Q &1) in distr. and by the law of large numbers, 1 T plimn64 j xjxj ' Q . n j'1
n

(14)

(15)

(Exercise: Verify the conditions of the law of large numbers). These two results imply that j uj & j u j '
n 2 n 2 j'1 j'1

n(&)T

1 T j x jx j n j'1
n

n(&) 6 k in distr.

(16)

(Exercise: Which result in Bierens (2004) has been applied?). The lemma follows now from (16) (Exercise: Why?).

LEMMA 2: Under Assumptions 1-2 and the null hypothesis (2) we have plimn64 [ (1/ n)'j'1u j & (1/ n)'j'1 (uj &32uj ) ] ' 0 .
n 3 n 3

Proof: Using (12) and the well-known equality (a&b)3 ' a 3&3a 2b%3ab 2&b 3 , we have
3 3 T T x 3 2 x j uj ' j u j&u & (&) (xj&) ' j (uj&u) & 3(&) j (uj&u ) (xj& ) n n n n j'1 j'1 j'1 j'1

x x x % 3(&)Tj (u j&u )(xj& )(xj& )T(&) & j (&)T(xj& )

n n j'1 j'1

(17)
3

Since under the conditions of the lemma, n 1 plimn64 j (u j& )2(xj& ) ' 0 u x n j'1 and n 1 plimn64 j (uj& )(xj& )(xj& )T ' O u x x n j'1 (exercise: why?), it follow that 4

(18)

(19)

plimn64

n j'1

j
n

3 uj

&

n j'1

u j uj&
n

' 0.

(20)

Next, we have 1 j (uj&u ) ' n j'1

n 3

j n j'1
n n

3 uj

1 2 & 3 nu j uj % 2 nu 3 n j'1
n

' hence plimn64

n j'1
n

2 u j uj & 3 uj & 3 n 3

1 2 2 u3 j u j & % 2 n , n j'1
n

(21)

n j'1

j uj&u

&

n j'1

2 j u j &3 u j n 3

' 0.

(22)

(Exercise: Why?). Combining (20) and (22), the lemma follows.

LEMMA 3: Under Assumptions 1-2 and the null hypothesis (2) we have plimn64 (1/ n)'j'1u j & (1/ n)'j'1uj ' 0 .
n 4 n 4

Proof: Again using (12), and the well-known equality (a-b)4 = a4-4a3b+6a2b2-4ab3+b4, we have T x j u j ' j uj&u & (&) (xj& )
n 4 n j'1 j'1 n n 4

' j (uj&u)4 & 4(&)Tj (uj&u )3(xj& ) x

j'1 j'1

% 6(&)Tj (uj&u )2(xj& )(xj& )T(&) x x

n j'1

(23)

&4j (uj&u ) &)T(xj& ) x

n j'1

% j (&)T(xj& ) x
n j'1

Therefore, similar to the proof of Lemmas 2 we have:

plimn64

n j'1

j
n

4 uj

&

n j'1

u j u j&
n

' 0.

(24)

(Exercise: Prove this). Moreover, 1 j (uj&u ) ' n j'1

n 4

j n j'1
n

4 uj

21 4 1 1 1 3 2 & 4 n j u j % 6 u nu n . u j uj & 3 n j'1 n j'1 n n n n n

(25)

Since nu - N(0 , 2) , plimn64(1/n)'j'1uj ' 0 and plimn64(1/n)'j'1uj ' 2 , it follows now that plimn64 1 u j u j&
n 4

&

n j'1

n j'1

j uj
n

' 0.

(26)

(Exercise: Why?). Combining (24) and (26), Lemma 3 follows.

LEMMA 4: Under Assumptions 1-2 and the null hypothesis (2) we have plimn64 [ (1/ n)'j'1(u j &34) & (1/ n)'j'1 (uj & 62uj % 34 ) ] ' 0 .
n 4 n 4 2

Proof: It follows from (10), Lemma 1, and the law of large numbers and the central limit theorem that plimn642 ' 2 , (Exercise: Why?) and thus plimn64 n( 4&4) & 22 1
2 j (uj & ) n 2

n 2&2 6 N(0 , 24) in distr ,

(27)

' 0

(28)

n j'1

(Exercise: Why?). Combining the latter result with Lemma 3, Lemma 4 follows. Finally, it follows from (6) that E uj &32uj uj &62uj %34
4 2 3

'

0 0

, Var

uj &32uj uj &62uj%34
4

'

66 0

0 248

(29)

hence, using the central limit theorem, it follows that 6

j n j'1 u j4&62uj%34
n

u j &32uj

6 N2

0 0

66 0

0 248

in distr.

(30)

Using this result and those of Lemmas 1-4, it follows that 1 j

n

(uj/)3

n j'1 (uj/)4 & 3 hence

6 N2

0 0

6 0 0 24

in distr.

(31)

THEOREM 1: Under Assumptions 1-2 and the null hypothesis (2) we have TN ' n [(1/n)'j'1 (u j/ )3 ]2 6
n

[(1/n)'j'1 (uj/)4 & 3 ) ]2 24

6 2 in distr.

(32)

Proof: Exercise.

The statistic TN is the test statistic of the Kiefer-Salmon (1983) test. The Jarque-Bera (1980) test is derived in a slightly different way, but is essentially the same.

REFERENCES Bierens, H.J. (2004), Introductions to the Mathematical and Statistical Foundations of Econometrics, Cambridge University Press. Jarque, C.M. and A.K. Bera, (1980), "Efficient Tests for Normality, Homoscedasticity and Serial Independence of Regression Residuals". Economics Letters 6, 255BB259. Kiefer, N. and M. Salmon (1983), "Testing Normality in Econometric Models", Economic Letters 11, 123-127.