OLS Under Heteroskedasticity Testing for Heteroskedasticity

Heteroskedasticity and Weighted Least Squares

Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009

April 14, 2009

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

The Classical Linear Model:

1 2 3 4

Linearity: Y = X + u. Strict exogeneity: E(u) = 0 No Multicollinearity: (X) = K. No heteroskedasticity/ serial correlation: V (u) = 2 In .

Gauss/Markov Theorem: = (X X)1 X Y is best linear unbiased. V () = S 2 (X X)1 is an unbiased estimate of V () = 2 (X X)1 .

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

What happens if we drop the homoskedasticity assumption? (the OLS estimator) is still linear and unbiased (Why?). Though linear and unbiased, is not the minimum variance estimate (inecient). V () = S 2 (X X)1 is biased. This makes standard t and F tests invalid.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Intuition

The presence of heteroskedastic errors should not alter the central position of the OLS line (unbiasedness). OLS weigths all observations equally, but in this case it makes more sense to pay more attention to observations where the variance is smaller.
Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

The plan: what to do with heteroskedasticity.

1

Before abandoning OLS we will see how to test for heteroskedasticity. Strategy 1: Propose another more ecient and unbiased estimator for (weighted least squares (WLS)) and a suitable estimator for its variance. Strategy 2: Keep using OLS (it is still unbiased, though inecient), but nd a replacement for its variance (the old one is biased under heteroskedasticity).

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Testing for heteroscedasticity

a) The White test

H0 : no heteroscedasticity, HA : there is heterocedasticity of some form. Consider a simple case with K = 3: Yi = 1 + 2 X2i + 3 X3i + ui 1, . . . , n

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Steps to implement the test:

1 2

Estimate by OLS, save squared residuals in e2 . Regress e2 on all variables, their squares and all possible non-redundant cross-products. In our case, regress e2 on 2 2 1, X2 , X3 , X2 , X3 , X2 X3 , and obtain R2 in this auxiliar regression. Under H0 , nR2 2 (p). p = number of explanatory variables in the auxiliar regression minus one. Reject Ho if nR2 is too large.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Intuition: The auxiliar model can be seen as trying to model the variance of the error term. If the R2 of this auxiliar regression were high, then we could explain the behavior of the squared residuals, providing evidence that they are not constant. Caveats: Valid for large samples. Informative if we do not reject the null (no heterocedasticity). When it rejects the null: there is heterocedasticity. But we do not have any information regarding what causes heterocedasticity. This will cause some trouble when trying to construct a GLS estimator, for which we need to know in a very specic way what causes heterocedasticity.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

b) The Breusch-Pagan/Godfrey/Koenker test

Mechanically very similar to Whites test. Checks if certain variables cause heterocedasticity. Consider the following heteroscedastic model: Y = X + u, ui normal, with E(u) = 0 and V (ui ) = h(1 + 2 Z2i + 3 Z3i + . . . + p Zpi ) where h( ) is any positive function with two derivatives. When 2 = . . . = p = 0, V (ui ) = h(1 ), a constant!! Then, homoscedasticity H0 : 2 = 3 = . . . = p = 0 ,and HA : 2 = 0 3 = 0 . . . p = 0.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Steps to implement the test:

1 2

Estimate by OLS, and save squared residuals e2 . i Regresss e2 on the Zik variables, k = 2, . . . , p and get (ESS). i The test statistic is: 1 ESS 2 (p 1) 2 (p) 2 under H0 , asymptotically. We reject if it is too large.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Comments: Intuition is as in the White test (a model for the variance). By focusing on a particular group, if we reject the null we have a better idea of what causes heterocedasticity. Accepting the null does not mean there isnt heterocedasticity (why?). Also a large sample test.
2 Koenker (1980) has proposed to use nRA as a test, which is still valid if errors are non-normal.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Estimation and inference under heteroscedasticity

For simplicity, consider the two variable case Yi = 1 + 2 Xi + ui where now the error term is heteroskedastic, that is
2 V (ui ) = i ,

i = 1, . . . , n

We will assume all the other classical assumptions still hold

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Divide each observation of the linear model by i : Yi 1 Xi ui = 1 + 2 + i i i i Yi = 1 X1i + 2 X2i + k Xki + u i Note that V (u ) = V (ui /i ) = 1, then the residuals of this i transformed model are homoscedastic.
2 Then, if we know i , the BLUE is simply the OLS estimator using the transformed variables.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

In our case, the OLS with the transfored model is 2,wls =

n i=1 xi yi n 2 i=1 xi

1,wls = Y 2,wls X with Yi = Yi /i , Xi = Xi /i and lowercase letters are deviations from sample means, as usual. This is the weighted least squares estimator.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

The name weighted least squares come from the fact that the estimator can be obtained by solving the following minimization problem n 1 2 min e 2 i i=1 i that is, errors enter the SSR weighted by the inverse of the variance for each observation: we pay more attention to observations with smaller variance.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

2 Problem: in practice we do not know i . This leads to two strategies

1

Use WLS: Pros: estimates will be unbiased and ecient. Cons: we need to know the variances in advance (or make assumptions) Keep OLS but change its variance estimator: Think again about the eects of heteroscedasticity on standard estimation procedures. OLS is still unbiased though not ecient (not that bad...). But, S 2 (X X)1 is biased, which invalidates inference (this is bad!). Then, a second strategy: keeping OLS for and look for a valid estimator for its variance. Pros: no assumptions needed, unbiased. Cons: we will lose eciency with respect to the WLS case (if available).

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

1) Known variance structure: WLS

Consider our simple two-variable case: Yi = 1 + 2 Xi + ui Strategy: assume some particular forms of heteroscedasticity. a) V (ui ) = 2 Xi2 2 is an unknown constant. Divide all observations by Xi Yi Xi 1 ui + 2 + Xi Xi

= 1

Yi = 1 X0i + 2 + u i

Note E(u ) = E(ui /Xi )2 = 2 Xi2 = 2 i

i

X2

Errors of the transformed model are homocedastic. Do OLS on the transformed model!
Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

We do not need to know 2 . What have just divided by the part of the standard error that varies over observations, that is, by Xi . This strategy provides a WLS. Careful with interpretations. The intercept of the transformed model is the slope of the original model and that the slope of the transformed model is the intercept of the original one.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

b) V (u) = 2 Xi Y i Xi 1 = 1 + 2 Xi ui Xi + Xi

Yi = 1 X0i + 2 X1i + u i

As for implementation and interpretation, note that the transformed model has no intercept, and that the coecient of the rst explanatory variable corresponds to the intercept of the original model, and the coecient of the second variable corresponds to the slope of the original model. Problem with these strategies: it is dicult to nd an exact form for heterocedasticity

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

2) Unknown variance structure

Alternative strategy: retain OLS (still unbiased though not ecient) and look for valid estimators for its variance. Variance matrix of OLS under heteroscedasticity can be shown to be: V (OLS ) = (X X)1 X X(X X)1
2 2 2 = diag(1 , 2 , . . . , n ).

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

White (1980): a consistent estimator for X X is X DX, D = diag(e2 , e2 , . . . , e2 ), ei s OLS residuals n 1 2 Then, a heteroscedasticity consistent estimator of the variance matrix is: V (OLS )HC = (X X)1 X DX(X X)1 Strategy: use OLS but replace S 2 (X X)1 by Whites consistent estimator. This strategy is not ecient, but it does not require assumptions about the structure of heteroscedasticity.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares

OLS Under Heteroskedasticity Testing for Heteroskedasticity

Summary Heteroskedasticity makes OLS inecient and invalidates the standard estimator of its variance, and hence invalidates inference (t tests, F tests, etc.). The WLS estimator is ecient and unbiased but it depends on knowing the variance structure. In practice is seldom available. In practice it is more common to keep OLS and replace its variance estimator by Whites consistent method, which does not require any assumptions.

Walter Sosa-Escudero

Heteroskedasticity and Weighted Least Squares