Sample Variance as a Consistent Estimator

for the Variance

Stat 305 Spring Semester 2011
The purpose of this document is to show that, for any random variable W , the sample
variance,
n
c 1 X ¹ )2
S2 = (Wi ¡ W
n ¡ 1 i=1

is a consistent estimator for the variance ¾2 of W .

To prove that the sample variance is a consistent estimator of the variance, it will be
helpful to have available some facts about convergence in probability. Recall from elementary
analysis that if fang and fbn g are sequences of real numbers and an ! a and bn ! b, then
it is easy to show that an § bn ! a § b. Analogous types of results hold for convergence
in probability. These results are collected in the following theorem. We present them
without proof, but the proof is relatively straightforward. In what follows, the notation
‘¡!P ’ stands for convergence in probability.
0
Theorem 1 If ^µ n ¡!P µ and µ^n ¡!P µ 0, then
0
(i) ^µn § ^µ n ¡!P µ § µ 0 ,
0
(ii) ^µ n ¢ ^µn ¡!P µ ¢ µ 0,
0
(iii) ^µ n= ^µ n ¡!P µ=µ 0 assuming that µ 0 6= 0,
(iv) if g is any real-valued function that is continuous at µ, then g(^µ n ) ¡!P g(µ), and
(v) if fan g and fbn g are any two sequences of real numbers such that an ! 1 and bn ! 0,
then a n^µ n + bn ¡!P µ.

Proof. Omitted.
We are now ready to prove the main theorem of this document. The idea of the proof
is to break up the sample variance into su¢ciently small ‘pieces’ and then combine using
Theorem 1.

Theorem 2 Let W be any random variable such that ¹, ¹ 2, and ¹4 are all …nite. Then S 2
is consistent for the variance ¾ 2 of W .

Proof. Recall that

1 X
n
c2 =
S (Wi ¡ W¹ n)2
n
n ¡ 1 i=1
à n !
n 1X ¹ n)2
= (Wi ¡ W
n ¡ 1 n i=1
ÃÃ n ! !
n 1X 2 ¹ n )2 .
= W ¡ (W
n¡1 n i=1 i

1
Note that

1X 2
n
W
n i=1 i

is an unbiased estimator for ¹ 2, the second moment of W . But observe that

à n ! n
1X 2 1 X
V ar Wi = 2
V ar(Wi2 )
n i=1 n i=1
1 X
n
= (¹ ¡ ¹22)
n 2 i=1 4
1
= (¹ ¡ ¹22).
n 4
Clearly, 1n (¹ 4 ¡ ¹22) ! 0 as n ! 1. Therefore, since the variances approach zero,
n
1X 2
W ¡!P ¹2 .
n i=1 i

¹ n ¡!P ¹ and g(x) = x2 is a continuous function, W

Since W ¹ n2 ¡!P ¹2 by part (iv) of the
previous theorem. Therefore, by part (i),
n
1X 2 ¹ n2 ¡!P ¹2 ¡ ¹2 = ¾2.
W ¡W
n i=1 i

Finally, since n=(n ¡ 1) ! 1 as n ! 1,

cn2 ¡!P ¾ 2
S

by part (v). This completes the proof.