Communications in Statistics - Theory and Methods

ISSN: 0361-0926 (Print) 1532-415X (Online) Journal homepage: http://www.tandfonline.com/loi/lsta20

A Note on Determining the p -Value of Bartlett's

Test of Homogeneity of Variances
J. Wu & A. C. M. Wong
To cite this article: J. Wu & A. C. M. Wong (2003) A Note on Determining the p -Value of
Bartlett's Test of Homogeneity of Variances, Communications in Statistics - Theory and
Methods, 32:1, 91-101, DOI: 10.1081/STA-120017801
To link to this article: http://dx.doi.org/10.1081/STA-120017801

Published online: 02 Sep 2006.

Submit your article to this journal

Article views: 49

View related articles

Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=lsta20
Download by: [Kangwon National University - Samcheok Campus]

Date: 23 December 2015, At: 22:51

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

COMMUNICATIONS IN STATISTICS
Theory and Methods
Vol. 32, No. 1, pp. 91101, 2003

A Note on Determining the p-Value of

Bartletts Test of Homogeneity of Variances
J. Wu1 and A. C. M. Wong2,*
1

Department of Biostatistics, St. Jude Childrens

Research Hospital, Memphis, Tennessee, USA
2
SASIT, York University, North York,
Ontario, Canada

ABSTRACT
Bartletts test for homogeneity of variances is rather nonrobust.
However, when it is applicable, it is more powerful than various
other tests. Dyer and Keating (1980) tabulate the exact critical
values for Bartletts test based on equal sample sizes from several
normal populations. Moreover, they use these values to obtain highly
accurate approximations to the critical values for unequal sample
sizes. In this note, a simple and accurate method is proposed to
obtain the p-value for Bartletts test. Theoretically, the proposed
method has third order accuracy. Numerical examples illustrate
that it is extremely accurate even for very small sample sizes and a

*Correspondence: A. C. M. Wong, SASIT, York University, 4700 Keele Street,

North York, Ontario, Canada M3J 1P3; E-mail: august@yorku.ca.
91
DOI: 10.1081/STA-120017801
Copyright & 2003 by Marcel Dekker, Inc.

0361-0926 (Print); 1532-415X (Online)

www.dekker.com

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

92

Wu and Wong

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

large number of populations. Furthermore, the proposed method can

easily be implemented with standard statistical softwares.
Key Words: Bartletts test; Homogeneity of variances; p-Value;
Saddlepoint approximation.

1. INTRODUCTION
Testing for homogeneity of variances in normal populations is
frequently of interest in many statistical analyses. For example, in
experimental design, it is important to test the validity of the assumption
that the within-group variances in a one-factor analysis of variance model
is homogeneous.
More specically, consider the model
i 1, . . . , k,

yij
i ij

j 1, . . . , ni

where yij is the jth response from the ith treatment,
i is the ith treatment
mean, and ij are independent random errors from N0, i2 . To test for
homogeneity of variances, the null hypothesis is H0 : 12    k2
whereas the alternative hypothesis is H1 : s2 6 t2 for some s 6 t.
Bartlett (1937) proposed a modied likelihood ratio test and showed
that the test statistic, Q=c, is asymptotically distributed as 2 with k  1
degrees of freedom, 2k1 , where
Q  log Sp2 

k
X

i log Si2 ,

i1
k
X
1
1  1
c1
3k  1 i1 i

and
i ni  1,

k
X
i1

k
i
1X
1X
yij  y i 2 , Sp2
 S2 :
i j1
 i1 i i
n

i ,

Si2

Let q be the observed value of Q calculated from the observed data. Then
the p-value for Bartletts test can be approximated by
PrQ > q PrQ=c > q=c  Pr2k1 > q=c:
The accuracy of this approximation is somewhat dicult to assess;
however, in certain situations: very small sample sizes and/or large

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

p-Value of Bartletts Test

93

numbers of populations, it is known to be inadequate (Bishop and Nair,

1939; Barnett, 1962).
Brown (1939) showed that Bartletts test is consistent against all
alternatives. Moreover, Pitman (1939) showed that it is an unbiased
test. However, as shown in Box (1953) and Box and Andersen (1955),
Bartletts test is rather nonrobust (i.e. sensitive to departures from
normality). It is therefore recommended that it be used only when
preceded by a preliminary test, which based on the data, does not reject
normality. When normality can be relied on, Bartletts test is apparently
more powerful than various other tests (Gartside, 1972). More specically,
Conover, Johnson and Johnson (1981) demonstrated that Bartletts test is
sensitive to non-normal kurtosis.
Chao and Glaser
Q (1978) showed that the exact null density of
L 1 expfQ=g ki1 Si2 i = =Sp2 (hereinafter referred to as the Bartlett
distribution) to be
gk u; n1 , . . . , nk Ku=21  log uk3=2 Eu,

0<u<1

where
"
K 2

k1=2

k   1=2
Y
i i
i1

Eu

1
X

#


k
 Y

1

i1

 
i
2

wr  log ur

i0

wr

X xt1 xt3 xt5   

1
1 3 5
,
k  1=2 r t1 !t3 !t5 !   

r 0

P
with indicating summation over all distinct sequences t1 , t3 , t5 , . . .
of nonnegative integers
satisfying t1 3t3 5t5    r; and xr
P
1r1 rr 11 ki1 i =r  1Br1 , r 1, and B1 , B2 , . . . are the
Bernoulli numbers.
For equal sample sizes, the size  exact critical values bk ; n1 , . . . , nk
for Bartletts test are tabulated by Dyer and Keating (1980) and
determined by the nonlinear equation
K=2k1=2

1
X
r0


wr r=2r  r k  1=2,


 log bk ; n1 , . . . , nk r=2 1  

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

94

Wu and Wong

where a, b is the incomplete gamma function. Highly accurate approximations to the critical values for unequal sample sizes are also obtained
by Dyer and Keating (1980). Madansky (1988) referred this approximation as the Dyer-Keating approximation.
The aim of this note is to provide a simple approximation for
PrQ > q PrL 1 < expfq=g. This approximation is extremely
accurate even for small sample sizes or large numbers of populations.
Numerical examples illustrate that the results are comparable to those
obtained by the Dyer-Keating approximation. The main advantage of the
proposed method is the simplicity in implementation with standard
statistical softwares such as Splus and R.
The proposed approximation is derived in Sec. 2. Numerical examples are examined in Sec. 3. Some concluding remarks are given in Sec. 4.

2. MAIN RESULT
Consider rst the classical problem involving random variables
X1 , . . . , Xn that are identically and independently distributed from a distribution whose cumulant generating function t is nite throughout an
interval for t which includes 0 in its interior. Then the saddlepoint
approximation due to Lugannani
and Rice (1980) for the tail probability
P
PrX > x , where X ni1 Xi =n is the sample mean, is
1  fw w1=w  1=ug;

an alternative approximation, due to Barndor-Nielsen (1986, 1991) is

1  fw w1 logu=wg:

In both cases,
h n
oi1=2
w sgn^ 2n ^x  ^

and

n
o1=2
u ^ n 00 ^

while ^, the saddlepoint, is dened via the saddlepoint equation

0 ^ x

and 0  and 00  are the rst and second derivatives of  with respect
to . Other related tail area approximations are given in Daniels (1987).

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

p-Value of Bartletts Test

95

For further background, see also Barndor-Nielsen and Cox (1979,

1989), and Reid (1996).
The saddlepoint approximation to the tail area of X is known to be
extremely accurate, even for values of n as low as 3, 2, or even 1.
Theoretically, the Lugannani and Rice formula and the Barndor
Nielsen formula has order of accuracy On3=2 (see for example
Daniels (1987), Lugannani and Rice (1980) and Barndor-Nielsen and
Cox (1979, 1989)).
Bartletts test statistic does not involve a sample mean or total;
nevertheless, it does involve a signicant amount of convolution so
that the saddlepoint method is applicable. All we have to do is to
obtain the cumulant generating function for Q, K, so that K
plays the role of n . Thus PrQ > q PrL 1 < expfq=g can be
approximated from Eqs. (1) and (2) with
h n
oi1=2
n
o1=2
w sgn^ 2 ^q  K^
and u ^ K 00 ^
,
5
and
K 0 ^ q:

6
12

k2

Si2

Assume that

 , then
is distributed as  =i 2i ,
and the moment generating function for log Si2 is
!
h  i
 

2
2
mlog Si2  E exp  log Si
E 2i
i
!


 
2 2

 i  1 i :
i
2
2
Hence, the cumulant generating function for log Si2 is
!


 
2 2
log  i   log  i :
 log
i
2
2
P
Note that Sp2 is distributed as  2 =2 . Let W1  ki1 i log Si2 and
W2  log Sp2 . Hence the moment generating functions for W1 and W2 are
!i 
k


 
Y
2 2
mlog Si2 i 
 i  i  1 i
mW1 
i
2
2
i1
i1
!



2 2
   1
mW2 
2
2

k
Y

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

96

Wu and Wong

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

respectively. Moreover, Q and Sp2 are independent. Since we have W1

Q W2 , the moment generating function for Q is mW1 =mW2 . Thus
the cumulant generating function for Q can be written as
K

k n


 o
X
i log i log  i  i   log  i
2
2
i1


n
o
  log  log     log 
:
2
2

Hence
K 0 

k n
X

2i

i1

K 00 




o
i
 
 i    log  
2
2




i
0
 i   2 0  
2
2

i log i  i

where  d log =d is the digamma function and 0  d =d

is the trigamma function. Solution of the saddlepoint ^ requires a numerical
algorithm. The iterative Newton-Raphson algorithm may be used to
obtain the saddlepoint ^ for a xed value of q. Thus PrQ > q can be
approximated from either Eqs. (1) or (2) with w and u dened in Eq. (5),
and the cumulant generating function K and its rst two derivatives
are dened in Eqs. (7)(9) respectively.

3. NUMERICAL STUDIES
Example 1. (comparing with Dyer and Keatings results)
To illustrate the accuracy of the saddlepoint approximation, suppose
k 5 and n1 8, n2 10, n3 5, n4 12, n5 13, the approximate 25th
percentile b4 0:25; 8, 10, 5, 12, 13 of the Bartlett distribution is 0.8762
(Dyer and Keating, 1980). In other words PrL 1 < 0:8762  0:25. Since
expfq=g 0:8762, q 43 log0:8762 5:6829. By solving Eq. (6),
we have ^ 0:1231. Thus from Eqs. (1) and (2), PrQ > 5:6829
PrL 1 < 0:8762 is 0.2498 and 0.2505 respectively.
Consider the small sample size case: k 5, and n1 5, n2 4, n3 6,
n4 4, n5 5, Dyer and Keating (1980) gives PrL 1 < 0:5782  0:05.
In this case, q 19 log0:5782 10:4089. By solving Eq. (6), we
have ^ 0:2673. Thus from Eqs. (1) and (2), PrQ > 10:4089
PrL 1 < 0:5782 is 0.0511 and 0.0513 respectively.

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

p-Value of Bartletts Test

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

Table 1.
Group
Group
Group
Group

1
2
3
4

1
12
12
13

97

Data from paired-association learning task experiment.

8
10
4
14

9
13
11
14

9
13
7
17

4
12
8
11

0
10
10
14

1
12
13

5
14

For a large number of populations, say k 10, from Dyer and

Keating (1980), the approximate 25th percentile bk 0:25, 9, 6, 5, 9, 10,
8, 7, 6, 8, 5 of the Bartlett distribution is 0.82488, and correspondingly
q 63 log0:82488 12:1286. That is PrL 1 < 0:82488  0:25. By
solving Eq. (6), we have ^ 0:1004, and from Eqs. (1) and (2),
PrQ > 12:1286 PrL 1 < 0:82488 is 0.2481 and 0.2483 respectively.
These examples illustrate that the proposed method give very good
approximations even for small sample sizes and a large numbers of
populations.
Example 2. (Paired association learning task experiment)
The data in Table 1 are from a paired-association learning task
experiment performed on subjects under the inuence of two drugs
(Cochran and Cox, 1957). Group 1 is a control (no drug), Group 2 was
given drug 1, Group 3 was given drug 2, and Group 4 was given both
drugs. Cochran and Cox (1957) assumed that the groups are independently
distributed as normal.
From the data, we have q 8:3426. Moreover, the saddlepoint ^ is
0.2924. From Eqs. (1) and (2), PrQ > 8:3426 PrL 1 < 0:7163 is
0.0501 and 0.0503 respectively. Applying Dyer and Keatings method,
PrL 1 < 0:7169  0:05. The proposed method gives approximately the
same result as Dyer and Keatings method.
Example 3. (Bids on Texas oshore oil and gas leases)
Dyer and Keating (1980) reported the sealed bids on each of ve Texas
oshore oil and gas leases selected from 110 leases issued on May 21, 1968.
Crawford (1970) concluded that bid on each of these 5 leases were
log-normally distributed. Table 2 reported some summary statistics on
the log bid, where yij log (bids of ith leases from jth company).
Dyer and Keating (1980) reported that the hypothesis of equal
variances is not rejected at a signicance level of 0.25 or lower.
From the data, we have q 3:9642. Applying the saddlepoint
method, we have ^ 0:0433. Thus for testing the hypothesis of equal

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

98

Wu and Wong

Table 2.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

i
ni
y i
Si2

Summary statistics for the log (bids of ith leases from jth company).
1

8
14.859
0.842

10
14.306
1.282

5
12.726
0.859

12
15.237
1.883

13
14.663
2.635

variances, the p-value PrQ > 3:9642 PrL 1 < 0:9141 is approximately 0.4535 (if using (Eq. 1)) and 0.4544 (if using (Eq. 2)). This
agrees with the result of Dyer and Keating.

4. DISCUSSION
In this note, we have proposed a simple and accurate approximation
to the p-value of Bartletts test of homogeneity of variances. Numerical
examples illustrated that the proposed method is extremely accurate even
for small sample sizes or large numbers of populations. An advantage of
the proposed method is the ability to compute p-values with standard
statistical softwares. To illustrate the last point, an R source code for
Example 2 is given in Appendix.

APPENDIX
(R Source Code for Example 2)
#

Input data

y1
y2
y3
y4

<<<<-

Calculate the required summary statistics and q

c(1, 8, 9, 9, 4, 0, 1)
c(12, 10, 13, 13, 12, 10)
c(12, 4, 11, 7, 8, 10, 12, 5)
c(13, 14, 14, 17, 11, 14, 13, 14)

vi <- c(length(y1)-1, length(y2)-1,

length(y4)-1)
v <- sum(vi)
S12 <- sum((y1-mean(y1))^2)/vi[1]

length(y3)-1,

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

p-Value of Bartletts Test

99

S22 <- sum((y2-mean(y2))^2)/vi[2]

S32 <- sum((y3-mean(y3))^2)/vi[3]
S42 <- sum((y4-mean(y4))^2)/vi[4]
Sp2 <- (vi[1]*S12vi[2]*S22vi[3]*S32vi[4]*S42)/v
q <- v*log(Sp2)-(vi[1]*log(S12)vi[2]*log(S22)vi[3]
*log(S32)vi[4]*log(S42))
# Define the cgf and its first two derivatives
K <- function(t){
sum(t*vi*log(vi)lgamma(vi/2-vi*t)-lgamma(vi/2))(t*v*log(v)lgamma(v/2-v*t)-lgamma(v/2))
}
dK <- function(t) {
sum(vi*log(vi)-vi*digamma(vi/2-vi*t))-v*log(v)
v*digamma(v/2-v*t)
}
d2K <- function(t) { sum(vi*vi*trigamma(vi/2-vi*t)) v*v*trigamma(v/2-v*t)}
# Use Newton-Raphson method to solve for the saddlepoint xihat
xihat <- 0.1
for (i in 1:20) { xihat <- (q-dK(xihat))/d2K(xihat)
xihat }
#

Calculate equation (5)

u <- xihat*sqrt(d2K(xihat))
w <- sign(xihat)*sqrt(2*(xihat*q-K(xihat)))
#

Equation (1) is "lr" and equation (2) is "bn"

lr <- 1-pnorm(w)-dnorm(w)*(1/w-1/u)
bn <- 1-pnorm(wlog(u/w)/w)

ACKNOWLEDGMENTS
The rst authors research was supported in part by Cancer Center
Support Grant CA 21765 and American Lebanese Syrian Associated
Charities (ALSAC).The second authors reasearch was supported in

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

100

Wu and Wong

part by National Science and Engineering Research Council of Canada.

The authors would also like to thank the referrees, and the associate
editor for their insightful recommendations that greatly improved this
article.

REFERENCES
Barndor-Nielsen, O. E. (1986). Inference on full or partial parameters
based on the standardized signed log likelihood ratio. Biometrika
73:307322.
Barndor-Nielsen, O. E. (1991). Modied signed log likelihood ratio.
Biometrika 78:557563.
Barndor-Nielsen, O. E., Cox, D. R. (1979). Edgeworth and saddlepoint
approximations with statistical applications (with discussions).
J. Royal Statist. Soc. B 41:279312.
Barndor-Nielsen, O. E., Cox, D. R. (1989). Asymptotic Techniques for
Use in Statistics. New York: Chapman and Hall.
Barnett, V. D. (1962). Large sample tables of percentage points
for Hartleys correction to Bartletts Criterion for testing the
homogeneity of a set of variables. Biometrika 49:487494.
Bartlett, M. S. (1937). Properties of suciency and statistical tests.
Proceedings of the Royal Society of London, Ser. A 160:268282.
Bishop, D. J., Nair, U. S. (1939). A note on certain methods of testing for
the homogeneity of a set of estimated variances. J. Royal Statist.
Soc. B 6:8999.
Box, G. E. P. (1953). Nonnormality and tests of variances. Biometrika
40:318335.
Box, G. E. P., Andersen, S. L. (1955). Permutation theory in the derivation
of robust criteria and the study of the departures from assumption.
J. Royal Statist. Soc. B 17:126.
Brown, G. W. (1939). On the power of the L1 test for equality of several
variances. Ann. Math. Statist. 10:119128.
Chao, M., Glaser, R. E. (1978). The exact distribution of Bartletts test
statistic for homogeneity of variances with unequal sample sizes.
J. Amer. Statist. Assoc. 73:422426.
Cochran, W. G., Cox, G. M. (1957). Experimental Designs. New York:
Wiley.
Conover, W. J., Johnson, M. E., Johnson, M. M. (1980). A comparative
study of tests for homogeneity of variances with applications to the
outer continental shelf bidding data. Technometrics 23:351362.

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

p-Value of Bartletts Test

101

Crawford, P. B. (1950). Texas oshore bidding patterns. J. Petro. Techno.

22:283289.
Daniels, H. E. (1957). Tail probability approximations. Int. Statist. Rev.
55:3748.
Dyer, D. D., Keating, J. P. (1980). On the determination of critical values
for Bartletts test. J. Amer. Statist. Assoc. 75:313319.
Gartside, P. S. (1972). A study of methods for comparing several
variances. J. Amer. Statist. Assoc. 67:342346.
Lugannani, R., Rice, S. (1980). Saddlepoint approximation for the
distribution of the sum of independent random variables. Adv. in
Appl. Probab. 12:475-490.
Madansky, A. (1988). Prescriptions for Working Statisticians. New York:
Springer-Verlag.
Pitman, E. J. G. (1939). Tests of hypotheses concerning location and
scale parameters. Biometrika 31:200215.
Reid, N. (1996). Likelihood and higher order approximations to tail
areas: a review and annotated bibliography. Canad. J. Statist.
24:141166.

MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016

Downloaded by [Kangwon National University - Samcheok Campus] at 22:51 23 December 2015

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.