Kruskal-Wallis Multiple Comparisons with a MINITAB Macro

Dunns Test

Steven Orlich
Minitab, Inc.

Introduction

The Kruskal-Wallis test is the nonparametric analog to One-way Analysis of Variance.
This test is used to determine if there are significant differences among the population
medians (instead of the population means). The Kruskal Wallis H statistic
1
is an overall
test statistic that enables one to test the general hypothesis that all population medians are
equal. Often, the investigator is not extremely interested in this general hypothesis but is
interested in comparisons amongst the individual groups. This paper presents a
MINITAB macro for performing multiple comparisons in a nonparametric setting.

Dunns Test

An effective way of doing pairwise simultaneous inference was introduced by Dunn
(1964). We first combine the data, rank it, find the group mean ranks, and then take the
standardized absolute differences of these average ranks.

Let k = the number of treatments, 2 f k

Let i R = the sum of the ranks for the ith treatment, i = 1,,k

Let
i
i
i
n
R
R

Where i n = the number of observations for the ith treatment

Let

j i R R


Where j = 1,,k and ji

Where
1
]
1

+
1
]
1

j i n n
N N 1 1
12
) 1 (



1
H statistic: ) 1 ( 3
) 1 (
12
2
+

,
_

+

N
n
R
N N
i
i
i


Where

i
ni N

We will then declare significance if:

z

Where
) 1 (

k k
Z z



Where is a specified family alpha value, 1 0 p p

Macro Command and Subcommands

1. %Krusmc CC: This is the main command where you specify your treatment
groups. By default, this command expects that your data are in stacked format
(you have a column of responses and a column of subscripts). If your data are not
in stacked format, you must specify the Unstacked subcommand. By default,
this command will do all pairwise comparisons among your groups. If you want
comparisons to a control group, you must specify the Control subcommand.
2. Unstacked: Specify this subcommand if your groups are in unstacked format.
3. Control C: Specify this subcommand if you want to make comparisons to a
control group. If your data are in stacked format, you must specify the control
group in a separate column. Be careful not to specify the control group twice:

Ex. %Krusmc c1-c4;
Control c3;
Unstacked.

This command will generate an error message because the column C3 is given twice.
The reason this error message is necessary is because we do not want to make a
comparison between the control group and itself.

4. Falpha k: You can specify a family alpha value. The default is 0.20.

Example 1: All pairwise comparisons

The following example (Appendix 1) was adopted from Gibbons (p. 175).

To make all pairwise comparisons among the four groups, we would need to enter the
following commands (the data are in unstacked format so we must also enter the
Unstacked subcommand):

MTB> %KrusMc c1-c4;
SUBC> unstacked.

Executing from file: krusmc.MAC

Macro is running ... please wait


Kruskal-Wallis: Multiple Comparisons

Kruskal-Wallis Test on the data

Group N Median Ave Rank Z
Group 1 10 25.500 26.0 1.72
Group 2 10 12.500 12.2 -2.59
Group 3 10 8.500 9.0 -3.59
Group 4 10 34.500 34.8 4.47
Overall 40 20.5

H = 31.89 DF = 3 P = 0.000


Kruskal-Wallis: All Pairwise Comparisons

----------------------------------------
Comparisons: 6
Family Alpha: 0.2
Bonferroni Individual Alpha: 0.033
Bonferroni Z-value (2-sided): 2.128
----------------------------------------

Standardized Absolute Mean Rank Differences
|Rbar(i)-Rbar(j)| / Stdev

Rows: Group i = 1,...,n
Columns: Group j = 1,...,n

1. Table of Z-values

Group 1 0.00000 * * *
Group 2 2.63957 0.00000 * *
Group 3 3.25164 0.61207 0.00000 *
Group 4 1.68320 4.32277 4.93484 0

2. Table of P-values

Group 1 1.00000 * * *
Group 2 0.00830 1.00000 * *
Group 3 0.00115 0.54049 1.00000 *
Group 4 0.09234 0.00002 0.00000 1

----------------------------------------------------------


Sign Confidence Intervals controlled at a family error rate of 0.2

Desired Confidence: 86.761

Sign confidence interval for median

Achieved
N Median Confidence Confidence interval Position
Group 1 10 25.50 0.6563 ( 24.00, 27.00) 4
0.8676 ( 23.20, 27.80) NLI
0.8906 ( 23.00, 28.00) 3

Group 2 10 12.50 0.6563 ( 10.00, 16.00) 4
0.8676 ( 7.61, 16.80) NLI
0.8906 ( 7.00, 17.00) 3

Group 3 10 8.50 0.6563 ( 5.00, 12.00) 4
0.8676 ( 4.20, 12.80) NLI
0.8906 ( 4.00, 13.00) 3

Group 4 10 34.50 0.6563 ( 33.00, 36.00) 4
0.8676 ( 32.20, 37.59) NLI
0.8906 ( 32.00, 38.00) 3


Kruskal-Wallis: Conclusions

The following groups showed significant differences:

Groups Z-value vs. Critical value P-value

Group 3 vs. Group 4 4.93484 >= 2.128 0.0000
Group 2 vs. Group 4 4.32277 >= 2.128 0.0000
Group 1 vs. Group 3 3.25164 >= 2.128 0.0011
Group 1 vs. Group 2 2.63957 >= 2.128 0.0083


The first part of the session window output will display the number of comparisons (k)
being made,

2
) 1 (

k k
k

the family alpha (), the Bonferroni individual alpha (),

k

and the 2-sided Critical z-value.

The next section displays our standardized group mean rank differences () and the p-
values associated with these differences. Notice that these tables are symmetric so there
are asterisks in the upper triangular part of the table. Also, there are zeroes down the
diagonal of the table because the diagonal represents a comparison between a group and
itself (these comparisons are meaningless and arent considered in the analysis). An
example of how to read this table is the following: What is the difference between
groups 2 and 4? In the Table of Z-values, we would go down to row 4 (Group 4) and go
over 2 columns (Group 2), and we would see that the z-value is 4.32277. If we did the
same thing in the Table of P-values, we would find that the p-value is 0.00002.

The third section displays sign confidence intervals for the medians. The confidence
levels for these intervals are controlled at your family alpha. Since these intervals are
controlled at an overall family alpha, we can compare these intervals pairwise (Appendix
2). It is important to keep in mind that the desired confidence may not be achieved for all
or some intervals. Also, this overall coverage by the family alpha is not exact when the
sample sizes are different, but it is often a reasonable approximation.
The final section tells us what our significant differences are (if there are any). In this
section I present the z-value, the critical z-value, and the p-value associated with the z-
value.

layout;


Group 1 Group 2 Group 3 Group 4
0
10
20
30
40
Groups
R
e
s
p
o
n
s
e
Z -Z 0
Group 1
Group 2
Group 3
R
e
f
e
r
e
n
c
e

G
r
o
u
p
Normal (0,1) Distribution
Group 2
Group 3
Group 4
Group 3
Group 4
Group 4
Multiple Comparisons Chart
Family Alpha: 0.2
Bonferroni Individual Alpha: 0.033
Desired Confidence: 86.761
Boxplots with Sign Confidence Intervals Pairwise Comparisons
Comparisons: 6
|Bonferroni Z-value|: 2.128

There are two graphs displayed here. The graph on the left displays boxplots of the
groups with their sign confidence intervals for the medians. This graph is extremely
useful because you can visually see the locations of the groups with respect to the others.
The graph on the right displays the non-absolute group mean rank standardized
differences. This graph is also extremely useful because we can look at not only the
magnitude of the group differences but the direction as well. It also displays the positive
and negative critical z-values so you can see if a difference is significant. If we look at
the Y-axis Group 3, we can see that Group 4 is much positively larger than Group 3
and it is significant since this distance goes beyond the z-value.

Example 2: Comparisons to a control group

Lets take the same example above but consider Group 3 (C3) as the control group. We
would need to enter the following commands if our data are unstacked:

MTB> %KrusMc c1 c2 c4;
SUBC> Control c3;
SUBC> Unstacked.



Executing from file: krusmc.MAC

Macro is running ... please wait

Kruskal-Wallis: Multiple Comparisons

Kruskal-Wallis Test on the data

Group N Median Ave Rank Z
Group 1 10 25.500 26.0 1.72
Group 2 10 12.500 12.2 -2.59
Group 4 10 34.500 34.8 4.47
Group 3 10 8.500 9.0 -3.59
Overall 40 20.5

H = 31.89 DF = 3 P = 0.000


Kruskal-Wallis: Comparisons to a Control

----------------------------------------
Comparisons: 3
Family Alpha: 0.2
Bonferroni Individual Alpha: 0.067
Bonferroni Z-value (2-sided): 1.834
----------------------------------------

Standardized Absolute Mean Rank Differences
|Rbar(i)-Rbar(Control)| / Stdev

Rows: Group i = 1,...,n-1
Column: Control Group: Group 3

Z-value vs. Critical value P-value

Group 1 3.25164 >= 1.83391 * 0.0011
Group 2 0.61207 < 1.83391 0.5405
Group 4 4.93484 >= 1.83391 * 0.0000

*: Significant difference
----------------------------------------------------------


Sign Confidence Intervals controlled at a family error rate of 0.2

Desired Confidence: 80.529

Sign confidence interval for median

Achieved
N Median Confidence Confidence interval Position
Group 1 10 25.50 0.6563 ( 24.00, 27.00) 4
0.8053 ( 23.57, 27.43) NLI
0.8906 ( 23.00, 28.00) 3

Group 2 10 12.50 0.6563 ( 10.00, 16.00) 4
0.8053 ( 8.72, 16.43) NLI
0.8906 ( 7.00, 17.00) 3



Group 4 10 34.50 0.6563 ( 33.00, 36.00) 4
0.8053 ( 32.57, 36.86) NLI
0.8906 ( 32.00, 38.00) 3

Group 3 10 8.50 0.6563 ( 5.00, 12.00) 4
0.8053 ( 4.57, 12.43) NLI
0.8906 ( 4.00, 13.00) 3

The formulas and calculations are the same except for the number of comparisons (),

= the number of treatment groups

the z-value (z),

k
Z z
2

and I place an asterisk next to any differences that are significant.

Group 3 Group 4 Group 2 Group 1
40
30
20
10
0
Groups
R
e
s
p
o
n
s
e
Z -Z 0
Group 4
Group 2
Group 1
G
r
o
u
p
N (0,1) Distribution
Multiple Comparisons Chart
Family Alpha: 0.2
Bonferroni Individual Alpha: 0.067
Desired Confidence: 80.529
Boxplots with Sign Confidence Intervals Comparisons to a Control
Comparisons: 3
|Bonferroni Z-value|: 1.834
Control Group: Group 3


These charts are similar to those in Example 1. The chart on the right shows the direction
and magnitude of the differences between the control group and the treatment groups.


Ties in the data

If there are ties in the data, a correction factor
2
is computed. The H statistic is then
adjusted
3
. The standard deviation () is also adjusted by this correction factor
4
. The
session window output will give both adjusted and non-adjusted tables. However, I will
only give p-values for the adjusted tables because these are the tables that should be used.
The main reason for displaying the non-adjusted tables is to show what the effects of the
ties had on the z-values. If the ties are extremely extensive, the validity of the data
should be questioned because these tests assume that the distributions are continuous.
Often, ties will have little or no impact on your conclusions.

Acknowledgements

Thanks to Dr. Tom Hettmansperger (The Pennsylvania State University) for his review of
this macro, our numerous discussions concerning this work, and for his overall time and
patience. Also thanks to Mr. Nicholas Bolgiano and Mr. Mike Delozier (Minitab, Inc.)
for their macro related suggestions and criticisms.


2




N q N q
u u let
,..., 1 ,..., 1
3

where u is the number of observations in all samples combined that are tied for any rank.

3
We divide the original H statistic by the following quantity:
( )

,
_

1
1
2
N N

4

[ ]
) 1 ( 12
1 1
) 1 (
2

1
]
1

N
n n
N N
let
j i

References

Gibbons, Jean D. Nonparametric Methods for Quantitative Analysis. New York: Holt,
Rinehart, and Winston, 1976.

Noether, Gottfried E. Introduction to Statistics: The Nonparametric Way. New York:
Springer-Verlag, 1991.



Appendix 1

Data from Examples 1 and 2

Group 1 Group 2 Group 3 Group 4
19 14 12 38
22 21 1 39
25 2 5 40
24 6 8 30
29 10 4 31
26 16 13 32
37 17 9 33
23 11 15 36
27 18 3 34
28 7 20 35


Appendix 2
Choosing the appropriate confidence levels for comparing
2
) 1 ( k k
intervals pairwise
controlled at an overall family alpha.

Let k = the number of treatment groups

Let = the family alpha

Let =
1
]
1

! 2 )! 2 (
!
k
k

We declare significance if the 2-sided confidence intervals do not overlap:

P
2
2
2
1
1

,
_

+
n
Z X
n
Z X p

P
2
1 1
1 1
1 1
2 1
2 1
2 1
2 1

,
_

,
_

+
+

n n
n n
Z
n n
X X
p


) 1 ( 2
2 1
2 1
1 1
1 1

,
_

+
k k
Z Z
n n
n n
Z

,
_

+
+

2 1
2 1
) 1 (
1 1
1 1
n n
n n
Z Z
k k

2 1
2 1
) 1 (
n n
n n
Z Z
k k
+
+

If n n n 2 1

n
n
Z Z
k k
2
2
) 1 (

2
1
) 1 (

k k
Z Z



let = the desired confidence level

We know that

1 ) ( 2 Z

So

1
2
1
) 1 (
2

,
_

k k
Z



Example

Let = 0.20

Let k = 4

167 . 0
) 1 (

k k

128 . 2 167 . 0 Z

50 . 1
2
167 . 0

Z


933 . 0 ) 50 . 1 (

86 . 0 1 ) 933 . 0 ( 2

Thus, if we have 4 groups and we want to compare intervals pairwise at a family alpha
rate of 0.20, we must have 86% confidence intervals.

Two points to keep in mind

1. An assumption of this proof is that 2 1 n n . However, if the sample sizes arent
equal, the approximation is still excellent for the confidence levels. This is
because
2 1
2 1
n n
n n
y
+
+
is a very flat function. Lets take the rare case where the
sample size of one treatment group is three times that of the other. Thus, 1 2 3n n
and
1 1
1
3
4
n n
n
y
+
. After simplification, 732 . 0
3 1
2

+
y . The estimate if
the sample sizes are equal is 707 . 0
2
1
so you can see that even in this rare
situation, the approximation is excellent.
2. Because the sign confidence intervals use linear interpolation, there may be
circumstances where the desired confidence levels may not be attained.