ANALYSIS OF VARIANCE

(ANOVA)

Dr. Agus Saptoro

Bandung, 17th April 2017

Agus Saptoro, 2017

About me
Educational backgrounds
BEng (Hons, Gadjah Mada University, Indonesia)
PhD (Curtin University, Australia)
Research Fellowship (University of Hyogo, Japan)

Professional membership
Senior Member of American Institute of Chemical Engineers
(AIChE)
Member of Institution of Chemical Engineers (IChemE), UK

2
 Outline of presentation
ANOVA technique
One-Way ANOVA
Systematic calculation
Example
Two-Way ANOVA
Systematic calculation
Example

How much students learns is more than important

than how much the lecturers teach
 Reference

Walpole, R.E., Myers, R.H., Myers, S.L.

and Ye, K., Probability & Statistics for
Engineers & Scientists, 8th Ed., (2007),
Prentice Hall, Inc., New Jersey, Chapter
13.
 Unit learning outcomes

After completing this lecture, I should be able

to:
Understand basic principles of one-way and two-
way ANOVA

Apply one-way and two-way ANOVA in solving

engineering related problems
 Experimental design
- ANOVA technique -

ANOVA method for testing the differences between more

than 2 means and variances

The generalized test:

H0 1 2 ...k
H1 i j for at least one pair i,j

Ex: compare the performance of 4 kind of catalyst test 3

samples of each type and measure the product yields (y)

Treatment / Variety ( j ) 1 2 3 4

Yields 10.1 11.4 13 14.4

( yij ) 8.9 16.4 8.2 19.4
4.7 8.5 9.4 20.80

k = no. of treatments = 4 n = no. of samples for each treatment = 3

 ANOVA technique
- assumptions -

All observations are independent

The data is normally distributed
The mean of each treatment may be different
The variance of all treatments are the same
The null hypothesis:
H0 1 2 ...k
The alternative hypothesis:

H1 i j for at least one pair i,j

Note: - sum over one subscript

- sum over both subscripts
 ANOVA technique
- notation / general calculation -

Treatment 1 2 K Full exp.

Y11 Y12 Y1k
Observations
Yn11 Yn2 2 Ynk k
k nj
Treatment Totals Y1 Y2 Yk Y Yij
j 1 i 1

Sample sizes n1 n2 nk n
Y
Sample means Y1 Y2 Yk Y
n
Residual SS (SSR) k nj

SST Yij Y
nj 2

SS j Yij Y j
2
SS1 SS2 SSk j1 i1
i1
k
Pooled SS SSE SS j

j 1

 ANOVA technique
- example / general calculation -

Treatment / Variety ( j ) 1 2 3 4 Full exp.

10.1 11.4 13.0 14.4
Yields ( yij )
8.9 16.4 8.2 19.4
4.7 8.5 9.4 20.8
k nj

Treatment Totals ( y j ) 23.7 36.3 30.6 54.6 Y Yij 145.2

j 1 i 1

Sample sizes ( n j ) 3 3 3 3 n 12
Y
Sample means ( y j ) 7.9 12.1 10.2 18.2 Y 12.1
n
k nj
SSR
SST Yij Y
2
nj

SS j Yij Y j
2
16.0 31.94 12.48 22.64 j1 i1

i1 258.52
k
SSE SS j 83.14
Pooled SS
j 1
 ANOVA technique
- variation between / within treatments -

2 kinds of variation:
Random variation within treatments - affects the
width of each distribution = type I
Systematic variation between treatments - causes
the i to be different from each other = type II

If H0 is true:
We have a single distribution - type I
We can estimate the variance of data ( ) by ignoring
2

the treatments - treating all observations as single
sample

 ANOVA technique
- variation between / within treatments -

How between - treatment & within - treatment

add to the total sum of squares ?

Theorem:

k n k k n

y
y n y i y y ij y i
2 2
ij
i1 j1 i1 i1 j1

SST SSA SSE

total sum of between within treatments
squares treatments sum of squares
sum of squares (error SS)
 ANOVA technique
- One-way ANOVA -

One-Way ANOVA:
variation between - treatment & within - treatment

Treatment / Variety ( j ) 1 2 3 4

Yields 10.1 11.4 13 14.4

( yij ) 8.9 16.4 8.2 19.4
4.7 8.5 9.4 20.80

within-treatment

between-treatments

k = no. of treatments = 4
n = no. of samples for each treatment = 3
 ANOVA technique
- ANOVA table / equal sample sizes -

H0 1 ... k
s12 reject H0
f f (k 1,k(n 1))
f 2
H1 i j s if

Degrees of Mean
Source of Sum of Squares f
variation freedom square
SSA
Between SSA k 1 s12
s12
k 1
treatments f 2
Within SSE s
treatments SSE k(n 1) s2
(error)
k(n 1)

Total SST nk 1

 ANOVA technique
- ANOVA table / unequal sample sizes -
k ni n

SST y ij y i SSA ni y i y
2 2

i1 j1 i1
k
SSE SST SSA n ni
i 1

Source of
Sum of Squares Degrees of Mean f
variation freedom square
SSA
Between SSA k 1 s12
s12
k 1
treatments f 2
Within SSE s
treatments SSE n k s
2

(error)
n k

Total SST n 1

 ANOVA technique
- One-Way ANOVA calculation-

Steps:
1. Find the treatment totals, square them, divide by nj and sum to get:

y 2 j
k
B
j 1 n j

2. Find the grand total, square it, divide by n and to obtain:

y 2
C
n
3. Square each observation and sum over the whole data:
k nj
A yij2
j 1 i 1
 ANOVA technique
- One-Way ANOVA calculation -
4. Complete the ANOVA table:

Source of Sum of Squares Degrees of Mean f

variation freedom square

SSA SST SSE SSA

Between k 1 s12
s12
k 1
treatments f 2
Within SSE s
treatments SSE k(n 1) s
2

k(n 1)
(error)

Total SST nk 1

H0 1 ... k s12 reject H0
f 2 f f (k 1,k(n 1))
H1 i j s if
 ANOVA technique
- One-Way ANOVA example-

1. Find the treatment totals, square them, divide by nj and sum to get:

Treatment 1 2 3 4
i 10.1 11.4 13.0 14.4
j 8.9 16.4 8.2 19.4
Yields ( yij )
4.7 8.5 9.4 20.8
Treatment Totals ( y j ) 23.7 36.3 30.6 54.6
Sample sizes ( n j ) 3 3 3 3

y 2 j (23.7) 2 (36.3) 2 (30.6) 2 (54.6) 2

k
B
j 1 n j 3 3 3 3
B 1932 .30
 ANOVA technique
- One-Way ANOVA example-

2. Find the grand total, square it, divide by n and to obtain:

Treatment 1 2 3 4
i 10.1 11.4 13.0 14.4
j 8.9 16.4 8.2 19.4
Yields ( yij )
4.7 8.5 9.4 20.8
k nj
Treatment Totals ( y j ) 23.7 36.3 30.6 54.6 Y Yij 145.2
j 1 i 1

Sample sizes ( n j ) 3 3 3 3 n 12

23.7 36.3 30.6 54.6 145.2

2 2 2
y
C
n 12 12
C 145.2
 ANOVA technique
- One-Way ANOVA example -

3. Square each observation and sum over the whole data:

Treatment 1 2 3 4
10.1 11.4 13.0 14.4
Yields ( yij )
8.9 16.4 8.2 19.4
4.7 8.5 9.4 20.8

k nj
A yij2 (10.1) 2 (8.9) 2 (4.7) 2
j 1 i 1
(11.4)2 (16.4)2 (8.5)2
(13.0)2 (8.2)2 (9.4)2
(14.4)2 (19.4)2 (20.8)2
k nj
A yij2 2015 .44
j 1 i 1
 ANOVA technique
- One-Way ANOVA example -
k
4. Complete the ANOVA table: n

Source of Sum of Squares Degrees of Mean f

variation freedom square
Between SSA B C SSA
treatments 175.38 k 1 3 s12
k 1
58.46
s12
f 2 5.63
Within SSE A B k (n 1) 8 SSE
s
treatments s2 10.39
(Error) 83.14 k (n 1)

SST A C
Total
258.52 nk 1 11

H0 1 ... k
s12 reject H0
f f (k 1,k(n 1))
f 2
H1 i j s if

 ANOVA technique
- One-Way ANOVA example -

k
for treatments
n

f (k 1, k (n 1))
0.05 f 0.05 (3,8) 4.07
k 4 1 3
n3 2 8
f 5.63 f0.05 (3,8)

Reject H0 Significant differences between

treatments
 ANOVA technique
- Two-way ANOVA -

Addition to One-Way ANOVA: variation between the blocks

Treatment / Variety ( j ) 1 2 3 4

Yields 10.1 11.4 13 14.4

( yij ) 8.9 16.4 8.2 19.4
4.7 8.5 9.4 20.80

block

no. of blocks = 3
 ANOVA technique
- Two-Way ANOVA calculation-
Steps:
1. Find the treatment totals, square them, divide by nj and sum to get:
y 2 j
k
B
j 1 n j

2. Find the grand total, square it, divide by n and to obtain:

y 2
C
n
3. Find the block totals, y i square them, divide by n i and sum to get:
nj
y 2 i
D
i 1 ni

4. Square each observation and sum over the whole data:
k nj
A yij2
j 1 i 1
 ANOVA technique
- Two-Way ANOVA calculation-
5. Complete the ANOVA table:

Source of Sum of Squares Degrees of Mean f

variation freedom square
Treatments s12
SSA B C k 1
SSA
s f 2
2

k 1
1
s
Blocks SSB
SSB D C n 1 sB 2 sB2
n 1
fB 2
s
Error SSE SST (n 1)(k 1)
s2
SSE
SSA
SSB (n 1)(k 1)

Total SST A C nk 1


treatments
H0 1 ... k f f (kt 1, (kt 1)(nt 1))
H1 i j blocks
fB f (kB 1, (kB 1)(nB 1))
 ANOVA technique
- Two-Way ANOVA example-

1. Find the treatment totals, square them, divide by nj and sum to get:

Treatment 1 2 3 4
i 10.1 11.4 13.0 14.4
j 8.9 16.4 8.2 19.4
Yields ( yij )
4.7 8.5 9.4 20.8
Treatment Totals ( y j ) 23.7 36.3 30.6 54.6
Sample sizes ( n j ) 3 3 3 3

y 2 j (23.7) 2 (36.3) 2 (30.6) 2 (54.6) 2

k
B
j 1 n j 3 3 3 3

B 1932 .30
 ANOVA technique
- Two-Way ANOVA example-

2. Find the grand total, square it, divide by n and to obtain:

Treatment 1 2 3 4
i 10.1 11.4 13.0 14.4
j 8.9 16.4 8.2 19.4
Yields ( yij )
4.7 8.5 9.4 20.8
k nj
Treatment Totals ( y j ) 23.7 36.3 30.6 54.6 Y Yij 145.2
j 1 i 1

Sample sizes ( n j ) 3 3 3 3 n 12

y 2 23.7 36.3 30.6 54.6 145.2

2 2
C
n 12 12
C 145.2
 ANOVA technique
- Two-Way ANOVA example-

3. Find the block totals, y i square them, divide by n i and sum to get:

Treatment 1 2 3 4 Block Totals ( yi )

i 10.1 11.4 13
.0 14.4 48.9
j 8.9 16.4 8.2 19.4 52.9
Yields ( yij )
4.7 8.5 9.4 20.8 43.4
Treatment Totals ( y j ) 23.7 36.3 30.6 54.6
Sample sizes ( n j ) 3 3 3 3

nj
y 2i (48.9) 2 (52.9) 2 (43.4) 2
D
i 1 ni 4 4 4

D 1768 .30
 ANOVA technique
- Two-Way ANOVA example -
4. Square each observation and sum over the whole data:

Treatment 1 2 3 4
10.1 11.4 13.0 14.4
Yields ( yij )
8.9 16.4 8.2 19.4
4.7 8.5 9.4 20.8

k nj
A yij2 (10.1) 2 (8.9) 2 (4.7) 2
j 1 i 1
(11.4)2 (16.4)2 (8.5)2
(13.0)2 (8.2)2 (9.4)2
(14.4)2 (19.4)2 (20.8)2
k nj
A yij2 2015 .44
j 1 i 1
 ANOVA technique
- Two-Way ANOVA example -
5. Complete the ANOVA table:

Source of Sum of Squares Degrees of Mean f

variation freedom square
SSA B C
k 1 3
SSA s12
Treatments s
2
58.46 f 2 4.89
175.38 1
k 1 s
Blocks SSB D C SSB sB2
n 1 2 sB
2
5.69 f B 2 0.48
11.38 n 1 s
SSE SST (n 1)(k 1) 2
Error
SSE
11.96
SSA SSB 71.76 6
s
(n 1)(k 1)
SST A C
Total
258.52 nk 1

H0 1 ... k treatments
f f (kt 1, (kt 1)(nt 1))
H1 i j blocks
fB f (kB 1, (kB 1)(nB 1))
 ANOVA technique
- Two-Way ANOVA example -
kt nB
for treatments for blocks
nt kB
f (kt 1, (kt 1)(nt 1)) f (kB 1, (kB 1)(nB 1))
0.05 0.05
kt 4 1 3 kB 3 1 2
nt 3 2 6 nB 4 2 6

f 0.05 (3,6) 4.76 f 0.05 (2,6) 5.14

f 4.89 f0.05 (3,6) f B 0.48 f 0.05 (2,6)

Significant differences between No significant differences

treatments between blocks
 ANOVA technique
- example Excel-
10.1 11.4 13 14.4
8.9 16.4 8.2 19.4
4.7 8.5 9.4 20.80

Tools
Data analysis

Anova: Single Factor

SUMMARY
Groups Count Sum Average Variance
Column 1 3 23.7 7.9 8.04
Column 2 3 36.3 12.1 15.97
Column 3 3 30.6 10.2 6.24
Column 4 3 54.6 18.2 11.32

ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 175.38 3 58.46 5.62521 0.022673 4.06618
Within Groups 83.14 8 10.3925

Total 258.52 11
Q & A session

Have Fun and Good luck

with ANOVA analysis!