AA – AGREEMENT ANALYSIS

Copyright © 2015 Francis Pitard Sampling Consultants LLC and HonuaTek LLC - All Rights Reserved
Paired estimates of a lot

2
Typical Uses
• External due diligence
– comparing independent estimates made between
buyers and sellers of a commodity
• Internal accounting
– mine and processing plant
– processing plant and smelter
• Comparing
– sampling systems
– laboratories / analytical techniques / CRM
– drilling techniques / core halves
3
Combines Techniques
• Measure correlation
between estimates
• Compare differences over
time and grade
• Investigate and characterize
variability over time
• Compare short-term trends
in estimates
• Identify statistically
significant bias between
estimates
4
Scatter Plot
• Initial understanding of the relationship
between the estimates.
• How well set A is a predictor for set B and
vice-versa (i.e. is relationship linear)
• Correlation coefficient (r)
• Percent of pairs having a suspicious
correlation, 𝑟′ = 100 ∙ 1 − 𝑟 2

5
Scatter Plot

6
Relative Difference
• Percent Relative Difference (PRD)
𝐴𝑖 −
𝑃𝑅𝐷𝑖 = 100 𝐵𝑖 𝑥 , 𝑖 = 1, 2, … , 𝑁
∙

• Mean of the PRD

• Moving average of the PRD
• How differences compare over time
• How differences compare over content

7
Relative Difference – chronologic

8
Relative Difference – by grade

9
CuSum
• Tabular CuSum
• Used in SPC to detect small persistent changes
in a process
• For agreement analysis we use it to compare
the accumulation of small non-random
changes on the estimates of A & B

10
CuSum (tabular)
Defined by:
𝐶0+ = 0
𝐶 + = 𝑚𝑎𝑥 0, 𝐶𝑖+−1 + 𝑥𝑖 𝜇0 + 𝐾
𝐶0− =
−0
− −
𝐶𝑖 = 𝑚𝑎𝑥 0, 𝐶𝑖 −1 − 𝑥𝑖 𝜇0 − 𝐾
+
where:
𝜇0 is the target value
K is the reference value (a.k.a. slack value)

11
CuSum (𝜇0 =1.0, k=0.1)

12
Variogram
• Provides information on the dynamics of the
process
• Provides information on the potential source
of differences between the estimates
• Parameters such as V(0) provide guidance for
other techniques

13
Variogram 𝑁−𝑗

1 2 𝑁
𝑉𝑗 = ෍ ℎ 𝑖 +𝑗 − ℎ𝑖 , 𝑗 = 1,2, … ,
2 𝑁− 2
𝑖 =1
𝑗

14
Variogram (absolute)

15
Bias testing: Pierre Gy’s approach
• Assume sampling bias is
not constant
• Consider risk from both a
producer and consumer
POV
• Can prove presence of
bias, not absence
• Nothing replaces correct,
well inspected and
maintained equipment
16
Background – Hypothesis Test
• population standard deviation (σ) - unknown
• sample standard deviation (s)
• population mean (µ)
• sample mean (xത )
• H0 (=) two-tailed
ഥ− μ
X
• t= s with n-1 degrees of freedom
n

17
Defining Risk (α)
• Risk of finding a difference when one does not
exist (a.k.a. seller’s risk)
• Positive test (Ho rejected)
– Serious Presumption – not to exceed 5%
– Practical Certainty – not to exceed 1%
– Certainty – limit case
• Negative test (Ho not rejected)
– Uncertainty

18
Preliminary Test - Definition
• Hypothesis: Ho ≡ D = 0
– D is the systematic difference of A & B
DN N
• Define the random variable: WN = SN tμ α
– DN is the estimate of D after a series of N trials.
n  1,2, ... N
DN  1
N d
n
n

– 𝑠𝑁2 is the unbiased estimate of σ2 after a series of

2

 (d  D )
1
N trials. s 
2
N
N 1 n
n
N

19
Interpreting Results

20
The Complementary Test
“Do not confuse the absence of proof of bias
with the proof of absence of bias; these are two
different conclusions.” – Pierre Gy

• β—the risk of not finding a difference when

one does exist (a.k.a. buyer’s risk)
• For small systematic differences (ɛ) the risk β
can be large (close to 1 – α).
21
Complementary Test - Definition
• Hypothesis: Ho′ ≡ D′ = 0
– D′ = D − Da
– D is the absolute value of the systematic
difference
– Da is the tolerated systematic difference of the
estimates of A & B
𝐷𝑁′ N
• Define the random variable: W′N = SN tμ α

22
Interpreting Results

23
Combining Results
W W’ Interpretation
> +1 > +1 With the given risk we can conclude that A is systematically higher than B, and
beyond the tolerated systematic difference.
> +1 < -1 With the given risk we can conclude that A is systematically higher than B, but
within the tolerated systematic difference.
> +1 [-1, +1] With the given risk we can conclude that A is systematically higher than B — but
we cannot determine if it is beyond the tolerated systematic difference.
< -1 > +1 With the given risk we can conclude that A is systematically lower than B, and
beyond the tolerated systematic difference.
< -1 < -1 With the given risk we can conclude that A is systematically lower than B, but
within the tolerated systematic difference.
< -1 [-1, +1] With the given risk we can conclude that A is systematically lower than B — but
we cannot determine if it is beyond the tolerated systematic difference.
[-1, +1] [-1, +1] With the given risk, we do not have enough samples to draw a conclusion.
[-1, +1] < -1 With the given risk we cannot conclude there exists a systematic difference
between A and B — if it does exist, it is less than the tolerated systematic
difference.
[-1, +1] > +1 The results are erroneous and no conclusion can be drawn from them.

24
Bias Test

25
Effect of Risk α

26
Effect of Random Error

27
Bias w/correction – memory effect

28
Impact of ɛ on β

29
Impact of α on β

30
Impact of Sample Size on β

31
Conclusions
• Practical, easy to interpret control chart
• Negatively influenced by:
– large bias corrections
– large random error
– small # of samples (caution when using < 30)
• Improvement on simple hypothesis testing
– Perform test in a continuous way
– Addresses both the buyer’s and seller’s risk

32
AGREEMENT ANALYSIS
SOFTWARE
Agreement Analysis-Internet Installation

• Launch Internet Explorer

• Go to: www.honuatek.com/software/agreementanalysis/install/publish.htm

34
Agreement Analysis-Internet Installation

• Copy-paste license key

c592e694-10ca-4409-976e-bb5db9610b27

35
Agreement Analysis-USB Drive Installation

Sampling

Software AA
Data
EMPV
Help

OGC Install
setup.exe
OSP License

• Copy-paste license key

c592e694-10ca-4409-976e-bb5db9610b27
Copy demo data to My Documents
Sampling

Software

AA
Data
Copy all files…
…
…
 Main Parts
Graph Viewing & Reporting

Workspace Management
*.s2.xml

Data Editing
Typical Workflow
Scatter • Simple linear correlation

Relative • Differences by time and grade

Difference

Cumulative Sum • Small persistent changes

• Components of variability
Variogram • Independence

Bias Test • Statistically significant differences

Support Info
Max Pitard
HonuaTek LLC
P.O. Box 151057
Lakewood, CO 80215-9057
U.S.A.
phone: +1.303.517.2114
email: mpitard@honuatek.com
web: www.honuatek.com