June 2010

Journal No. 3

Summary

Point of view

Lab news

Focus on a laboratory

Research

Methods

Agenda

Research for reference

The accuracy profile: a tool for choosing
an analytical method and evaluating
its validity
M. Laurentie, JM. Delmas. AFSSA, Fougres(France)
Laurentie M., Delmas JM. (2010). The accuracy profile, a tool for choosing an analytical
method and evaluating its validity, EuroReference, No. 3, ER03-10R01.
http://www.afssa.fr/euroreference/numero3/PN50I0.htm
Validation and determination of performance criteria are often
confused. A commission from the French Society of Pharmaceutical Science and Technology (SFSTP) has developed
a new tool for comparing domain of validation and domain of validity. This tool, called the accuracy profile, is
based on the use of the total error derived from performance criteria such as trueness and precision. This is a
decision tool as well as a diagnostic tool, as shown below in an example of application.

Introduction

guarantee its suitability for its intended routine purpose.

Validation of the method must make it possible to verify that
the performance criteria are in line with the methods objective.
In addition, if at the end of the study the method has not been
validated, it would be useful to be able to identify the sources
of error.
How is this question formalised in statistical terms? For the
analyst and the statistician, as well as for the customer, a good
method means that the result provided is not too far from the
true value, which can be written (eq. 1):

There is often confusion between validation of analytical

methods and of the performance criteria of an analytical
method. For example, in Clause 5.4.5.1 of ISO Standard
17025:2005, validation is defined as follows: Validation is the
confirmation by examination and the provision of objective
evidence that the particular specifications for an intended use
are fulfilled (Feinberg, 2009). There are many other definitions,
with regulatory or legislative texts having defined for each field
what is meant by validation of methods. The determination of
performance criteria is less clearly defined. Although the criteria
are generally universal and accepted (linearity, repeatability,
trueness, accuracy, etc.), the protocols or rules for validating
a method have not been clearly specified. The concept of
total error makes it possible to overlay a range defined by the
performance criteria onto an acceptance range determined by
regulatory specifications or laboratories.

Equation 1

x T

where T is the true value.

To ensure that the difference is acceptably small it must be
bounded by the acceptance limits denoted , which can be
written (eq. 2):

Equation 2

Objectives
The provision of objective evidence generally involves the use of
statistics. However, these statistics are not always used properly
because they are not discussed together ahead of time by the
analyst and the statistician. Different points of view have to be
addressed:
an analyst who knows from experience whether or not his
method is sound, and then sees it accepted or refused during
the statistical analysis;
the statistician who wants to ensure that the method is sound
from a statistical perspective;
the customer who wants a sound result and expects the
method to be guaranteed as such.
To reconcile these three points of view, the analyst needs a tool
which is easy to use, which can guarantee the results obtained,
and which is acceptable to the customer.
Based on this realisation, a commission from the French Society
of Pharmaceutical Science and Technology (SFSTP) has
developed a validation concept based on the total error and the
accuracy profile (Hubert et al., 2004; Hubert et al., 2007a; Hubert
et al., 2007b; Hubert et al., 2008; AFNOR 2010).

< x T <

x T<

This in turn leads to two distinct concepts: (i) a performance

acceptance limit, and (ii) a decision by the analyst to accept or
reject a method based on its performance.
Guaranteeing results means setting the expected measurement
risk beyond the acceptance limits. This risk must be determined
a priori and will result in a probability (eq. 3):

Equation 3

(x T< )

where is the proportion of measurements within the acceptance

limits .
Creating the experimental design for assessing performance
criteria usually involves repetition and several concentration
levels. The results must therefore be guaranteed for a range of
concentrations. To satisfy this condition, a confidence interval
must be established for the measurements expected for the
domain covered by the concentrations. This confidence interval
for measurements expected at the level is expressed by (eq. 4):

Equation 4

, { [x T< ]/ ,}

Accuracy profile
where E stands for the expected value calculated at the
moment the measurements are taken depending on the

Accuracy is the sum of trueness (statistical bias) and precision

(systematic error). A methods performance criteria do not

Point of view

Lab news

Focus on a laboratory

Research

estimators of bias ( ) and variance ( ) available to the analyst.

Figure 1 shows an accuracy profile.

Agenda

a function of the concentrations introduced for the calibration

standard range and the validation standard range.
4,000
Calibration standard

3,500

+ 25

3,000
2,500

Are

2,000
1,500

Validation standard

1,000

25

500
0
Domain of validity

Concentration

500

1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000

Concentration (ng/mL)

Domain of validation

Figure 2: Raw data observed. The results were obtained

for three days and for two repetitions per level. Note the matrix
effect between the calibration standards (blue line) and validation
standards (pink line).

Figure 1: The dashed lines delineate the acceptability range

at , here 25%, i.e. where the bias may vary between 75
and 125%. This interval represents the domain of validation.
The solid lines represent the tolerance interval or accuracy profile.
The intersection between the lines of the interval of acceptability
and the bounds of the tolerance interval delineate the domain
of validity. Within this domain, the method is capable of producing
a high and identifiable proportion of acceptable results.

The data were processed in three steps:

Analysis of raw data and plotting of the accuracy profile;
An analysis to determine the correction coefficient for the
matrix effect (i.e. the inverse of the rate of recovery);
Calculation of the accuracy profiles after correction of
concentrations.

This tool helps to determine whether or not the method is valid by

overlaying the desired domain of validation and the acceptable
domain of validity depending on the methods performance
criteria.
It can also be used as a diagnostic tool, as the following example
shows.

Analysis of raw data

The response function used to describe the relationship between
the concentrations and the response is a weighted quadratic
regression (weighting factor of type 1/X where X = introduced
concentration).
Figure 3 shows the accuracy profile obtained.

Application example: determination

of acrylamide in pig plasma
Acrylamide is a compound neoformed during the cooking
of certain foods. It results from a Maillard reaction by a
combination of sugar (i.e. glucose) and certain amino acids such
as asparagine. While many studies have documented levels of
acrylamide in foods, few have quantified its absorption after
ingestion. A pharmacokinetic study conducted in pigs should
determine the bioavailability of acrylamide. Initially, an analytical
method is needed to quantify the concentrations of acrylamide
in pig plasma. The method adopted for this assay is an HPLC
method coupled with mass spectrometry detection (MS). The
range studied varies from 10 to 5000 ng/mL, given the absence
of information on circulating plasma levels after ingestion of food
containing acrylamide.
Two ranges are prepared: a calibration standard range and
a range in pig plasma (the validation standard). 100 L of
saturated ZnSO4 solution, then 1,000 L of acetonitrile and
100 L of internal standard (acrylamide-d5) are added to 200 L
of plasma. After agitation and centrifugation the supernatant is
evaporated. The eluate is resuspended with 200 L of 0.01M
ammonium acetate (pH 6). The injection volume is 50 L.
The analytical conditions used are a chromatographic rate of
0.2 mL/min through a Hypercarb column (5 50-2 mm) and MS
detection of the molecular ion 72 of acrylamide. The calibration
standard range for acrylamide varies from 10 to 5,000 ng/mL
and consists of a 0.01M ammonium acetate solution restored to
pH 6 with formic acid.
The experimental design used requires three days, six levels
and two repetitions (362), or 36 tests for the calibration and
validation standards. Figure 2 shows the responses obtained as

Relative bias (%)

June 2010

Methods

Research for reference

Bias (%)

Journal No. 3

Summary

30
25
20
15
10
5
0
-5 0
- 10
- 15
- 20
- 25
- 30
- 35
- 40
- 45
- 50
- 55
- 60
- 65
- 70

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

5,000

Concentration (ng/mL)

Figure 3: Accuracy profile achieved with the raw data.

The results are obtained with a quadratic regression that is weighted
to model the response function.

An offset is observed between the accuracy profile and the

acceptability limits that were set at 25%. This offset is due to the
matrix effect. A correction factor must therefore be estimated to
take the bias of the method into account.
For this calculation an equation of type ax+b is used to plot
the theoretical values and the observed values. The slope of
the equation between the observed concentrations and the
introduced concentrations corresponds to the inverse of the
correction factor considered.
Figure 4 shows these calculations.

10

Point of view

Lab news

Focus on a laboratory

Research

Methods

Agenda

Observed concentration (ng/mL)

Research for reference

(25%) for the concentration level equal to 10 ng/mL.
The calculation of the lower and upper limits of quantitation gives
the respective values 14.05 ng/mL and 5000 ng/mL. The limit of
detection is estimated at 1.02 ng/mL. Table 1 summarises the
results obtained for all the concentration levels.
The validation shows, on the basis of a weighted quadratic
regression model, that the assay range is established between
14.05 and 5000 ng/mL. It also shows that the matrix effect is
systematic and that a correction factor of 1.457 must be applied
to correct it.
The accuracy profile is a diagnostic tool that enables the critical
point(s) of a method to be located. If the method is found to
be biased the accuracy profile enables a correction factor to
be validated. There may be strong variability in the method,
resulting in a tolerance interval in the profile that is greater than
the domain of validation. Breaking it down into repeatability and
intermediate precision enables the source of this variability to
be identified.

4,000

y = 0,686x + 1,983
R = 0,9982

3,500
3,000
2,500
2,000
1,500
1,000
500
0
- 500 0

1,000

2,000

3,000

4,000

5,000

6,000

Theoratical concentration (ng/mL)

Figure 4: Regression line between the introduced and observed

concentrations.

The equation for the line is:

(observed concentration) = 0.686 * (theoretical concentration)
+ 1.983.
The coefficient of determination (r2) is 0.9982.
The correction factor (Fc) to be applied is therefore:
Fc = 1
= 1 = 1,457
slope
0,686

References
AFNOR. 2010. Standard NF V03-110:2010. Protocole de caractrisation
en vue de la validation dune mthode danalyse quantitative par
construction du profil dexactitude [Protocol of characterization for the
validation of a quantitative method of analysis by construction of an
accuracy profile], AFNOR, Paris
Feinberg M. 2009. Labo-Stat, Guide De Validation Des Mthodes
Danalyse. Tec & Doc Lavoisier, Paris, France: 361 pp.
Hubert P, Nguyen-Huu J, Boulanger B, Chapuzet E, Chiap P, Cohen N,
Compagnon P, Dewe W, Feinberg M, Lallier M, Laurentie M, Mercier
N, Muzard G, Nivet C, Valat L. 2004. Harmonization of strategies for
the validation of quantitative analytical procedures. A SFTP Proposal
Part1. J. Pharma. Biomed. Anal., 36 (3): 579-586.
Hubert P, Nguyen-Huu J, Boulanger B, Chapuzet E, Chiap P, Cohen N,
Compagnon P, Dewe W, Feinberg M, Lallier M, Laurentie M, Mercier N,
Muzard G, Nivet C, Valat L, Rozet E. 2007a. Harmonization of strategies
for the validation of quantitative analytical procedures. A SFSTP
proposal Part II. J. Pharma. Biomed. Anal., 45 (1): 70-81.
Hubert P, Nguyen-Huu J, Boulanger B, Chapuzet E, Chiap P, Cohen N,
Compagnon P, Dewe W, Feinberg M, Lallier M, Laurentie M, Mercier N,
Muzard G, Nivet C, Valat L, Rozet E. 2007b. Harmonization of strategies
for the validation of quantitative analytical procedures A SFSTP proposal
Part III. J. Pharma. Biomed. Anal., 45 (1): 82-96.
Hubert P, Nguyen-Huu J, Boulanger B, Chapuzet E, Chiap P, Cohen N,
Compagnon P, Dewe W, Feinberg M, Lallier M, Laurentie M, Mercier N,
Muzard G, Nivet C, Valat L, Rozet E. 2008. Harmonization of strategies
for the validation of quantitative analytical procedures: A SFSTP
oroposal. Part IV. Examples Of Application. J. Pharma. Biomed. Anal.,
48 (3): 760-771.

A new calculation of the accuracy profile is performed, taking

the correction factor into account. The correction is applied to
the responses. Figure 5 shows the accuracy profile obtained.

Relative bias (%)

June 2010

Journal No. 3

Summary

30
25
20
15
10
5
0
-5 0
- 10
- 15
- 20
- 25
- 30
- 35

500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000

Concentration (ng/mL)

Figure 5: Accuracy profile achieved with corrected data.

The results are obtained with a 1/X type weighted quadratic
regression to model the response function.

The method is not validated for the studied domain, as the lower
bound of the accuracy profile is beyond the acceptance limit set

Table 1. Results of calculations of the accuracy profile after correction of raw data
Level of theoretical concentrations (ng/mL)
10
20
50
500
1,000
Lower
6.98
16.52
44.93
192.5
894.8
Limit of the -expectation tolerance interval
Upper
12.26
24.16
60.07
223.0
1,145
Lower
30.18
17.40
10.14
3.77
-10.52
Limit of the relative -expectation tolerance
interval (%)
Upper
22.59
20.82
20.15
11.50
14.48
Standard deviation of repeatability
0.61
0.43
1.62
5.78
32.26
Standard deviation of intermediate precision
0.99
1.31
2.81
6.32
48.01
Repeatability RSD (%)
6.08
2.14
3.25
2.89
3.23
Intermediate reliability RSD (%)
9.92
6.57
5.63
3.16
4.80
Inverse predicted concentration (ng/mL)
9.62
20.34
52.50
207.7
1,020
Absolute bias (ng/mL))
0.38
0.34
2.50
7.73
19.78
Relative bias (%)
3.80
1.71
5.00
3.87
1.98
Recovery (%)
96.20
101.7
105
103.9
102

11

5,000
4,482
5,515
10.35
10.30
136.7
199.2
2.73
3.98
4,999
1.35
0.03
99.97