Explaining the Saddlepoint Approximation
Constantino GOUTIS and George CASELLA
Saddlepoint approximations are powerful tools for obtain-
ing accurate expressions for densities and distribution func-
tions. We give an elementary motivation and explanation of
approximation techniques, starting with Taylor series ex-
pansions and progressing to the Laplace approximation of
integrals. These approximations are illustrated with exam-
ples of the convolution of simple densities. We then turn to
the saddlepoint approximation and, using both the Fourier
inversion formula and Edgeworth expansions, we derive the
saddlepoint approximation to the density of a single ran-
dom variable. We next approximate the density of the sam-
ple mean of iid random variables, and also demonstrate the
technique for approximating the density of a maximum like-
lihood estimator in exponential families.
KEY WORDS: Edgeworth expansions; Fourier trans-
form; Laplace method; Maximum likelihood estimators;
Moment-generating functions; Taylor series.
1. INTRODUCTION
The saddlepoint approximation has been a valuable tool
in asymptotic analysis. Various techniques of accurate ap-
proximations, relying on it in one way or another, have been
developed since the seminal article by Daniels (1954). Reid
(1988, 1991) gave a comprehensive review of the applica-
tions and a broad coverage of the relevant literature.
The number of applications of the saddlepoint approxi-
mation is quite impressive, as warrants this extremely pow-
erful approximation (see Section 5 for a partial list). Typ-
ically, derivations and implementations of saddlepoint ap-
proximations rely on tools such as exponential tilting, Edge-
worth expansions, Hermite polynomials, complex integra-
tion, and other advanced notions. Although these are impor-
tant tools for researchers in the area, they may obscure the
fundamental idea of the saddlepoint approximation. A goal
of this article is to illustrate that there is a simple basic idea
behind this useful technique. Namely, write the quantity one
wishes to approximate as an integral, expand the integrand
with respect to the dummy variable of integration, keep the
George Casella is Liberty Hyde Bailey Professor of Biological Statistics,
Department of Biometrics, Cornell University, 434 Warren Hall, Ithaca,
NY 14853. The authors thank Luis Tenorio for useful discussions, and
the reviewers for providing detailed comments on earlier versions of this
article, which resulted in a much improved presentation. This research was
supported by NSF Grants DMS 9305547 and DMS 9625440, and this is
paper BU-131 1-M in the Biometrics Unit, Cornell University, Ithaca, NY
14853. The original version of this article was written in December 1995,
before the tragic death of Costas Goutis in July 1996.
216 The Amnerican Statistician, Autguist 1999, Vol. 53, No. 3 (?) 1999 Americanl Statistical Association
This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
first few terms and integrate. The integral can be over the
complex plane, corresponding to the inversion formula of a
Fourier transform, but this is a secondary point.
We start with an elementary motivation of the technique,
stressing familiar Taylor series expansions. At the begin-
ning we will somewhat ignore the statistical applications,
because saddlepoint approximations are general techniques,
and quite often references to random variables and distri-
butions may be more confusing than illuminating. Once
the approximation is developed, however, we will examine
some statistical applications. Throughout the article, we as-
sume that the functions are as regular as needed. In other
words, when we write a derivative or an integral, we as-
sume that they exist. Furthermore, we develop the methods
in the univariate case. This is almost without loss of gen-
erality as the multivariate case is essentially the same but
with a somewhat more complicated notation.
To keep the technical level reasonable, we avoid any rig-
orous asymptotic analysis, but a few remarks are in or-
der. The accuracy of an approximation is assessed by ex-
amining the size of the error of approximation. We use
the notation 0(l) which denotes a function that satisfies
1imt<O tO (') = constant, where for a random sample of
size n, standard techniques typically give approximations
of order 0( ). The saddlepoint can improve this to 0( n)
and even 0(n-3/2) in some circumstances. Unfortunately,
calculation of these error terms often requires detailed tech-
nical arguments, and we do not include them here (see Field
and Ronchetti 1990 or Kolassa 1994). This omission does
not speak to the importance of such calculations. Indeed, in
application, the size of the approximation error is perhaps
the most important concern, and this error must be assessed
either through analytical or numerical means.
In Section 2 we introduce approximation techniques from
the point of Taylor series and Laplace approximations, and
give some examples. Section 3 is an attempt to explain the
original derivation of the saddlepoint, which has its roots
in Fourier transforms and complex analysis, along with an-
other derivation based on Edgeworth expansions. Those un-
willing to wade through these derivations need only look at
formula (25), which gives the formula for the density ap-
proximation of a sample mean. Section 4 treats the case of
the MLE in exponential families, with the important for-
mulas being (34) and (36). Section 5 contains a short dis-
cussion.
2. FIRST EXPANSIONS
We begin by looking at some basic principles of approx-
imation, using the familiar tool of the Taylor expansion. As
we will see, the underlying strategy of this approximation
carries through to more sophisticated approximations. Note,
however, that the approximations in this section can have
large errors. As we are working with densities of single
random variables rather than means, the order of the error
can be expected to be about 0(1).
2.1 From Taylor to Laplace
Perhaps the simplest way to approximate a positive func-
tion f(x) is to use the first few terms of its Taylor series
expansion. We will use that idea, but not for f(x) itself but
rather for h (x) log f(x). Writing f(x) = exph( x) and
choosing xo as the point to expand around, we obtain
f (x) exp {h(xo) + (x - xo)h (xo)
+ (X tXo)2 h//} (1)
The above approximation simplifies if we choose xo
where h'(si) = 0. The second term disappears and we have
f(x) exp {h(G) + 2 hl} (2)
The approximation (2) will be exact if h(x) is a quadratic
function. If not, obviously for an x far from s~, the omitted
terms of order (x - s)3 and higher will be important and
the approximation will not be good.
Although the approximation (2) may be useful in its own
right, it also can be used for computing integrals of positive
functions, such as f f(x)dx. Expanding the integrand as in
(2), we obtain
If(x)dx Jexp{hG)+ ( j) h"(xI)} dx. (3)
If iz is a maximum, h"I(,) is negative and the right side
of (3) can be explicitly computed by recognizing that the
kernel of the integral is the same as the kernel of a normal
density with mean iz and variance -1/h" (l). Hence
J f(x) dx exp {hQI%)} (-h"(x~)) 1/ (4)
The above technique is called the Laplace approximation,
and as its name suggests, it has been known for ages. It
implicitly requires that the integral is computed over the
whole line, but it may be sufficiently accurate as long as
the mass of the approximating function is within the limits
of the integration.
The next, perhaps most natural step is to combine the
two methods (2) and (4)-that is, to try to approximate a
function by a Laplace type approximation of an integral.
We first write the function f as
f (x) Jm(x,t) dt, (5)
for some positive m(x, t). This is always possible by con-
renormalize
sidering, for example, m(p, t) f()mO(t), where mO (t) is
a function integrating to one, but the latter representation
This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
is not particularly useful or illuminating. (We will later see
a number of cases where the representation (5) arises fairly
naturally.)
By defining k(x, t) = log m(x, t) we consider the Laplace
approximation of the integral of exp k(x, t) with respect to
the variable t. For any fixed x, we write
f(x) Jexp{k(x,t(x))+ -(2 k(t) ( dt
( 1/2
=exp {k (x, i(x))}-f2)
- ~ ~~~~~~~~~,2k (x, t)(6 (6)
at2 t(x)
where, for each x, i(x) satisfies &k(x,t)/0t = 0 and
02k(x, t)/&t2 < 0, and hence maximizes k(x, t).
In the above expressions, the notation makes it explicit
that the maximum t depends on x. In that sense, (6) is a
set of integrated Taylor expansions, one for each x, as op-
posed to (2) which is a single series around iz. Because we
are continually recentering the approximation, there is hope
that it is more accurate than (2). However, there is a price
to be paid: if we want the values of f (x) at various points
x, we must compute i(x), k(x, i(x)), and &2k(x, t)/&t2 each
time afresh.
It is clear that (6) will be exact if k(x, t) is a quadratic
function of t for each x but, as we will see, there are other
cases where the approximation is also exact. However, the
choice of the form of k(x, t) usually is determined by other
considerations. Furthermore, it is worth noting that the ac-
curacy of (6) depends on the value of x where we want to
approximate f (x) because, in general, the omitted third and
higher order derivatives with respect to t depend on x.
We illustrate the foregoing approximations by deriving
the distribution of the sum of two random variables by ap-
proximating the convolution integral.
Example 1: Gamma Distribution. We first look at a sim-
ple, somewhat artificial example. Suppose that the function
we wish to approximate is a gamma density with shape pa-
rameter equal to 2co, (with co > 1) and scale parameter equal
to 1; that is,
X (X) = 1 x2c-1exp(-X) (7)
F(2oz)
for positive x. The Taylor series approximation is obtained
by applying (2). The maximum is achieved at s 2oc - 1
and the second derivative of the logarithm of (7) evaluated
at iz is equal to -1/(2c - 1). Hence
fx (2) ) exp (2ca-1) [log(2ca -1) -1]
9(9n -2 }. (8)
Recalling that f (x) is actually a density function, we can
the approximation by calculating the constant
of the right side of (8) 50 that f f (cc) dcc= 1. Doing this,
The American Statisticiani, August 1999, Vol. 53, No. 3 217
we obtain It is interesting to note that the convolution formula (10)
can be solved for the maximum in some generality. If we
write
[f [2-F(2oz
2oX1)] -1)]
- exp
1 {L <f(
2(2a c-)1-
J 1)k (9)
k(x, y) = log[g(x - y)g(y)] = log[g(x - y)] + log[g(y)],
a normal density with mean and variance both equal to 2oc -
1. then under mild regularity conditions, &k(x, y)/&y has a
We can probably write f(x) in the form (5) in several zero at y = x/2, and we can apply the convolution formula
ways, but a simple one is motivated from elementary dis- somewhat straightforwardly to other densities.
tribution theory. We know that the sum of two independent
Example 2: Student's t distribution. Let X be a random
F(a, 1) random variables is a F(2ca, 1) random variable, so
variable with a Student's t distribution with v degrees of
if g( ) is the F(ca, 1) density then f(x) is a convolution of
freedom (X tj). Then X has density
the form
x

f(x) g(x - y)g(y) dy, f(x) = C where C =- 2

exp(?X)( y)ya-y1 dy
exp ( 'x) X exp (a - 1) [log(x - y) + log y]} dy.
(10)
The exponent in the integrand is maximized for y = x/2
and its second derivative with respect to y is equal to - (ca -
complex, and y = x/2 is the maximum. Applying (6) we
{(X _ y)-2 + y-2}. Applying (6) we obtain
(I + X2/V) 2 r (2 ) 7
To approximate the density of the sum T =X1 + X2, where
X1 and X2 are independent t, random variables, we use (10)
with g(y) = 1 + y2/v and find the zeros of
k(x, y) = log (1 + (x -y)2/v) + log (1 + y2/V)
The three solutions of &k(x, y)/9y = 0 are y = x42 and
y = (x + vxc2- 4v)/2. For x2 < 4v the last two roots are
can approximate the distribution of T by

Jexp { (c- 1) [log(x -y) + log y] } dy

fT(X) C2 4wv [ 722 i]l [1 V 1 (12)
( T 1/2 2oa-1
To evaluate the approximation, we compare it to the exact
density of T for v = 9, which is
Substituting in (10), we see that the kernel of the approxi-
mating function is a gamma density with a shape parameter C25w 9
equal to 2c - 1 and scale parameter equal to one. Thus, if fT JT~X} 64 - 9
(X) = C64
we renormalize it so that it integrates to one we obtain the
622336 + 56576 A- + 48496 A- + 272 A + 7x94 (1
F(2ca, 1) density exactly. X (x2/9 A- 4)9 .(3

(The exact density of the sum of two iid t,

exact was found with Roger Berger during a fit of Mathemnatica-
--- approximation induced frenzy.)
(212 (shr drenormnaslzed
Figure 1 compares the approximation (12), and its renor-
malized version, with the exact density. The agreement is
quite good.

2.2 Approximating Marginals

Another application, where a function is naturally repre-

sented by an integral, arises quite naturally in the calcula-
/V
tion of marginal densities. For example, if (X, Y) f(x, y),
then by a direct application of (6) the marginal density of
X, fx (x) can be approximated by

fx (x) Jf (c Y) dy

apoxiain are vitull the sam in th tails.

where y - y(c) satisfies & logf (c,y)/&y - 0 and is a local
maximum. It is easy to see that by repeating this approxima-
tion, we can also marginalize higher dimensional densities.

218 General

This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
 Formula (14) can also be applied to integrated likeli-
hoods, a desirable enterprise in the presence of nuisance
parameters. Given a likelihood function L(O, A x), where 0
is the parameter of interest, A is a nuisance parameter, and
x is the data, an integrated likelihood LI for 0 is obtained
by integrating out A. This leads to the approximation
Li(O|x) JL(0, Alx) dA
(92log L(0, A x) ~-1/2
L(, Ax) y )A2 ) (15)
This approximation is the Cox and Reid (1987) approximate
conditional likelihood, as first noted by Sweeting (1987),
and can be considered a version of the modified profile like-
lihood of Barndorff-Nielsen (1983). The factor that is "ad-
justing" the likelihood is the observed Fisher information.
See Barndorff-Nielsen (1988) for an alternate derivation and
further discussion.
3. THE REAL THING
The examples in the previous section are, of course,
rather artificial. The function f(x) has a closed form and
there is no need to use any approximations. Indeed, we
do not gain anything as the approximating functions are
equally complicated as the original one. As one might ex-
pect, this is not always the case. In this section we look at a
number of important statistical problems where saddlepoint
approximations are useful.
The first statistical application of the saddlepoint approx-
imation was derived by Daniels (1954). He approached the
problem of finding a density approximation through the in-
version of a Fourier transform. Such an approach has the
advantage of automatically providing a function m(.) sat-
isfying (5), but also carries the disadvantage of making us
deal with complex integration. We begin with some details
about the inversion formula.
3.1 The Inversion Formula
We recall that for a density f(x), the moment generating
function (mgf) is defined as
r+0
ox(t) J exp(tx)f(x) dx, (16)
-00
provided that the integral is finite for t in some open neigh-
borhood of zero. (That is, there exists c > 0 for which
bx(t) < oc for -c < t < c.) From Xx(t), we can obtain
Viewed
f(x) by using the inversion formula (Feller 1971, chap. XV; as a point in the complex plane, t(x) is neither a
Billingsley 1995, sec. 26) maximum nor a minimum but a saddlepoint of Kx (t) -ttx, as
the function is constant in the imaginary direction and has
1 r+0 an extrema in the real direction. (See Field and Ronchetti
f (x) = 2 O/x (it) exp(- itx) dt
1990,
1 r+
- 2/ j exp {Kx(it) - itx} dt, (17)
where i -1/= and we have defined Kfx(t) =logbx (t).
These formulas are common in statistical contexts where
f(x) is a density and $x (it) is the characteristic function.
[Waller, Turnbull, and Hardin (1995) discussed exact nu-
This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
merical inversion of (17).] The function Kx (t) = log /x (t)
is also called the cumtulant generating function (cgf) of a
random variable X. (Mathematically, the cgf and mgf are
equivalent. The fact that the cgf generates the mean and
variance, instead of the uncentered moments, is statistically
more appealing. We can think of K" as a variance.) How-
ever, we do not have to think of a random variable at all,
and (17) is applicable even if f(x) is negative or does not
integrate to one. There is a complication in that we have
to deal with complex rather than real numbers, but (17) is
similar to (5) and this suggests that we can use the same
ideas as in the previous section.
We first make a change of variable t' = it. We can then
write (17) as
1 fT+i
f](X) _ exp{Kx(t) -tx} dt (18)
for T in a neighborhood of zero. It is a consequence of a
theorem from complex analysis (the closed curve theorem)
that the integral (18) is the same over all paths that are
parallel to the imaginary axis in a neighborhood of zero
where OX (t) exists. Thus, we are free to choose a value for
T over which to do the integration.
We take k(x, t) = Kx (t) - tx and, as in (6), we find the
point i(x) that satisfies
I> (t) = x. (19)
Expanding the exponent in (18) around i(x) we have
-Kx(t) - tx _Kx(t(x)) - F(x)x ? (- 2()) K>(t(x)).
2
(20)
We now substitute in (18) and integrate with respect to t
along the line parallel to the imaginary axis through the
point t(x); that is, we choose the point -r in the limits of
the integral to be t(x). To treat this maneuver with rigor re-
quires great care (as in Daniels 1954 or Field and Ronchetti
1990, chap. 3), but if we proceed informally (as if it were
a real integral) we again see that there is the kernel of a
normal density. Similar to (6) we obtain
1/2
fx(x) 2Y2 Kx' (i())) ex
(21)
sec. 3.2 for details.) Note that here this extrema
is a minimum, and to do the integration here we require
Kx(t(x)) > 0 (compare to (3), where we needed x to be a
maximum). [It follows from the Cauchy-Schwarz inequal-
ity that I?1j > 0. Daniels (1954, sec. 6) gave conditions
under which the equation K> (t) inx has a unique root.]
We also note that choosing 'r = (xn) in (18) is an applica-
tion of a method called steepest ascent. This approximation
The American1 Statistician, August 1999, Vol. 53, No. 3 219
method takes advantage of the fact that, since t(x) is an
extreme point, the function is falling away rapidly as we
move from this point. Thus, the influence on the integral of
neighboring points is diminished, making the approxima-
tion (21) seem more reasonable. (See Field and Ronchetti
1990, sec. 3.2, or Kolassa 1994, sec. 4.4, for details.)
Expression (21) is what is commonly thought of as the
saddlepoint approximation to a density. Its error of approx-
imation is much better than the Taylor series approximation
to a function. In classical, or "first-order" asymptotics, the
error terms usually decrease at the rate n1/2, for a sample
of size n. The saddlepoint is "second-order" asymptotics,
and can have error terms decrease as fast as n3/2, which
yields a big improvement in accuracy for small samples.
We return to this point in Section 3.2.
Example 3. Noncentral chi-squared. Hougaard (1988)
presented an interesting application of the saddlepoint ap-
proximation, of which the following is a special case. The
noncentral chi-squared density has no closed form, and is
usually written
xp/2?kle x/2 ke-A
kF F(p/2 + k)2P/2+k k! (22)
where p is the degrees of freedom and A is the noncentrality
parameter. The density is an infinite mixture of central chi-
squared densities, where the weights are Poisson probabili-
ties. It turns out that calculation of the moment-generating
function is simple, and it can be expressed in closed form
as
e2At/(1-2t)
ox(t) (1 - 2t)P/2 (23)
o - - - Exact
o --- saddlepoint
co - - renormalized
co
C)
-J~~~~~~
Figur 2.Eatdniyo0 7(oi ie oehrwt h ade
pitappoiain(hr ahs ndrnraie adeon p
prxmto0 ln ahs.Ternomlzdsdlpitapoia
tin6svrulytesm steeatdniy
6 o 5 lo 15 20 25 30 35 40 45 50
220 Genzeral
This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
Solving the saddlepoint equation & log Ox (t) /0t = x yiel
the saddlepoint
t(x) -p 2x- Ax (24)
4x
and applying (21) yields the approximate density. For p = 7
and A = 5, Figure 2 displays the saddlepoint, renormalized
saddlepoint, and exact densities. Here, the saddlepoint and
the renormalized saddlepoint are remarkably accurate.
3.1.1 Saddlepoints for Sums
A useful application of the saddlepoint is the approxima-
tion of the density of a sum or average. Perhaps the simplest
nontrivial example, which is also the oldest one (Daniels
1954), is the derivation of the density of a sample mean of
independent and identically distributed random variables.
The key here is that the moment generating function of a
sum of iid random variables can be easily computed from
the original moment generating function, and (21) can be
directly applied.
Consider X to be the sample mean of X1, X2,.. ,Xn,
iid random variables. Each Xi has a moment generating
function qx(t) and a cumulant generating function Kx (t).
An elementary statistical argument shows that the moment
generating function of X is OX(t) = bx(t/n)T and the cu-
mulant generating function is Kg (t) = nKx (t/n). A direct
application of (21) then gives
fx (X) 21rKx(i(x))
x exp {rn [Kx (i(z)) - (z)] }. (25)
The right side of expression (25) is the saddlepoint ap-
proximation to the density of X. Of course, there are several
loose ends in the derivation which should be formalized in
order to be legitimate. Daniels (1954, 1987) presented all
the details. Note the fact that we are dealing with densities,
and random variables enter only in the derivation of the
cumulant generating function Kx(t) from the cumulants
of the individual random variables. We can also appeal to
it to renormalize fx(x-) so that it integrates to 1, which
amounts to adjusting the constant (n/(2wF))1/2. This typi-
cally requires numerical integration.
The saddlepoint approximation can also be used on dis-
crete distributions, as the next example shows.
Example 4: Poisson Distribution. Let Xl... . ,Xn be iid
from the Poisson distribution with mean A. The cumulant
generating function of Xi is Kx (t) = A (exp(t) - 1) which
yields t(x) = log (x/A) as the saddlepoint. The formula (25)
can be used directly, but now the mean can take only values
x = r/n for integer r. Substituting we obtain
/\1/2
-x (X) ( r 2r ) exp {n ( - I - (log >) x] }
2 (1)n X- Ar+?/2
This amounts to replacing the factorial in the exact distri-
bution of x by Stirling's approximation.
 It is interesting to compare the approximation (25) with
the simpler one, obtained by expanding the integrand in the
inversion formula around 0 rather than around its maxi-
mum. If we keep the first two terms we obtain
fx(x) = J exp {nKx(it) - itx4 dt
2_ exp {n [Kx(0) + (Kx(0) - x)-
K/ (O) t2 ] }dt.
This corresponds to the inversion of a quadratic cumulant
function iKx (O)t-Kx (O)t2 /(2n), and it can be easily seen
that we have nothing more or less than the central limit the-
orem. So the gain in the accuracy of the saddlepoint approx-
imation over the central limit theorem can be attributed to
a more clever expansion of the logarithm of the integrand.
This expansion is around a value that depends on x, the
value at which we would like to approximate the density,
rather than around 0. The implementation of such recenter-
ing, and its advantages, will become more apparent in the
next section.
3.2 The Edgeworth Approach
Although the original saddlepoint derivation of Daniels
(1954) was based on inversion of the characteristic function,
there is an alternate derivation, that Reid (1988) called "a
more statistical version," based on Edgeworth expansions.
In many ways it is similar to the original Daniels deriva-
tion (as it involves inversion of an integral), but it has the
advantage helping us make our statements about the "order
of the approximation" somewhat clearer. Moreover, it pro-
vides a more precise picture of how the greater accuracy is
achieved.
An Edgeworth expansion of a distribution is accom-
plished by first expanding the cumulant generating func-
tion in a Taylor series around 0, then inverting (see Feller
1971, chap. XVI, or Stuart and Ord 1987, chap. 6 and sec.
11.13-11.16).
Let X1 X2,.. . Xn be iid with density f with mean and
variance and a2. A useful form of an Edgeworth expan-
sion was given by Hall (1992, equation 2.17), and can be
written
p (( - )< w)
w @ (w) + qf(w) (W-1) ? ()], (26)
where 4D and X are, respectively, the distribution and density
function of a standard normal and , = E(X1 - At)3 is the
skewness. For the Edgeworth expansion, the (n ) term is
of the form p(w)/n, where p is a polynomial. Because it
is multiplied by b(w), it follows that its derivatives (in w)
maintain the same order of approximation. Thus, (26) can
be differentiated to obtain a density approximation with the
same order of accuracy. If we do that, and then make the
transformation x =r ?w +A,u we obtain the approximation
mation, and what we needed to do to obtain that order. The
This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
to the density of X to be
fx(x)> =qnfjX At)
a _ a ~___ _
[ 6+ {u (( /
(27)
Ignoring the term in braces produces the usual normal ap-
proximation, which is thus accurate to O( j9. If we are
using (26) for values of x near ,u, then the value of the ex-
pression in braces is close to zero, and the approximation
will then be accurate to O(n) The trick of the saddlepoint
approximation is to make this always be the case.
3.2.1 A Family of Edgeworth Expansions
To make the Edgeworth expansion accurate to 0(n), we
use a family of densities so that, for each x, we can choose
a density from the family to zero out the term in braces
in (26). One method of creating such a family is through
a technique known as exponential tilting (see Efron 1981;
Stuart and Ord 1987, sec. 11.13, or Reid 1988.) Starting
with a density f of interest, we create the family
.F= {f(. I): f(x 7) = exp{Tx -K(-)If(x),
where K(r) is the cumulant generating function of f. It
immediately follows that if Xl, X2... Xn are iid from
f(x r), the density of X is
fx_(x l7) = exp{n[7x - K(T)]}fx(x), (28)
where fx (x) is the density of the mean of an iid sample
from f. (This can be verified by calculating the mgf of
an iid sample from f(x Ir), and then directly calculating
the mgf of fx(xI -r).) Let Atu, a2, and /,, denote the mean,
variance, and skewness of f(. -r), respectively, and apply
(27) to f_(x (Ir) to obtain a family of Edgeworth expansions
for fx (x) given by
fx(x) = exp-n[r7x - K(T-)]} ??$ ( jL)
(29)
Now the parameter -r is free for us to choose in (28) and,
given x, we choose T so that utr = x. Recalling that K(T)
is the cumulant generating function, we can equivalently
choose -r so that K'(r) = x, the familiar saddlepoint equa-
tion. Denoting this value by '~, we get the approximation
fx(x) =exp{-n[fx-K(l)]}j -(O) 1?+O (I)].
(30)
Since op,= K"(T), we see that (30) is equivalent to (25),
the saddlepoint approximation.
However, we now clearly see the order of the approxi-
The Americanz Statistician, August 1999, Vol. 53, No. 3 221
saddlepoint improves on the usual normal approximation by eralizations). Consider Xl, ... , X , independent ran-
eliminating the skewness term in the asymptotic expansion dom variables with density
of the density. And this is accomplished by using a new ap-
proximating density for each value of x. So this is similar f (xIO) = exp {Os(x) - K(O) - d(x)}. (31)
in flavor to the derivation in Section 2.1.
A version of the sufficient statistic is S - j> s(Xi) which
We have only shown that the order of the approximation
has a density
is Q(n-r/2), not the 0(n-3/2) that is often claimed. This
better error rate is obtained by renormalizing (30) so that it f(sJ0) = exp {Os - nK(O) - h(s)}. (32)
integrates to one. Details of this are contained in Daniels'
From sufficiency considerations, we need to consider only
original 1954 paper, Field and Ronchetti (1990, chap. 3),
f(s 0). We also realize that there is no need to apply the
or Kolassa (1994, chap. 4). We omit them here, but note
saddlepoint approximation to the entire density (32), but
that such renormalization may be quite computer intensive;
only to the function exp {-h(s)}, as the part of (32) that
see, for example, the discussion in Section 3.3 of Field and
involves 0 is relatively simple.
Ronchetti (1990).
Next, note that because f(s 0) is a density, it integrates
Last, there is an interesting analogy between the exponen-
to one, and hence by integrating and rearranging (32) it
tial tilting derivation and the inversion formula derivation
follows that
of the saddlepoint approximation. Here, by using a fam-
ily of densities, for any value x we were able to select the
exp {nK(0)} = Jexp (Os) exp{- h(s)} ds. (33)
member of the family with the value of ,uM in (28) that ze-
roed out a term in the expansion. In the inversion formula
The right side is exactly (16) with 0 instead of t and s
derivation of Section 3.1, the complex integration resulted
instead of x, so the cgf of exp {-h(s)} is nK(0). (Here,
in a family of paths over which we could integrate. There,
we have to think of 0 as a dummy variable rather than a
we chose the path that also zeroed out a term in the ex-
parameter of a distribution.) Hence, we can apply the ap-
pansion (20). Thus, either derivation results in us having an
proximation (21) directly to exp {-h(s)}.
extra parameter that we can control to help produce a good
Alternatively, we can realize that (32) is in the form of
approximation.
an "exponential tilt" of exp {-h(s)}, so the method of Sec-
tion 3.2.1 can be applied. In either case, we obtain that the
4. MAXIMUM LIKELIHOOD ESTIMATION
density of f (s I0) is approximated by

The saddlepoint approximation is useful in obtaining ap- f(slO) ~ 1

proximations to the density of the maximum likelihood [27rn K" (F(s))] 1/2
estimator (MLE), particularly in exponential families (see
X exp { [O - (s)] s -n [K(0) -K (ts) ,(34)
Daniels 1983 or Field and Ronchetti 1990, chap. 4, for gen-
where t(s) solves the equation

nK'(t) = s. (35)

Thus, the approximate density of the sum is obtained al-

0 ~~~~~~~Exact most automatically from K(O), the cgf of an individual ob-
---saddlepoint
servation. By using an approximation for h(s) we avoid the
- - Prenormalized
possibly difficult job of computing it.
Now, it seems almost an accident that if we take the
derivative of the log-likelihood corresponding to the den-
sity of the sufficient statistic s and set it equal to zero, we
0)
obtain (35), and, hence, t(s) is the maximum likelihood esti-
mate. Equation (35) suggests that we can obtain the density
of the maximum likelihood estimate by a transformation of
f(sIO). Since t and s are related by s = nK'(t), the Jaco-
bian of the transformation is nIK"(i). Writing s as s(t), a
o, , , , _

function of F, we obtain

Fiuec xc estyo h L rmaPreodsrbtoJae

on=10obevtos(oilietoehrwtthsadeinap -nvK"(t 1/2
f(F0o) _
prxmtod sotdse)adrnraie adeon prxmto
?; 10 20 30 40 50 60 70 B0
(ln0ahs.Ternraie adeon prxmto seat
x
X exp l(ea- hwts(h) -rn (K()-bK(t)) a (36)

222 General

This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
parameter 0 enter the formula. Though the approximation
is quite accurate, it typically requires numerical methods to
compute s(i), K"(t) and the exact normalizing constant.
As it turns out, the renormalized (36) is exact in many of
the simple cases that we examined. [Daniels (1980) showed
that the three cases where the saddlepoint approximation to
the density of a sample mean or sum is exact are the gamma,
the normal, and the inverse Gaussian distributions.] To take
full advantage of (36) requires more complicated settings
than we will look at here, so we content ourselves with a
simple, exact example.
Example 5: Pareto Distribution. Let Xl,... X, be iid
from the Pareto distribution with known lower limit ae. The
density is
f (xJ13) = B0+1) X > at, (37)
a member of the exponential family. From (32) we see that
s = - Z log xi, K(,3) - log (,j3O), and the saddlepointvery
is accurate approximation that only requires the evalu-
given byt= s+n-log(o) The saddlepoint approximation is
ation of one saddlepoint, and no integration. It is derived
straightforward to compute and, from (36), the density of
,3, the maximum likelihood estimator of ,3, is approximated
tion; see the discussions by Field and Ronchetti (1990,
by
f(13113) I[n] ( e3) T(1//)1 (38)
Figure 3 shows this approximation and its renormalized ver-
sion, which is exact.
5. BEYOND BASICS
The technique of the saddlepoint has widespread appli-
cability, far beyond that illustrated here. In Section 4. we
applied the saddlepoint to MLEs in exponential families,
but more general classes of estimators can be handled. For
example, approximations to the density of multivariate M-
estimators can be obtained (Field 1982; Field and Ronchetti
1990, chap. 4). Moreover, the cgf itself may be estimated,
with application to L estimators (Easton and Ronchetti
1986) and general nonlinear statistics (Gatto and Ronchetti
1996).
Other applications include finite population models
(Wang 1993a), bootstrapping, and related confidence meth-
ods (Wang 1993b; Booth, Hall, and Wood 1992; DiCiccio,
Martin, and Young 1992), ANOVA and MANOVA (Butler,
Huzurbazar, and Booth 1992ab, 1993), prior distributions
(Eichenhaur-Herrmann and Ickstadt 1993), generalized lin-
ear models (Strawderman, Casella, and Wells 1996), expo-
nential linear models (Fraser et al. 1991), multiparameter
exponential families (Pierce and Peters 1992), studentized
means (Daniels and Young 1991), and 2 x 2 tables (Straw-
derman and Wells 1998).
There is another use of the saddlepoint approximation
that is, perhaps, even more important than the approxima-
tion of a density function. That is the use of the saddlepoint
to approximate the tail area of a distribution. From (25) we
This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
have the approximation
P(X >a) Jj (2rKxj(t()))1
x exp {n [Kx (t(x)) -t(x)x] } dx
= ' n )1/2 [KX,,(t) ]1/ 2
x exp {rn [Kx (t) - tKx (t)] } dt,
where we make the transformation Kx (t) = x and t(a)
satisfies Kx (i(a)) = a. This transformation was noted by
Daniels (1983), and allows the evaluation of the integral
with only one saddlepoint evaluation. However, one still
must do the integration, probably using a numerical method.
[See Robert and Casella (1999), sec. 6.3, for an illustration
of the use of the Metropolis algorithm to do such a calcu-
lation.]
Tail area approximations have seen much more develop-
ment. The work of Lugannini and Rice (1980) produced a
by further transformations of the saddlepoint approxima-
sec. 6.2) or Kolassa (1994, sec. 5.3). There are other ap-
proaches to tail area approximations; for example, the work
of Barndorff-Nielsen (1991), which takes advantage of an-
cillary statistics or the Bayes-based approximation of Di-
Ciccio and Martin (1993). Also, Wood, Booth, and Butler
(1993) gave generalizations of the Lugannini and Rice for-
mula.
[Received Decenmber 1995. Revised Junie 1998.]
REFERENCES
Barndorff-Nielsen, 0. (1983), "On a Formula for the Distribution of the
Maximum Likelihood Estimator," Biomnetrika, 70, 343-365.
(1988), Discussion of "Saddlepoint Methods and Statistical Infer-
ence," by N. Reid, Statistical Science, 3, 228-229.
(1991), "Modified Signed Log-Likelihood Ratio," Biomnetrika, 78,
557-563.
Barndorff-Nielsen, O., and Cox, D. R. (1994), Inferences and Asymptotics,
London: Chapman and Hall.
Billingsley, P. (1995), Probability and Measure (3rd ed.), New York: Wiley.
Booth, J. G., Hall, P., and Wood, A. (1992), "Bootstrap Estimation of
Conditional Distributions," The Annals of Statistics, 20, 1594-1610.
Butler, R. W., Huzurbazar, S., and Booth, J. G. (1992a), "Saddlepoint
Approximations for the Generalized Variance and Wilks' Statistic,"
Biometrika, 79, 157-169.
(1992b), "Saddlepoint Approximations for the Bartlett-Nonda-
Pillai Trace Statistic in Multivariate Analysis," Biomnetrika, 79, 705-515.
(1993), "Saddlepoint Approximations for Tests of Block Indepen-
dence, Sphericity and Equal Variances and Covariances," Journal of the
Royal Statistical Society, Ser. B, 55, 171-183.
Cox, D. R., and Reid, N. (1987), "Parameter Orthogonality and Approx-
imate Conditional Inference" (with discussion), Journial of the Royal
Statistical Society, Ser. B, 49, 1-39.
Daniels, H. E. (1954), "Saddlepoint Approximations in Statistics," Annals
of Mathematical Statistics, 25, 631-650.
(1980), "Exact Saddlepoint Approximations," Biometrika, 67, 59-
63.
(1983), "Saddlepoint Approximations for Estimating Equations,"
Biometrika; 70, 89-96.
The American Statistician, August 1999, Vol. 53, No. 3 223
 (1987), "Tail Probability Approximations," International Statistics
Review, 55, 37-48.
Daniels, H. E., and Young, G. A. (1991), "Saddlepoint Approximations for
the Studentized Mean," Biometrika, 78, 169-179.
DiCiccio, T. J., and Martin, M. A. (1993), "Simple Modifications for
Signed Roots of Likelihood Ratio Statistics," Journal of the Royal Sta-
tistical Society, Ser. B, 55, 305-316.
DiCiccio, T. J., Martin, M. A., and Young, G. A. (1992), "Fast and Accurate
Double Bootstrap Confidence Intervals," Biometrika, 79, 285-295.
Easton, G. S., and Ronchetti, E. (1986), "General Saddlepoint Approx-
imations With Application to L Statistics," Journal of the American
Statistical Association, 81, 420-430.
Efron, B. (1981), "Nonparametric Standard Errors and Confidence Inter-
vals" (with discussion), Canadian Journal of Statistics, 9, 139-172.
Eichenhaur-Herrmann, J., and Ickstadt, K. (1993), "A Saddlepoint Charac-
terization for Classes of Priors With Shape-Restricted Densities," Statis-
tics and Decisions, 11, 175-179.
Feller, W. (1971), An Introduction to Probability Theory and its Applica-
tions (Vol. II), New York: Wiley.
Field, C. (1982), "Small Sample Asymptotic Expansions for Multivariate
M Estimators," The Annals of Statistics, 10, 672-689.
Field, C., and Ronchetti, E. (1990), Small Sample Asymptotics, Hayward,
CA: Institute of Mathematical Statistics.
Fraser, D. A. S., Reid, N., and Wong, A. (1991), "Exponential Linear Mod-
els: A Two-Pass Procedure for Saddlepoint Approximation," Journal of
the Royal Statistical Society, Ser. B, 53, 483-492.
Gatto, R., and Ronchetti, E. (1996), "General Saddlepoint Approximations
of Marginal Densities and Tail Probabilities," Journal of the American
Statistical Association, 91, 666-673.
Hall, P. (1992), The Bootstrap and Edgeworth Expansion, New York:
Springer-Verlag.
Hougaard, P. (1988), Discussion of "Saddlepoint Methods and Statistical
Inference," by N. Reid, Statistical Science, 3, 230-231.
Kolassa, J. E. (1994), Series Approximation Methods in Sstatistics, New
York: Springer-Verlag.
Lugannani, R., and Rice, S. (1980), "Saddlepoint Approximation for the
224 General
This content downloaded from 39.50.186.67 on Sun, 01 Jul 2018 00:55:15 UTC
All use subject to http://about.jstor.org/terms
Distribution of the Sum of Independent Random Variables," Advances
in Applied Probability, 12, 475-490.
McCullagh, P. (1987), Tensor Methods in Statistics, London: Chapman and
Hall.
Pierce, D. A., and Peters, D. (1992), "Practical Use of Higher Order
Asymptotics for Multiparameter Exponential Families" (with discus-
sion), Journal of the Royal Statistical Society, Ser. B, 54, 701-737.
Reid, N. (1988), "Saddlepoint Methods and Statistical Inference" (with
discussion), Statistical Science, 3, 213-238.
(1991), "Approximations and Asymptotics," in Statistical Theory
and Models, Essays in Honor of D. R. Cox, London: Chapman and Hall,
pp. 287-334.
Robert, C. P., and Casella, G. (1999), Monte Carlo Statistical Methods,
New York: Springer-Verlag
Strawderman, R. W., Casella, G., and Wells, M. T. (1996), "Practical Small
Sample Asymptotics for Regression Problems," Jourinal of the American
Statistical Association, 91, 643-654.
Strawderman, R. W., and Wells, M. T. (1998), "Approximately Exact In-
ference for the Common Odds Ratio in 2 x 2 Tables" (with discussion),
Journal of the American Statistical Association, 93, 1294-1320.
Stuart, A., and Ord, J. K. (1987), Kendall' s Advanced Theory of Statistics
(Vol. I, 5th ed.), New York: Oxford University Press.
Sweeting, T. J. (1987), Discussion of "Saddlepoint Approximations in
Statistics," by D. R. Cox and N. Reid, Journal of the Royal Statistical
Society, Ser. B, 49, 20-21.
Waller, L. A., Turnbull, B. W., and Hardin, J. M. (1995), "Obtaining Distri-
bution Functions by Numerical Inversion of Characteristic Functions,"
The American Statistician, 49, 346-350.
Wang, S. (1993a), "Saddlepoint Expansions in Finite Population Prob-
lems," Biometrika, 80, 583-590.
(1993b), "Saddlepoint Methods for Bootstrap Confidence Bands in
Nonparametric Regression," Australian Journal of Statistics, 35, 93-101.
Wood, A. T. A., Booth, J. G., and Butler, R. W. (1993), "Saddlepoint Ap-
proximations to the CDF of Some Statistics With Nonnormal Limit
Distributions," Journal of the American Statistical Associationi, 88, 680-
686.