CHAPTER 7.

ERGODIC THEOREMS

162
(d) for any F B

Z
m(F ) =

(e) and if f L1 (m), then so is

Z
px (F )dm(x) =

px (F )dm(x),

f dpx and
Z
Z Z
f dm = ( f dpx )dm(x).

(7.12)

(7.13)

(g) In addition, given any f L1 (m), then

lim n1

n1
X

f (T i x) = Epx f, m a.e. and m a.e.

(7.14)

i=0

Comment: The final result is extremely important for both mathematics and intuition: Say that
a system source is AMS and that f is integrable with respect to the stationary mean; e.g., f is
measurable and bounded and hence integrable with respect to any measure. Then with probability
one a point will be produced such that the limiting sample average exists and equals the expectation
of the measurement with respect to the stationary and ergodic measure induced by the point. Thus
if a system is AMS, then the limiting sample averages will behave as if they were in fact produced by
a stationary and ergodic system. These ergodic components are determined by the points themselves
and not the underlying measure; that is, the same collection works for all AMS measures.
Proof: We begin the proof by observing that the definition of Px implies that
PT x (F ) = 1F (T x) = 1F (x) = Px (F ); all F F; all x G(F)

(7.15)

since limiting relative frequencies are invariant. Since PT x and Px agree on generating field events,
however, they must be identical. Since we will select the px to be the Px on a subset of G(F), this

proves (b) for x E.

Next observe that for any x G(F) and F F
Px (F ) = 1F (x) = lim

n1
X

1F (T i x) = lim

i=0

n1
X

1T 1 F (T i x) = 1T 1 F (x) = Px (T 1 F ).

i=0

Thus for x G(F) the measure Px looks stationary, at least on generating events. The following
lemma shows that this is enough to infer that the Px are indeed stationary.
Lemma 7.4.1 A measure m is stationary on (, (F)) if and only if m(T 1 F ) = m(F ) for all
F F.
Proof: Both m and mT 1 are measures on the same space, and hence they are equal if and only if
they agree on a generating field.
2
Note in passing
that
no
similar
countable
description
of
an
AMS
measure
is
known,
that
is,
if
Pn1
limn n1 i=0 m(T i F ) converges for all F in a generating field, it does not follow that the
measure is AMS.
denote the subset of G(F) consisting of all of the points x for which Px is ergodic.
As before let E
as
From Lemma 6.7.4, the definition of Px , and the stationarity of the Px we can write E
= {x : x G(F); lim n1
E
n

n1
X
i=0

Px (T i F F ) = Px (F )2 , all F F}.

7.4. THE ERGODIC DECOMPOSITION

163

where the given limit exists and has

Since the probabilities Px are measurable functions, the set E
the given value is also measurable. Since Px (F ) = PT x (F ), the set is also invariant. This proves (a)
is
For x 6 E
we have that px = p . Since E
of the theorem. We have already proved (b) for x E.
and hence also pT x = p so that pT x = px .
invariant, we also know that T x 6 E
We now consider an AMS measure m. Although Lemma 6.7.4 provided a countable description
of ergodicity that was useful to prove that the collection of all points inducing ergodic measures is
an event, the following lemma provides another countable description of ergodicity that is useful for

computing the probability of the set E.

Lemma 7.4.2 Let (, (F), m, T ) be an AMS dynamical system with stationary mean m. Define
the sets
n1
1X
1F (T i x) = m(F )}; F F.
G(F, m) = {x : lim
n n
i=0
Thus G(F, m) is the set of sequences for which the limiting relative frequency exists for all events F
in the generating field and equals the probability of the event under the stationary mean. Define also
\
G(F, m).
G(F, m) =
F F

A necessary and sufficient condition for m to be ergodic is that m(G(F, m)) = 1.

Comment: The lemma states that an AMS measure m is ergodic if and only if it places all of its
probability on the set of generic points that induce m through their relative frequencies.
Proof: If m is ergodic then < 1F >= m(F )m-a.e. for all F and the result follows immediately.
Conversely, if m(G(F, m)) = 1 then also m(G(F, m)) = 1 since the set is invariant. Thus integrating
to the limit using the dominated convergence theorem yields
lim n1

n1
X

m(T i F F )

1F ( lim n1

=
G(F ,m)

i=0

n1
X

1F T i )dm

i=0

m(F ) , all F F,

2
which with Lemma 6.7.4 proves that m is ergodic, and hence so is m from Lemma 6.7.2.
= 1, then the remainder of the theorem will follow
Proof of the Theorem If we can show that m(E)
from (7.9) and (7.10) and the fact that px = m(|I)(x) m a.e. From the previous lemma, this will
be the case if with px probability one
1F = px (F ), all F F,
or, equivalently, if for all F F

1F = px (F ) px a.e.

This last equation will hold if for each F F

Z
dpx (y)|1F (y) px (F )|2 = 0.
This relation will hold on a set of x of m probability one if
Z
Z
dm(x) dpx (y) |1F (y) px (F )|2 = 0.

CHAPTER 7. ERGODIC THEOREMS

164

Thus proving the preceding equality will complete the proof of the ergodic decomposition theorem.
Expanding the square and using the fact that G(F) has m probability one yield
Z
Z
dm(x)
dpx (y)1F (y)2
G(F )
G(F )
Z
Z
Z
2
dm(x)px (F )
dpx (y)1F (y) +
dm(x)px (F )2 .
G(F )

G(F )

G(F )

Since < 1F > (y) = py (F ), the first and third terms onRthe right are equal from (7.10). (Set
f (x) =< 1F (x) >= px (F ) in (7.10).) Since px is stationary, dpx < 1F >= px (F ). Thus the middle
term cancels the remaining terms and the preceding expression is zero, as was to be shown. This
completes the proof of the ergodic decomposition theorem.
2

Exercises
1. Suppose that a measure m is stationary and ergodic and N is a fixed positive integer. Show
that there is a measure that is N -stationary and N -ergodic that has the property that
m=

N 1
1 X
T i .
N i=0

This shows that although an ergodic source may not be N -ergodic, it can always be decomposed
into a mixture of an N -ergodic source and its shifts. This result is due to Nedoma [49].

7.5

The Subadditive Ergodic Theorem

Sample averages are not the only measurements of systems that depend on time and converge. In
this section we consider another class of measurements that have an ergodic theorem: subadditive
measurements. An example of such a measurement is the Levenshtein distance between sequences
(see Example 2.5.4).
The subadditive ergodic theorem was first developed by Kingman [39], but the proof again follows
that of Katznelson and Weiss [38]. A sequence of functions {fn ; = 1, 2, . . .} satisfying the relation
fn+m (x) fn (x) + fm (T n x), all n, m = 1, 2, . . .
for all x is said to be subadditive. Note that the unnormalized sample averages fn = n < f >n of
a measurement f are additive and satisfy the relation with equality and hence are a special case of
subadditive functions.
The following result is for stationary systems. Generalizations to AMS systems are considered
later.
Theorem 7.5.1 Let (, B, m, T ) be a stationary dynamical system. Suppose that {fn ; n = 1, 2, . . .}
is a subadditive sequence of m-integrable measurements. Then there is an invariant function (x)
such that m-a.e.
1
lim fn (x) = (x).
n n
The function (x) is given by
(x) = inf
n

1
1
Em (fn |I) = lim Em (fn |I).
n n
n

7.5. THE SUBADDITIVE ERGODIC THEOREM

165

Most of the remainder of this section is devoted to the proof of the theorem. On the way some
lemmas giving properties of subadditive functions are developed. Observe that since the conditional
expectations with respect to the -field of invariant events are themselves invariant functions, (x)
is invariant as claimed.
Since the functions are subadditive,
Em (fn+m |I) Em (fn |I) + Em (fm T n |I).
From the ergodic decomposition theorem, however, the third term is EPx (fm T n ) = EPx (fm ) since
Px is a stationary measure with probability one. Thus we have with probability one that
Em (fn+m |I)(x) Em (fn |I)(x) + Em (fm |I)(x).
For a fixed x, the preceding is just a sequence of numbers, say ak , with the property that
an+k an + ak .
A sequence satisfying such an inequality is said to be subadditive. The following lemma (due to
Hammersley and Welch [32]) provides a key property of subadditive sequences.
Lemma 7.5.1 If {an ; n = 1, 2, . . .} is a subadditive sequence, then
lim

an
an
= inf
,
n1 n
n

where if the right-hand side is , the preceding means that an /n diverges to .

Proof: First observe that if an is subadditive, for any k, n
akn an + a(k1)n an + an + a(k2)n . . . kan .

(7.16)

Call the infimum and first consider the case where it is finite. Fix > 0 and choose a positive
integer N such that N 1 aN < + . Define b = max1k3N 1 ak . For each n 3N there is a
unique integer k(n) for which (k(n) + 2)N n < (k(n) + 3)N . Since the sequence is subadditive,
an ak(n)N + a(nk(n)N ) . Since n k(n)N < 3N by construction, ank(n)N b and therefore

ak(n)N + b
an

.
n
n

Using (7.16) this yields

k(n)aN
b
k(n)N aN
b
k(n)N
b
an

+
+
( + ) + .
n
n
n
n
N
n
n
n

By construction k(n)N n 2N n and hence

b
an
++ .
n
n

Thus given an , choosing n0 large enough yields n1 an + 2 for all n > n0 , proving the
lemma. If is , then for any positive M we can choose n so large that n1 an M . Proceeding
as before for any there is an n0 such that if n n0 , then n1 an M + . Since M is arbitrarily
2
big and arbitrarily small, n1 an goes to .

CHAPTER 7. ERGODIC THEOREMS

166

Returning to the proof of the theorem, the preceding lemma implies that with probability one
1
1
Em (fn |I)(x) = lim Em (fn |I)(x).
n n
n1 n

(x) = inf

As in the proof of the ergodic theorem, define

f (x) = lim sup
n

and
f (x) = lim inf
n

1
fn (x)
n
1
fn (x).
n

Lemma 7.5.2 Given a subadditive function fn (x); = 1, 2, . . ., f and f as defined previously are
invariant functions m-a.e.
Proof: From the definition of subadditive functions
f1+n (x) f1 (x) + fn (T x)
and hence
1 + n f1+n (x) f1 (x)
fn (T x)

.
n
n
1+n
n
Taking the limit supremum yields f (T x) f (x). Since m is stationary, Em f T = Em f and hence
f (x) = f (T x) m-a.e. The result for f follows similarly.
2
We now tackle the proof of the theorem. It is performed in two steps. The first uses the usual
ergodic theorem and subadditivity to prove that f a.e. Next a construction similar to that of
the proof of the ergodic theorem is used to prove that f = a.e., which will complete the proof.
As an introduction to the first step, observe that since the measurements are subadditive
n1
1X
1
fn (x)
f1 (T i x).
n
n i=0

Since f1 L1 (m) for the stationary measure m, we know immediately from the ergodic theorem
that
f (x) f1 (x) = Em (f1 |I).
(7.17)
We can obtain a similar bound for any of the fn by breaking it up into fixed size blocks of length,
say N and applying the ergodic theorem to the sum of the fN . Fix N and choose n N . For each
i = 0, 1, 2, . . . , N 1 there are integers j, k such that n = i + jN + k and 1 k < N . Hence using
subadditivity
j1
X
fN (T lN +i x) + fk (T jN +i x).
fn (x) fi (x) +
l=0

Summing over i yields

N fn (x)

N
1
X
i=0

fi (x) +

jN
1
X
l=0

fN (T l x) +

N
1
X
i=0

fnijN (T jN +i x),