Pierre Bremaud - Mathematical Principles of Signal Processing PDF

Mathematical Principles of
Signal Processing
Springer-Verlag Berlin Heidelberg GmbH
Pierre Bremaud
Mathematical Principles of
Signal Processing
Fourier and Wavelet Analysis
, Springer
Pierre Bremaud
Ecole Polytechnique Federale de Lausanne
Switzerland
and
INRIAJEcole Normale Superieure
France
bremaud@ens.fr
Library of Congress Cataloging in Publication Data

Bremaud, Pierre.
Matbematical principles of signal processing / Pierre Bremaud.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-4419-2956-3 ISBN 978-1-4757-3669-4 (eBook)
DOI 10.1007/978-1-4757-3669-4
1. Signal processing-Matbematics. I. TitIe.
TK5102.9.B72 2001
621.382'2'OI51-dc21 2001042957
Printed on acid-free paper.
2002 Springer Science+Business Media New York

Originally published by Springer Science+Business Media New York, Inc in 2002
Softcover reprint of tbe hardcover I st edition 2002
All rights reserved. This work may not be translated or copied in whole or in part witbout tbe written
permission oftbe publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010,
USA), except for brief excerpts in connection witb reviews or scholarly analysis. U se in connection
witb any form of information storage and retrievaI, electronic adaptation, computer software, or by
similar or dissimilar metbodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in tbis publication, even if tbe
former are not especially identified, is not to be taken as a sign tbat such names, as understood by tbe
Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
Production managed by Allan Abrams; manufacturing supervised by Joe Quatela.

Typeset by The BartIett Press, Inc., Marietta, GA.
987 6 5 4 3 2 1
SPIN 10845428
ToMarion
Contents
Preface xi
A Fourier Analysis in LI 1
Introduction 3
Al Fourier Transforms of Stable Signals 7
A 11 Fourier Transform in L I 7
Al2 Inversion Formula . . . . . . . . 16
A2 Fourier Series of Locally Stable Periodic Signals 23

A21 Fourier Series in L}oc . . . . . . . . 23
A22 Inversion Formula . . . . . . . . . . 26
A3 Pointwise Convergence of Fourier Series 31

A31 Dini's and Jordan's Theorems. 31
A32 F6jer's Theorem . . . 39
A33 The Poisson Formula 43
References . . . . . . . 46
B Signal Processing 49
Introduction 51
BI Filtering 55
B 11 Impulse Response and Frequency Response 55
Bl2 Band-Pass Signals . . . . . . . . . . . . . . 68
viii Contents
B2 Sampling 75
B21 Reconstruction and Aliasing . 75
B22 Another Approach to Sampling 82
B23 Intersymbol Interference . 84
B2-4 The Dirac Formalism . . . . . 88
B3 Digital Signal Processing 95
B31 The DFf and the FFf Algorithm 95
B32 The Z-Transform . . . . . . . . . 100
B33 All-Pass and Spectral Factorization 109
B4 Subband Coding 115
B41 Band Splitting with Perfect Reconstruction . 115
B42 FIR Subband Filters 120
References. . . . . . . . . . . 126
C Fourier Analysis in L 2 127

Introduction 129
Cl Hilbert Spaces 133
C11 Basic Definitions. 133
C12 Continuity Properties 136
C13 Projection Theorem . 139
C2 Complete Orthonormal Systems 145
C21 Orthonormal Expansions . 145
C22 Two Important Hilbert Bases 150
C3 Fourier Transforms of Finite-Energy Signals 155
C31 Fourier Transform in L 2 155
C32 Inversion Formula in L 2 159
C4 Fourier Series of Finite-Power Periodic Signals 161
C41 Fourier Series in LToc . . . . . . . . . . . 161
C42 Orthonormal Systems of Shifted Functions 163
References. . . . . . . . . . . . . . . . . . . . . 166
D Wavelet Analysis 167

Introduction 169
D1 The Windowed Fourier Transform 175
D11 The Uncertainty Principle . . . . . . . . . 175
D12 The WFf and Gabor's Inversion Formula . 178
D2 The Wavelet Transform 185
D21 Time-Frequency Resolution of Wavelet Transforms 185
D22 The Wavelet Inversion Formula . . . . . . . . . . . 187
Contents ix
03 Wavelet Orthonormal Expansions 195

D31 Mother Wavelet . . . . . . . 195
D32 Mother Wavelet in the Fourier Domain 202
D33 Mallat's Algorithm. . . . . . . . . . 211
04 Construction of an MRA 217
D41 MRA from an Orthonormal System . 217
D42 MRA from a Riesz Basis .. 220
D43 Spline Wavelets . . . . . . . . . . . 223
05 Smooth MuItiresolution Analysis 229
D51 Autoreproducing Property of the Resolution Spaces 229
D52 Pointwise Convergence Theorem . . . 231
D53 Regularity Properties ofWavelet Bases 234
References . . . . . . . . . . . . . . . . . . . 237
Appendix 239
The Lebesgue Integral 241
References. . . . 261
Glossary of Symbols 263
Index 267
Preface
Fourier theory is one of the most useful tools in many applied sciences, part-
icularly, in physics, economics, and electrical engineering. Fourier analysis is a
well-established discipline with a long history of successful applications, and the
recent advent of wavelets is the proof that it is still very alive. This book is an
introduction to Fourier and wavelet theory illustrated by applications in commun-
ications. It gives the mathematical principles of signal processing in such a way
that physicists and electrical engineers can recognize the familiar concepts of their
trade.
The material given in this textbook establishes on firm mathematical ground the
field of signal analysis. It is usually scattered in books with different goals, levels,
and styles, and one of the purposes of this textbook is to make these prerequisites
available in a single volume and presented in a unified manner.
Because Fourier analysis covers a large part of analysis and finds applications
in many different domains, the choice of topics is very important if one wants
to devise a text that is both of reasonable size and of meaningful content. The
coloration of this book is given by its potential domain of applications-signal
processing. In particular, I have included topics that are usually absent from the
table of contents of mathematics texts, for instance, the z-transform and the discrete
Fourier transform among others.
The interplay between Fourier series and Fourier transforms is at the heart of
signal processing, for instance in the sampling theory at large (including multireso-
lution analysis). In the classical Fourier theory, the formula at the intersection of the
Fourier transform and the Fourier series is the Poisson formula. In mathematically
oriented texts, it appears as a corollary or as an exercise and in most cases receives
little attention, whereas in engineering texts, it appears under its avatar, the formula
xii Preface
giving the Fourier transfonn of the Dirac combo For obscure reasons, it is believed
that the Poisson sum fonnula, which belongs to classic analysis, is too difficult,
and students are gratified with a result of distributions theory that requires from
them a higher degree of mathematical sophistication. Surprisingly, in the applied
literature, whereas distribution theory is implicitly assumed to be innate, the basic
properties of the Lebesgue integral, such as the dorninated convergence and the
Fubini theorem, are never stated precisely and seldom used, although these tools
are easy to understand and would certainly answer many of the questions that alert
students are bound to ask. In order to correct this unfortunate tradition, which has
a demoralizing effect on good students, I have insisted on the fact that the c1assical
Poisson fonnula is all that is needed in signal processing to justify the Dirac
symbolism, and I have devoted some time and space to introduce the Lebesgue
integral in a concise appendix, giving the precise statements of the indispensable
tools.
The contents are organized in four chapters. Part A contains the Fourier theory
in LI up to the c1assical results on pointwise convergence and the Poisson sum
fonnula. Part B is devoted to the mathematical foundations of signal processing.
Part C gives the Fourier theory in L 2 . Finally, Part D is concemed with the time-
frequency issue, inc1uding the Gabor transfonn, wavelets, and multiresolution
analysis. The mathematical prerequisites consist of a working knowledge of the
Lebesgue integral, and they are reviewed in the appendix.
Although the book is oriented toward the applications of Fourier analysis, the
mathematical treatment is rigorous, and I have aimed at maintaining a balance
between practical relevance and mathematical content.
Acknowledgments
Michael Cole translated and typed this book from a French manuscript, and Clau-
dio Favi did the figures. Jean-Christophe Pesquet and Martin Vetterli encouraged
me with stimulating discussions and provided the illustrations of wavelet analy-
sis. They also checked and corrected parts of the manuscript, together with Guy
Demoment and Emre Telatar. Sebastien Allam and Jean-Fran~ois Giovanelli were
always there when TEX tried to take advantage of my incompetence. To all of them,
I wish to express my gratitude, as well as to Tom von Foerster, who showed infinite
patience with my prornises to deliver the manuscript on time.
Gif sur Yvette, France Pierre Bremaud

May 2,2001
Part A
Fourier Analysis in L1
Introduction
In 1807 Joseph Fourier (1768-1830) presented a solution ofthe heat equation l
ae a2e
-=K-,
at a2x
where e(x, t) is the temperature at time t and at loeation x of an infinite rod, and
Kis the heat eonduetanee. The initial temperature distribution at time 0 is given:
e(x,O) = f(x).
(The solution of the heat equation is derived in Seetion A 11.)

In fact, Fourier eonsidered a cireular rod of length, say, 21T, whieh amounts to
imposing that the funetions x --+ f(x) and x --+ e(x, t) are 21T-periodie. He gave
the solution when the initial temperature distribution is a trigonometrie series
f(t) = L cne int .

neZ
Fourier claimed that his solution was general beeause he was eonvineed that alI21T-
periodie funetions ean be expressed as a trigonometrie series with the eoefficients
Cn = cn(f) = -
21T
1
1
0
2lT
f(t)e -int dt.
lThe definitive form of his work was published in Theorie Analytique de la Chaleur,
Finnin Didot ed., Paris, 1822.
4 Part A Fourier Analysis in L I
Special cases of trigonometric developments were known, for instance, by

Leonhard Euler (1707-1783), who gave the formula
1 . 1 . 1 .
"2 x = sm(x) - "2 sm(2x) + 3" sm(3x) - "',
true for - l ( < x < +l(. But the mathematicians of that time were skeptical about
Fourier's general conjecture. Nevertheless, when the propagation of heat in solids
was set as the topic for the 1811 annual prize of the French Academy of Sciences,
they surmounted their doubts and attributed the prize to Fourier's memoir, with the
explicit mention, however, that it lacked rigor. Fourier's results that were in any case
true for an initial temperature distribution that is a finite trigonometric sum, and be
it only for this, Fourier fully deserved the prize, because his proof uses the general
tricks (for instance, the differentiation rule and the convolution-multiplication
rule) that constitute the powerful toolkit of Fourier analysis.
Nevertheless, the mathematical problem that Fourier raised was still pending,
and it took a few years before Peter Gustav Dirichlet2 could prove rigorously,
in 1829, the validity of Fourier's development for a large class of periodic func-
tions. Since then, perhaps the main guideline of research in analysis has been the
consolidation of Fourier's ingenious intuition.
The classical era of Fourier series and Fourier transforms is the time when the
mathematicians addressed the basic question, namely, what are the functions adrnit-
ting a representation as a Fourier series? In 1873 Paul Dubois-Reymond exhibited
a continuous periodic function whose Fourier series diverges at O. For almost one
century the threat of painful negative results had been looming above the theory.
Of course, there were important positive results: Ulisse Dini3 showed in 1880 that
if the function is locally Lipschitz, for instance differentiable, the Fourier series
represents the function. In 1881, Carnille Jordan4 proved that this is also true for
functions of locally bounded variation. Finally, in 1904 Leopold Fejeii showed
that one could reconstruct any continuous periodic function from its Fourier coef-
ficients. These results are reassuring, and for the purpose of applications to signal
processing, they are sufficient.
However, for a pure mathematician, the itch persisted. There were more and
more examples of periodic continuous functions with a Fourier series that diverges
at at least one point. On the other hand, Fejer had proven that if convergence is
taken in the Cesaro sense, the Fourier series of such continuous periodic function
converges to the function at all points.
2Sur la convergence des series trigonometriques qui servent arepresenter une fonetion
arbitraire entre des limites donnees, J. reine und angewan. Math., 4,157-169.
3 Serie di Fourier e altre rappresentazioni analitiche delle funzioni di une variabile reale,
Pisa, Nistri, vi + 329 p.
4Sur la serie de Fourier, CRAS Paris, 92, 228-230; See also Cours d'Analyse de l'lfcole
Polytechnique, I, 2nd ed., 1893, p. 99.
5Untersuchungen ber Fouriersehe Reihen, Math. Ann., 51-69.
Introduction 5
Outside continuity, the hope for a reasonable theory seemed to be completely

destroyed by Nikola'i Kolmogorov, 6 who proved in 1926 the existence of a periodic
locally Lebesgue-integrable function whose Fourier series diverges at alt points!
It was feared that even continuity could foster the worst pathologies. In 1966 Jean-
Pierre Kahane and yitzhak Katznelson7 showed that given any set of null Lebesgue
measure, there exists a continuous periodie function whose Fourier series diverges
at all points of this preselected set.
The case of continuous functions was far from being elucidated when Lennart
Carleson8 published in the same year an unexpected result: Every periodic locally
square-integrable function has an almost-everywhere convergent Fourier series.
This is far more general than what the optirnistic party expected, since the periodic
continuous functions are, in particular, locally square-integrable. This, together
with the Kahane-Katznelson result, completely settled the case of continuous
periodic functions, and the situation finally tumed out to be not as bad as the 1873
result of Dubois-Reymond seemed to forecast.
In this book, the reader will not have to make her or his way through a jungle
of subtle and difficult results. Indeed, for the traveler with practical interests, there
is a path through mathematics leading directly to applications. One of the most
beautiful sights along this road may be Simeon Denis Poisson's9 sum formula
L J(n) = L jen),
nEZ nEZ
where J is an integrable function (satisfying some additional conditions to be

made precise in the main text) and where
j(v) = L
J(t)e-2irrvt dt
is its Fourier transform, where ~ is the set of real numbers. This striking formula
found very nice applications in the theory of series and is one of the theoretical
results founding signal analysis. The Poisson sum formula is the culrninating result
of Part A, which is devoted to the classical Fourier theory.
6Une serie de Fourier-Lebesgue divergente partout, CRAS Paris, 183, 1327-1328.

7S ur les ensembles de divergence des series trigonometriques, Studia Mathematica, 26,
305-306.
8Convergence and growth of partial sums of Fourier series, Acta Math., 116, 135-157.
9S ur la distribution de la chaleur dans les corps solides, J. Ecole Polytechnique, 1geme
Cahier, XII, 1-144, 145-162.
Al
Fourier Transforms of Stable Signals
A 11 Fourier Transform in LI
This first chapter gives the definition and elementary properties of the Fourier
transform of integrable functions, which constitute the specific calculus mentioned
in the introduction. Besides linearity, the toolbox of this calculus contains the
differentiation rule and the convolution-multiplication rule. The general problem
of recovering a function from its Fourier transform then receives a partial answer
that will be completed by the results on pointwise convergence of Chapter A3.
We first introduce the notation: N, Z, Q, ~, C are the sets of, respectively,
integers, relative integers, rationals, real numbers, complex numbers; N+ and ~+
are the sets of positive integers and nonnegative real numbers.
In signal theory, functions from ~ to C are called (complex) signals. We shall
use both terminologies (function, or signal), depending on whether the context is
theoretical or applied.
We denote by L~(~) (and sometimes, for short, LI) the set offunctions f(t) 10
from ~ into C such that
L If(t)1 dt < 00.
In analysis, such functions are called integrable. In systems theory, they are called
stable signals.
IOWe shall often use this kind of loose notation, where a phrase such as "the function
f(t)" means "the function f : lR ~ c." We shall also use the notation "I" or "fO" with
a mute argument. For instance, "f( - a)" is the function t --+ f(t - a).
P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002
8 Al. Fourier Transforms of Stable Signals
Let A be a subset of IR. The indicator function of Ais the function lA : IR 1-+
{O, 1} defined by
lA(t) = 1o
1 if tE A,
ift ~ A.
The function I(t) is called locally integrable if for any closed bounded interval
[a, b] C IR, the function I(t)l[a,bj(t) is integrable. We shall then write
I E L~ loe(lR)
or, for short, I E Lloe'

The set of functions I (t) from IR into C such that
L I/(t)1 2 < 00
is denoted by L~(IR). It is the set of square-integrable functions. A signal I(t) in

this set is said to have a finite energy
E = L I/(t)1 2 dt.
The function I(t) is called locally square-integrable if for any closed bounded
interval [a, b] C IR, the function I(t)l[a,bj(t) is square-integrable. We shall then
write
I E L~,loe(lR)
or, for short, I E L toe'

We recall that in L~(IR) or L~(IR) two functions are not distinguished if they are
equal almost everywhere with respect to the Lebesgue measure.
EXERCISE AI.I. Give an example 01 a function that is integrable but not 01finite
energy. Give an example 01 a function that is 01finite energy but not integrable 01
finite energy. Show that
A function I : IR 1-+ C is said to have bounded support if there exists a bounded

interval [a, b] c IR such that I(t) = 0 whenever t ~ [a, b].
If the function I(t) is n times continuously differentiable (that is, it has deriva-
tives up to order n, and these derivatives are continuous), we say that it is in Cn .
If it is in Cn for all n E N, it is said to belong to COO The kth derivative of the
function I (t), if it exists, is denoted jCk)(t). The Oth derivative is the function itself:
I(O)(t) = I(t); in particular, CO is the collection of continuous functions from IR
to C. The set of continuous functions with bounded support is denoted by C~.
AII Fourier Transfonn in LI 9
Fourier Transform
We can now give the basic definition.
DEFINITION AI.I. Let s(t) be a stable complex signal. The Fourier transform (FT)
ofs(t) is thefunctionfrom:IR into C:
s(v) = L s(t) e-2i1rvI dt. (1)
(Note that the argument of the exponential in the integrand is -2i7rvt.) The
mapping from the function to its Fourier transform will be denoted by
s(t) ~ s(v) or :F: s(t) --* s(v).

Table A 1.1 gives the immediate properties of the Fourier transform.
Table AI.I. Elementary Properties of Fourier Transfonns
Delay s(t - to) ~ e- 2ill'vtos(v)
Modulation e2i1rvot s(t) ~ s(v - vo)
Doppler s(at) Fr
--+ -IsA{V}
-
lai a
AISI (t) + A2S2(t) ~ AISI(V) + A2 S2(V)

s*(t) ~ s(-v)*
EXERCISE A1.2. Prove the assertions in Table Al.I.

EXERCISE AI.3. Prove the modulation theorem:
2(s(v - Vo ) + sA( v
Fr 1 A
s(t) cos(27rvot) --+ + Vo )). (2)
(See Fig. AI.I.)

EXERCISE AI.4. Show that the FT of a real signal is Hermitian even, that is,
s(- v) = s(v)*.
Show that the FT of an odd (resp., even; resp., real and even) signal is odd (resp.,
even; resp., real and even).
EXERCISE A1.S. Defining the rectangular pulse
recT(t) = I I- t .+ t1 (t)
s(v) Hs(v + vo) + s(v - vo)}
o v -vo o +vo v
Figure Al.l. Modulation theorem

10 Al. Fourier Transfonns of Stable Signals
and the cardinal sine

sin(Jrx)
sinc (x) = ---
Jrx
show that (see Fig. Al.2)
recT(t) ~ Tsinc (vT). (3)
T
1
-T/2 o +T/2 1 2 3
T T T
reCT(t) Tsinc(vT) = recT(v)
Figure A 1.2. Fourier transfonn of the rectangle function
We will show that the Gaussian pulse is its own Fr, that is,
(4)
In order to compute the corresponding Fourier integral, we use contour integration
in the complex plane. First, we observe that it is enough to compute the Fr s( v)
for v 2: 0, since this Fr is even (see Exercise A1.4). Take a 2: v (eventually, a
will tend to 00).
Consider the rectangular contour Y in the complex plane (see Fig. A1.3),
Y = Yl + Y2 + Y3 + Y4,
where the Yi 's are the oriented line segments
Yl : (-a, 0) -+ (+a, 0),

Y2 : (+a, 0) -+ (+a, v),
Y3 : (+a, v) -+ (-a, v),
Y4: (-a, v) -+ (-a, 0).
')'4
-a +a
Figure A1.3. The integration path in the proof of (4)
A 11 Fourier Transform in L I 11
We denote by -Yi the oriented segment whose orientation is opposite that of Yi.
i
Wehave
e- rrz2 dz = 11 + /z + h + 14,
where li is the integral of e- rr Z2 along Yi. Since the latter integrand is a holomorphic
function, by Cauchy's theorem (see, for instance, Theorem 2.5.2, p. 83, of [Al],
or Theorem 2.2, p. 101, of [A6]),
i e- rrz2 dz = 0,
and therefore,
h + /z + h + 14 = O.
We now show that
lim /z !im 14 = o.
= a-+oo
a~oo
For /Z, for instance, if we parameterize Y2 as folIows,

Y2 = {a + it;O,:::: t .:::: v},
l l
then
/z = v
e-rr(a+itf i dt = v
e-rr(a2-t2)e-2irrat i dt.
Therefore, since v .:::: a,
l/zl .:::: l a
e-rr(a-t)(a+t) dt .:::: l a
e-rra(a-t) dt
= e- rra 21o a
errat 1
dt = -(1 _
Jra
e- rra 2 ),
where the last quantity tends to 0 as a tends to +00. A sirnilar conclusion holds
for 14, with sirnilar computations. Therefore,
!im (h
a-HXl
+ h) = 0,
that is,
(5)
Using for YI the obvious parameterization
!im { = lim j+a e- rrt2 dt = { e- rrt2 dt = 1.

a--+oo JYl a--+oo -a JJ!I!.
Parameterizing -Y3 as folIows,
-Y3 = {iv + t; -a .:::: t .:::: +a},
12 Al. Fourier Transforms of Stahle Signals
wehave
1-)'3
= l+ a e-n(iv+tf dt
-a
Therefore,
Going back to (5), we obtain
which gives the announced result.

EXERCISE A1.6. Deduce Jrom (4) that, Jor all Cl > 0,
The Fr of the Gaussian pulse can be obtained by other means (see Exercise
A 1.16). However, in other cases, contour integration is often necessary.
Using contour integration in the complex plane, we show that, for a > 0,
1
s(t) = e-at IlR+(t) Fr
~ s(v)
A
= . (6)
a+2mv
1
First observe that
s(v) = 1
o
00
e- 2znvt
.
e- at dt = . 1
2mv + a 0
00
.
e-2znvt-a\2inv + a) dt
= 1
2inv + a
1 Y
e- Z dz.
(The reader is refered to Fig. A1.4 for the definition ofthe lines y, YJ. Y2, and Y3.)
Therefore, it suffices to show that
i e- z dz = 1.
By Cauchy's theorem,
1 Yl
e-Z dz + 1Y2
e- Zdz + 1 )'3
e- Zdz = 0.
A 11 Fourier Transform in LI 13
2i7f1/
Figure A1.4. The integration path in the proof of (6)
The limit as A t 00 of I Yl e- Z dz is I y e- Z dz, and that of I Y3 e- Z dz = IoA e- t dt

is 1. It therefore remains to show that the limit as A t 00 of J,Y2 e- Z dz is 0, and
this foIlows from the bound
I{ e-zdzl ::: e- A IY21,
where IY21 = K x A is the length of Y2.

EXERCISE AI.7. Deducefrom (6) that
Convolution-Multiplication Rule
THEOREM AI.I. Let h(t) and x(t) be two stable signals. Then the right-hand side
oJ
y(t) =1 h(t - s)x(s) ds (7)
is defined Jor almost all t and defines almost everywhere a stable signal whose FT
is y(v) = h(v)x(v).
Proof" By ToneIli's theorem and the integrability assumptions,
f1xIR Ih(t - s)llx(s)1 dt ds = (l'h(t)' dt) (l'X(t)' dt) < 00.
This implies that, for almost aIl t,
l'h(t - s)x(s)1 ds < 00.
The integral IIR h(t - s )x(s) ds is therefore weIl defined for almost aIl t. Also,
1,y(t)' dt = 111 h(t - s)x(s) dsl dt

::: 11'h(t - s)x(s)1 dt ds < 00.
Therefore, y(t) is stable. By Fubini's theorem,
L(Lh(t - S)X(S)dS) e-2irrvt dt
=L Lh(t - s)e-2irrv(t-s)x(s)e-2irrvs ds dt
= h(v)x(v).

The funetion y(t), the convolution of h(t) with x(t), is denoted by
y(t) = (h * x)(t).
We therefore have the convolution-multiplication rule,
(h * x)(t) --+
Fr A
h(v)x(v). (8)
EXAMPLE AI.L The convolution of the rectangular pulse reeT (t) with itself is the
triangular pulse of base [- T, + T] and height T,
TriT(t) = (T - Itl)1[-T,+T](t).
By the convolution-multiplication rule,
TriT(t) ~ (Tsine (vT)f (9)
(see Fig. Al.5).

EXERCISE ALS. Let x(t) be a stable complex signal. Show that its autoeorrelation
funetion
c(t) =L x(s + t)x*(s) ds

is well defined and integrable and that its FT is Ix(v)1 2
T2
&
T
A
I
I
I
i
C'>~
I
I
. 1
~C'>
j I
-T +T -~ -~ -~ 0 ~ ~ ~
TriT(t)
Figure A 1.5. FT of the triangle funetion

All Fourier Transform in LI 15
EXERCISEAl.9. Showthatthenthconvolutionpoweroff(t) = e-atlt~o(t), where

a > 0, is
t n- 1
fM(t) = (n - I)!
e- at I t>o(t).
-
(/*3 = f * f * f, etc.) Deducefrom this the FTofs(t) = tne-atlt~o(t).
Riemann-Lebesgue Lemma
The Riemann-Lebesgue lemmal! is one of the most important technical tools
in Fourier analysis, and we shall use it several times, especially in the study of
pointwise convergence of Fourier series (Chapter A3).
THEOREM Al.2. The FT of a stable complex signal s(t) satisfies
lim Is(v)1 = O. (10)
Ivl-+oo
Proof' The Fr of a rectangular pulse s(t) satisfies Is(v)1 ::; K/lvl [see Eq. (3)].
Hence every signal s(t) that is a finite linear combination of indicator functions
of intervals satisfies the same property. Such finite combinations are dense in
Lb(l~) (Theorem 28 of the appendix), and therefore there exists a sequence sn(t)
of integrable functions such that
lim (Isn(t) - s(t)1 dt =0

n-+oo JIR
and
A Kn
ISn(v)1 ::; ~'
for finite numbers Kn From the inequality
Is(v) - sn(v)1 ::; L Is(t) - sn(t)1 dt,
we deduce that
::; -K n +
lvi
iIR
Is(t) - sn(t)1 dt,
from which the conclusion follows easily.

The following uniform version of the Riemann-Lebesgue lemma will be needed
in the sequel.
11 Riemann, B., (1896), Sur la possibilite de representer une fonetion par une serie
trigonometrique, Oeuvre Math., p. 258.
THEOREM Al.3. Let f(t) be a 2:rr-periodic locally integrable function, and let
g: [a,b] CbeinC I , where [a,b] ~ [-:rr, +:rr]. Then
f-*
b
lim l fex - u)g(u) sin(Au) du = 0
....... 00 a
uniformly in x.
Proof For arbitrary E: > 0, choose a 2:rr-periodic function h(t) in Cl such that
r:rr If(x) - h(x)1 dx < E:
(Theorem 29 of the appendix). Integrating by parts yields
/(A) =l b
hex - u)g(u) sin(Au) du
COS(AU) Ib b
=- hex - u)g(u) - - +l [hex - u)g(u)]
, COS(AU)
du.
A a a A
Since h E Cl and is periodic, h and h' are uniformly bounded. The same is true of
g, g' (g is in Cl). Therefore,
lim /(A)
....... 00
=0 uniformly in x .
Now,
11 b
fex - U)g(U)Sin(AU)dUI
:S I/(A)I +l b
Ih(x - u) - fex - u)llg(u)1 sin(Au) du
:S I/(A)I + a~~ Ig(U)ll b Ih(x - u) - fex - u)1 sin(Au) du
:S I/(A)I + max Ig(u)IE:.

a:'Ou:'Ob
The conc1usion then follows because E: is arbitrary.

AI 2 Inversion Formula
EXERCISE Al.lO. Show that the FT of a stable signal is uniformly bounded and
uniformly continuous.
Despite the fact that the FT of an integrable signal is uniformly bounded and
uniformly continuous, it is not necessarily integrable. For instance, the FT of the
rectangular pulse is the cardinal sine, a non-integrable function. When its FT is
integrable, a signal admits a Fourier decomposition.
AI 2 Inversion Fonnula 17
THEOREM At.4. Let set) be an integrable complex signal with the Fourier
transform s(v). Under the additional condition
L Is(v)1 dv < 00, (11)
the inversion formula
set) = Ls(v)e+2iJrvt dv (12)
holdsfor almost all t.lf set) is, in addition to the above assumptions, continuous,
equality in (12) holds for all t.
(Note that the exponent ofthe exponential ofthe integrand is +2irrvt.)
EXERCISE At.ll. Check that the above result is true for the signal
(a E lR, a > 0, a E C).
Proof' We now proceed to the proof of the inversion formula. (lt is rather tech-
nical and can be skipped in a first reading.) Let set) be a stable signal and consider
the Gaussian density function
with the Fr
We first show that the inversion formula is true for the convolution (s * h u )(t).
Indeed,
(s * hu)(t) = JR{ s(u)hu(u)e...L ';;2 (t)du,2u 2

u (13)
and the Fr ofthis signal is, by the convolution-multiplication formula, s(v)hu(v).

Computing this Fr directly from the right-hand side of (13), we obtain
s(v)hu(v) = ( s(u)hu(u) ( { e I u (t)e-2iJrvt dt) du

J~ J~ ~;;r
= J~{ s(u)hu(u)e ~,;;r (v) du.

I u
Therefore, using the result ofExercise A1.11,
{ s(v)hu(v)e2iJrvt dv = {( { s(u)hu(u)e I ,U (v) dU) e2iJrvt dv

J~ J~ J~ ~ ;;r
= JR
{ s(u)hu(u)e Zc;2' -'<-(t)du
I
(12
= (s * h u )(t).
18 Al. Fourier Transforms of Stahle Signals
Thus, we have
(s * hu)(t) = L s(v)hu(v)e2i1rvt dv, (14)
and this is the inversion formula for (s * h u )(t).

Since for all v E IR, limu-l-o v t hu(v) = 1, it follows from Lebesgue's dom-
inated convergence theorem that when u ..I- 0 the right-hand side of (14) tends
L
to
s(v)e2i1rvt dv
for all t E IR. If we can prove that when u ..I- 0 the function on the left-hand side of
(14) converges in L~(IR) to the function s(t), then, for almost all t E IR, we have
the announced equality (Theorem 25 of the appendix).
To prove convergence in L~(IR), we observe that
L* I(s hu)(t) - s(t)1 dt = LIL (s(t - u) - S(t))hu(U)dul dt (15)
(using the fact frr~. hu(u)du = 1), and therefore, defining f(u) = iJR Is(t - u)-
s(t)1 dt,
L* Is hu(t) - s(t)1 dt ::::: L f(u)hu(u) du.
Now, If(u)1 is bounded (by 2 iJR Is(t)1 dt). Therefore, iflimu-l-o f(u) = 0, then, by
dominated convergence,
lim [ f(u)hu(u) = lim [ f(uu)ht(u)du = o.

uwk uwk (16)
Toprovethatlimu-l-O f(u) = 0, we begin with thecasewheres(t)iscontinuous with

compact support. In particular, it is uniformly bounded. Since we are interested in
a limit as u tends to 0, we may suppose that u is in a bounded interval around 0,
and in particular, the function t -+ Is(t - u) - s(t)1 is bounded uniformly in u by
an integrable function. It follows from the dominated convergence theorem that
limu-l-o f(u) = o.
Now, let s(t) be only integrable. Let {snO}n>t be a sequence of continuous
functions with compact support that converges i;' L~(IR) to sO (Theorem 27 of
the appendix). Writing
f(u) ::::: d(s( - u), sn(' - u)) + L ISn(t - u) - sn(t)1 dt + d(s(), snO),
where
d(s( - u), sn(' - u)) =L Is(t - u) - sn(t - u)1 dt,
the result easily follows.

Al2 Inversion Fonnula 19
Suppose that, in addition, set) is continuous. The right-hand side of (12) defines
a continuous function because s(v) is integrable. The everywhere equality in (12)
follows from the fact that two continuous functions that are almost everywhere
equal are necessarily everywhere equal (Theorem 8).
The Fourier transform characterizes a stable signal:

COROLLARY ALL lf two stable signals SI (t) and S2(t) have the same Fourier
transform, then they are equal almost everywhere.
Proof' The signal set) = Sl(t) - S2(t) has the FT s(v) = 0, which is integrable,

and thus by (12), set) = for almost all t.
EXERCISE AI.12. Give the FT of set) = 1/ A(a 2 + t 2). Deduce from this the value
of the integral
f ~dU,
l(t)= t > 0.
J.R.t+u
EXERCISE Al.13. Deduce from the Fourier inversion formula that
L(Si~(t) Y dt = Jr.
Exercise 1.14 is very important. It shows that for signals that cannot be called
pathological, the version of the Fourier inversion theorem that we have in this
chapter is not applicable, and therefore we shall need finer resuIts, which are given
in Chapter A3.
EXERCISE AI.14. Let set) be a stable right-continuous signal, with a limit from
the left at all times. Show that if s(t) is discontinuous at some time to, its FT cannot
be integrable.
Regularization Lemma
In the course of the proof of Theorem A1.4, we have used a special case of the
regularization lemma below, which is very useful in many circumstances.

DEFINITION AI.2. A regularizing function is a nonnegative function h a : IR ---+ IR
depending on a parameter a > and such that
L ha(u) du = 1, forall a > 0,
lim
a'\-O
l-a
+a
ha(u) du = 1, forall a > 0,
limha(u) = 1, forall u E IR.

a'\-O
LEMMA ALl. Let h a : IR ---+ IR be a regularizing function. Let set) be in Lb(IR).

Then
lim
a'\-O
f
JIR
I(s * h a )(t) - s(t)1 dt = 0.
20 Al. Fourier Transforms of Stable Signals
Proof" We ean use the proof of Theorem A1.4, starting from (15). The only
plaee where the speeifie form of h" (a Gaussian density) is used is (16). We must
therefore prove that
lim ( J(u)h,,(u) =0
"-1-0 JIR
independently. Fix e > O. Sinee limuto J(u) = 0, there exists a = aCe) such that
1-a~ J(u)h,,(u) du :s -2el~

-a
h,,(u) du :s -.
2
e
Sinee J(u) is bounded (say, by M),
( J(u)h,,(u) du :s M ( h,,(u) du.

JIR\[ -a.+a] JIR\[ -a.+a]
The last integral is, for suffieiently small a, less than e12M. Therefore, for
suffieiently small a,
(
JIR J(u)h,,(u) du :s
e e
2" + 2" = e.
The funetion h" is an approximation of the Dirae generalized funetion o(t) in
that, for all <p Ec2,
lim ( h,,(t) <p(t) dt = <p(0) = { o(t) <p(t)dt.
,,-1-0 JIR JIR
The last equality is symbolie and defines the Dirae generalized funetion (see See-
tion B24). The first equality is obtained as in the proof of the previous lemma,
this time letting J(u) = <p(u) - <p(0).
Differentiation in the Frequency Domain
We shall see how differentiation in the time domain is expressed in the frequeney
domain.
THEOREM A1.S. (a) Ifthe integrable signal set) is such that tks(t) E LU~)Jor
alt 1 :s k :s n, then its FT is in Cn , and
(-2imls(t) ~ s(k)(v) Joraltl:S k:s n.
(b) IJ the signal set) E cn and if it is, together with its n first derivatives,
integrable, then
S(k)(t) ~ (2inv)k s (v) Jor altl :s k :s n.
Proof" (a) In the right-hand side ofthe expression
s(v) = L e-2invIs(t)dt,
A12 Inversion Forrnula 21
we can differentiate k times under the integral sign (see Theorem 15 and the
hypothesis t ks(t) E Lb(IR)) and obtain
s(k)(v) = L (_2i:n:t)k e -2irrvt s(t) dt.
(b) It suffices to prove this for n = 1, and iterate the result. We first observe that
limlaltoo s(a) = O. Indeed, with a > 0, for instance,
s(a) = s(O) + l a
s'(t) dt,
and therefore, since s'(t) E Lb(IR), the limit exists and is finite. This limit must be
obecause s(t) is integrable. Now, the Fr of s'(t) is
[ e-2irrvts'(t)dt = lim [+a e-2irrvts'(t)dt.
JIR atoo La
Integration by parts yields
a
i : e-2irrvts'(t)dt = (e-2irrvts(t)):: + i:a(2i:n:V)e-2irrvtS(t)dt.
It then suffices to let a tend to 00 to obtain the announced result.

EXERCISE AI.IS. Let s(t) be a stable signal with a Fourier transform with compact
support. Show that s(t) E Coo, that all its derivatives are integrable, and that the
kth derivative has the FT (2i:n:v)k s (v).
EXERCISE AI.16. Give a differential equation satisjied by the Gaussian pulse, and
use it to deduce its Fourier transform. Could you do the same to prove (6)?
The Beat Equation

We now pay our tribute to the founder and give the solution of the heat equation,
which was announced in the introduction. Recall that the heat equation relative to
an infinite rod is the partial differential equation
aB a2B
(17)
at = K a2x'
where B(x, t) is the temperature at time t and at location x of the rod with heat
conductance K, and with the given initial temperature distribution
B(x, 0) = f(x). (18)
We assume that f is integrable.

Let ~ 1-* 8(~, t) be the Fr of x 1-* B(x, t). (We take different notations because
the variable with respect to which the Fr is taken is not the time variable t but the
space variable x.) In the Fourier domain, Eq. (17) becomes
d 8(c t)
--:-,"-'- = -K(4:n:2~2)e(~, t),
dt
with the initial condition

e(~, 0) = F(H
where F(O is the Fr of fex). The solution is

e(~, t) = F(~)e-4n2K~2f.
Since x 1-+ (4JrKt)-1/2 e (4Kf)-1/2 x 2 has the Fr ~ 1-+ e-4n2K~\ the convolution-
multiplication formula gives
1 ( 2
8(x, t) = (4JrKt)-Z J~ fex - y)e-i'ii dy,
or
8(x, t) = ..Jrr L fex - 2-JKiy)e-yl dy .
As we mentioned earlier, Fourier considered the finite rod heat equation, which
receives a similar solution, in terms of Fourier series rather than Fourier integrals
(see Chapter A2). The efficiency of the Fourier method in solving differential or
partial differential equations of mathematical physics has been, after the pioneering
work of Fourier, amply demonstrated 12 .
12See, for instance, the classic text of 1. N. Sneddon, Fourier Transfonns, McGraw-Hill,
1951; Dover edition, 1995.
A2
Fourier Series of Locally Stable
Periodic Signals
A21 Fourier Series in L loc
Fourier Coefficients
A periodic signal is neither stable nor of finite energy unless it is almost everywhere
null, and therefore, the theory of the preceding Chapter is not applicable. The
relevant notion is that of Fourier series. (Note that Fourier series were introduced
before Fourier transforms, in contrast with the order of appearance chosen in this
text.) The elementary theory of Fourier series of this section is parallel to the
elementary theory of Fourier transforrns of the previous section. The connection
between Fourier transforrns and Fourier series is made by the Poisson sum formula,
of which we present a weak (yet useful) version in this chapter.
A complex signal s(t) is called periodic with period T >
for all t E ~,
(or T -periodic) if,
s(t + T) = s(t).
AT -periodic signal s(t) is locally stable, or locally integrable, if s(t) E Lb([O, Tl),
that is,
l T
Is(t)1 dt < 00.
A T -periodic signal s(t) is locally square-integrable if s(t) E L~([O, Tl), that is,
l T
Is(t)1 2 dt < 00.

24 A2. Fourier Series of Locally Stable Periodic Signals
One also says in this case that s(t) hasfinite power, since
lim ..!.. {A Is(t)12 =..!.. (T Is(t)12 dt < 00.

A-+oo A 10 T 10
As the Lebesgue measure of [0, T] is finite,L~([O, Tl) c Lt([O, Tl). (See Theo-
rem 19 of the appendix.) In particular, a finite-power periodic signal is also locally
stable.
We are now ready for the basic definition.
DEFINITION A2.I. The Fourier transform {sn}, n E Z, of the locally stable T-
periodic signal s(t) is defined by theformula
Sn = -I
T
l0
T
s(t)e- 2'lJr TI
n dt,
(19)
and Sn is ca lIed the nth Fourier coefficient of the signal s(t).

EXERCISE A2.I. Compute the Fourier coefficients of the T -periodic function s(t)
such that on [0, T), s(t) = t.
EXERCISE A2.2. Let s(t) be a locally stable T -periodic signal. Defining
ST(t) = s(t)I[O,TJ(t),
show that the nth Fourier coefficient Sn ofs(t) and the FT i:;(v) OfST(t) are linked
by
(20)
EXERCISE A2.3. Compute the Fourier coefficients of the T -periodic signal s(t)
such that on [-T /2, +T /2), s(t) = 1[-a~,+a~l(t), where a E (0, 1).
EXERCISE A2.4. Let s(t) be a T -periodic locally stable signal with nth Fourier
coefficient Sn. Show that limlnltoo Sn = O.
One often represents the sequence {Sn}nEZ of the Fourier coefficients of a T-
periodic signal by "spectrallines" separated by 1/ T from each other along the
frequency axis. The spectralline at frequency n / T has the complex amplitude Sn
(see Fig. A2.1). This is sometimes interpreted by saying that the FT of s(t) is
s(v) = I)n(V -
nEZ
f)'
where (t) is the Dirac generalized function (see Section B2-4).
EXERCISE A2.5. Let s(t) be a T -periodic locally stable signal with nth Fourier
coefficient Sn. What is the nth Fourier coefficient of s(t - a), where a E llV What
can you say about the period and the Fourier coefficients of the signal s(t / a),
where a > O?
A21 Fourier Series in Lloc 25
Figure A2.l. From the Fourier transform to the Fourier coefficients
Convolution-MuItiplication Rule
THEOREM A2.1. Let x(t) be a T -periodie locaily stable signal, and let h(t) be a
stable signal. The signal
y(t) = L h(t - s)x(s)ds (21)
is almost everywhere weil defined, T -periodie, and locaily stable. Its nth Fourier
coefficient is
Yn
A
= hA(n)
T x n, A
(22)
where h(v) is the FT of h(t) (see Fig A2.2).

Proof We have
L1h(t - s)llx(s)1 ds = la T IhT(t - s)llx(s)1 ds,

where
hT(u) =L h(u + nT).
nEZ
Now
x(v) fI(v)
-~
I -q,I I I I I I
-~ 0
123
T T T v / "
" ,
Figure A2.2. Filtering aperiodie signal

and hence by the usual argument (see the proof of Theorem A 1.1),
L Ih(t - s)llx(s)1 ds < 00,
for almost aIl t E lR. Thus, y(t) is almost everywhere weIl defined by (21). Also,
y(t + T) = L x(t +T - s)h(s)ds
= L x(t - s)h(s)ds,
which shows that y(t) is periodic with period T. The same argument as in the proof
of Theorem AU shows that y(t) is locally stable. FinaIly,
l
Yn = -1
T 0
T
y(t)e- Z'l1!'j't
n dt
=- l 1 1 T T
hT(t
- n dt ds
- s)x(s)e- Z'l1!'j't
T 0 0

A22 Inversion Formula
The Poisson Kernel

In the proof of the Fourier series inversion formula, the Poisson kernel will play
a role similar to that of the Gaussian pulse in the proof of the Fourier transform
inversion formula of the previous seetion.
The Poisson kernel is the family of functions Pr : lR f-+ <C, 0 < r < 1, defined by
Pr(t) = L rlnleZin ~t. (23)
nEZ
For fixed r, Pr is T -periodie, and elementary computations reveal that

Pr(t) = LrneZin~t + Lrne-zin~t_l
n~O n~O
A22 Inversion Formula 27
and therefore,
Pr(t) :::: O. (24)
Also,
-1 I+ / Pr(t) dt
T 2
= 1. (25)
T -T/2
In view of the above expression of the Poisson kernel, we have the bound
1 [ (1 - r 2 )
- Pr(t)dt <
1 1 _ e 2i1ry 1
T [-t,+t]\[-e,+s] - 2 '
and therefore, for all e > 0,

. -1 [
hm Pr(t)dt = O. (26)
rtl T [-t,+t]\[-e,+e]
Properties (24)-(25) make of the Poisson kernel a regularizing kernel, and in

particular,
!im
rtl
-1
1
T
+t
-t
cp(t)Pr(t) dt = cp(O),
for all bounded, continuous cp : ffi. -+ <C (same proof as in Lemma ALl).
The following result is similar to the Fourier inversion formula for stable signals
(Theorem A1.4).
THEOREM A2.2. Let set) be aT -periodie localty stable complex signal with Fourier
coefficients {sn}, n E Z. lf
(27)
then, for almost alt t E ffi.,

set) = LSne+2i1rYI. (28)
nEZ
lfwe add to the above hypotheses the assumption that set) is a continuousfunction,
then the inversion formula (28) holds for all t.
Proof The proof is similar to that of Theorem A1.4. We have
LSnrlnle2i1ryl
11+[
= _ 2 s(u)Pr(t - u)du, (29)
nEZ T -t
and
10r I10r = 0,
T T
!im s(u)Pr(t - u) du - S(t)1 dt
rtl T
that is: The right-hand side of (29) tends to set) in Lb([O, T]) when r t 1. Since
LnEZ ISn I < 00, the function of t in the left-hand side of (29) tends toward the
28 A2. Fourier Series of Locally Stable Periodie Signals
function LnEZ sne+2irr(n/T)t, pointwise and in L~([O, T]). The result then follows
from Theorem 25.
The statement in the case where set) is continuous is proved exactly as the
corresponding statement in Theorem A1.4.
As in the case of stable signals, we deduce from the inversion formula the
uniqueness theorem.
COROLLARY A2.1. Two locally stable periodic signals with the same period T
that have the same Fourier coefficients are equal almost everywhere.
EXERCISE A2.6. Compute
using the expression ofthe Fourier coefficients ofthe 2-periodic signal set) such
that
fort E [-1,+1].
EXERCISE A2.7. Let x(t) be a T -periodic locally stable signal with nth Fourier
coefficient x n such that
L InlPlxnl < 00.

nEZ
Show that x(t) is p times differentiable and that if the pth derivative is locally
integrable, its nth Fourier coefficient is (2i7T!.f)P Xn.
The Weak Poisson Formula

The Poisson sum formula takes many forms. The strong version is
(30)
This aesthetic formula has a number of applications in signal processing (see Part
B).
The next result establishes the connection between the Fourier transform and
Fourier series, and is central to sampling theory. It is a weak form of the Poisson
sum formula (see the discussion after the statement of the theorem).
'THEOREM A2.3. Let set) be a stable complex signal, and let 0 < T < 00 be
fixed. The series LnEZ set + nT) converges absolutely almost everywhere to a
T -periodic locally integrable function (t), the nth Fourier coefficient of which is
(l/T)s(n/T).
We paraphrase this result as follows: Under the above conditions, the function
(t) := L set + nT) (31)
nEZ
A22 Inversion Forrnula 29
is T -periodie and locally integrable, and its formal Fourier series is
Sj(t) = ~ I)(!!..) e2inIfI. (32)

T nEZ T
(We speak of a "formal" Fourier series, because nothing is said about its conver-
gence.) Therefore, whenever we are able to show that the Fourier series represents
the function at t = 0, that is, if <1>(0) = S j(O), then we obtain the Poisson sum
formula (30).
For now, we are saying nothing about the convergence of the Fourier series.
This is why we talk about a weak Poisson's formula. A strong Poisson's formula
corresponds to the case where one can prove the equality everywhere (and in
particular at t = 0) of (t) and of its Fourier series. We shall say more about the
Poisson formula and, in particular, give strong versions of it in Seetion A33. The
version we have here, and that we shaH proceed to prove, is the one we need in the
Shannon-Nyquist sampling theorem (Chapter B2).
Proof: We first show that (t) is weH defined:
{T L Is(t + nT)1 dt = L (T Is(t + nT)1 dt

10 nEZ nEZ 10
= L l <n+l l T
Is(t)1 dt =
1 Is(t)1 dt < 00.
nEZ nT R
In particular,
L Is(t + nT)1 < 00 a.e.

nEZ
Therefore, the series LnEZ set +nT) converges absolutely for almost all t. In part-
icular, (t) is weH defined (define it arbitrarily when the series does not converge).
This function is c1early T -periodie. We have
{T 1(t)ldt = {T ILS(t+nT)ldt
10 10 nEZ
:s {T L Is(t + nT)1 dt = (ls(t)1 dt < 00.

10 nEZ 1R
Therefore, (t) is stable. Its nth Fourier coefficient is
cn((t)e- 2"''in l dt
= ~ {T {L set + kT)! e-2inIfI dt

T 10 kEZ
=~
T
{T
Ja
!LS(t + kT)e-
kEZ
2i1C ',f(t+kTl! dt
1 ~ (n )
= T1 JJR
{
s(t)e- l1C'it dt = T S T .
2 n

We have a function as weH as its formal Fourier series. When both are equal
everywhere, we obtain the strong Poisson sum formula. The next exercise gives
conditions for this.1t will be improved by Theorem A3.12.
EXERCISE A2.S. Let set) be a stable signal with the FT s(v), and suppose that
(a) LnEZ set + nT) is a continuous function, and
(b) LnEZ Is(n/T)I < 00.
Show that, for all t E lR.,
Ls(t+nT)= L s(f)e 2i1C ',ft.

nEZ nEZ
A3
Pointwise Convergence
of Fourier Series
A31 Dini' s and Jordan' s Theorems

The inversion formula for Fourier series obtained in Chapter A2 requires a rather
strong condition of summability of the Fourier coefficients series. Moreover, this
condition implies that the function itself is almost everywhere equal to a continuous
function. In this seetion, the dass of functions for which the inversion formula holds
is extended.
Recall Kolmogorov's negative result (see the Introduction):
THEOREM A3.1. There exists a locally integrable 2rr -periodic function f : ~ ~ ce
for which the Fourier se ries diverges everywhere.
This result challenges one to obtain conditions that a locally integrable 2rr-
periodic function f must satisfy in order for its Fourier series to converge to
f. Recall that the Fourier series associated with a 2rr-periodic locally integrable
function f is the formal Fourier series
(33)
where cn(f) is the nth Fourier coefficient
cn(f) = _1
2rr
j+Jr f(u)e-inu du.
-Jr
(34)
The series (33) is calledformal as long as one does not say something about its
convergence in some sense (pointwise, almost everywhere, in LI, etc). If one has

32 A3. Pointwise Convergence ofFourier Series
no more than the condition that f is 27T -periodic and locally integrable, the worst
can happen, as Kolmogorov's theorem shows.
The purpose of this section is to find reasonable conditions guaranteeing
convergence as n -+ 00 of the truncated Fourier series
+n
sI (x) = L ck(f)e ikx . (35)
k=-n
We have to specify (1) in what sense this convergence takes place and (2) what
the limit iso Ideally, the convergence should be pointwise and to fitself. The next
exercise gives a simple instance where this is true.
EXERCISE A3.1. Assume that the trigonometric series
+n
Sn(t) = L Ck eikt
k=-n
converges uniformly to some function f(t). Show that in this case, for all k E Z,
Ck = ck(f).
Dirichlet's Integral
We will first express the truncated series sI in a form suitable for analysis. For
this we write
sI (x) = L+n { -1 j+Jr .} .

f(s)e- ,ks ds e'kx
k=-n 27T
IL I
-Jr
= - 1 j+Jr +n eik(x-s) f(s)ds.

27T -Jr k=-n
Elementary computations give
+n sin((n + )t)
L eikt =
1
-2
(36)
k=-n sin(t /2)
(the function in the right-hand side is called the Dirichlet kerne!) and therefore,
f 1 j+Jr sin((n + i)(x - s))
Sn (x) = -2
7T -Jr
. (( _ )/2)
sm x s
fes) ds.
Performing the change of variable x - s = u and taking into account the fact that
fand the Dirichlet kernel are 27T-periodic, we obtain
f 1 j+Jr sin((n + !)u)

Sn (x) = -2
7T -Jr
. (/2)
sm u
fex + u)du. (37)
The right-hand side of (37) is called the Dirichlet integral.

A31 Dini's and Iordan's Theorems 33
If we let f(t) =1 in (35), we obtain 1; on substituting this in (37),

-
1 j+Jr sin((n + !)u) du-I.
_
(38)
27f -Jr sin(u/2)
Therefore, for any real number A,
I
S!(x)-A=-
j+Jr sin((n
.
+ ! )u)
2 (f(x+u)-A)du (39)
27f -Jr sm(u/2)
iJr
or, equivalently,
1 sin((n + ! )u)
S!(x) - A = - . 2 {fex + u) + fex - u) - 2A}du. (40)
27f 0 sm(u/2)
Therefore, in order to show that, for a given x E IR, S! (x) tends to A as n -+ 00,
it is neeessary and suffieient to show that the Diriehlet integral in the right-hand
side of (39) eonverges to zero as n -+ 00.
The localization principle states that the eonvergenee of the Fourier series is a
loeal property. More preeisely:
THEOREM A3.2. lf fand gare two locally integrable 27f -periodic complex-valued
functions such that, for a given x E IR and some 8 > 0, it holds that f (t) = g(t)
whenever t E [x - 8, x + 8], then
lim{S!(x) - S!(x)} = O.
ntoo
Proof" Using (39) we have
I j+Jr fex + u) - g(x + u)

s! (x) - S!(x) = -2
7f -Jr
sin((n + !)u) Ilul:::8 . ( /2)
sm u
du
= -
1 j+Jr sin((n + !)u) w(u) du,
27f -Jr
where
fex + u) - g(x + u)
w(u) = l lul ->8 . ( u /2)
sm
is integrable over [0, 27f]. The last integral therefore tends to zero as n -+ 00 by
the Riemann-Lebesgue lemma.
We now state the general pointwise convergence theorem.
THEOREM A3.3. Let f be a locally integrable 27f-periodic complex-valued

function, and let x E IR and A E IR be given. Then
lim S!(x) = A
ntoo
34 A3. Pointwise Convergence of Fourier Series
:s Ti,
.1
if,for some < 8
11m
8
.
smn ~(u)
+ !)u) - - du = 0, (41)
ntoo 0 uj2
where
~(u) = f(x + u) + f(x - u) - 2A. (42)
Proof" Taking ga constant equal to A, we have Sn(g) = A, and therefore we are

looking for a sufficient condition guaranteeing that Sn(f) - Sn(g) tends to as n
tends to 00. By the localization principle, it suffices to show that
lim [8 sinn + !)u) .~(u) du = 0. (43)

ntoo 10 Sln(uj2)
The two integrals in (41) and (43) differ by
1 8
sinn + !)u) v(u)du, (44)
where
v(u) = ~(u) {U~2 - Sin(~j2)}

is integrable on [0, 8]. Therefore, by the Riemann-Lebesgue lemma, the quantity
(44) tends to zero as n --+ 00.
Dini's Theorem

THEOREM A3.4. Let f be a 2Ti-periodic locally integrable complex-valued
function and let x E IR. Iffor some < 8 :s Ti and some A E IR, the function
f(x + t) + f(x - t) - 2A
t --+
is integrable on [0, 8], then
lim
ntoo
S! (x) = A.
Proof" The hypothesis says that the function ~(u)ju, where ~ is defined in (42),
is integrable, and therefore condition (41) ofTheorem A3.3 is satisfied (Riemann-
Lebesguelemma).
We shall give two corollaries ofDini's result.
COROLLARY

A3.1. If a 2Ti -periodie locally integrable complex-valued function
f(t) is Lipschitz continuous of order IX > about x E IR, that is,
If(x + h) - f(x)1 = O(lhIO') as h --+ 0,
then limntoo S! (x) = f(x).

A31 Dini's and Jordan's Theorems 35
Proof Indeed, with A = f(x),

f(x + t) + f(x - t) - 2A I < K _1_
I t - ItI 1 -0:'
for some constant K and for all t in a neighborhood of zero, and 1/ltI 1-0: is
integrable in this neighborhood, because I-ex< 1. Dini's theoremA3.4 concIudes
the proof.
COROLLARY A3.2. Let f(t) be a 21T-periodic locally integrable complex-valued
function, and let x E lR be such that
f(x + 0) = lim f(x + h) and f(x - 0) = lim f(x - h)
hW hW
exist and are finite, and further assume that the derivatives to the left and to the
right at x exist. Then
. SI() _
I1m n X -
f(x + 0) + f( x - 0)

ntoo 2
Prao!" By definition, one says that the derivative to the right exists if
lim f(x + t) - f(x + 0)
ttO t
exists and is finite, with a similar definition for the derivative to the left. The
differentiability assumptions imply that
lim f(x + t) - f(x + 0) + f(x - t) - f(x - 0)
ttO t
exists and is finite and therefore that

tjJ(t) f(x + t) + f(x - t) - 2A
is integrable in a neighborhood of zero, where

2A = f(x + 0) + f(x - 0).
Dini's theorem A3.4 concIudes the proof.
EXAMPLE A3.1. Apply the previous theorem to the 21T -periodic function defined
by
f(t) =t when 0 < t :::; 21T.
Onefinds
"" sin(nt)
t = 1T - ~ 2-- when 0 < t < 21T.
nEZ n
n#O
For t = 0, we can directly check that the sum of the Fourier series is
!(f(O+)+f(O-)) = !(0+21T)=1T,
as announced in the last corollary. For t = n /2, we obtain the remarkable identity
nIl 1 1
4=1-3+:5-7+
Jordan's Theorem
Jordan 's convergence theorem features funetions of bounded variation.
DEFINITION A3.1. A real-valued function q; : lR f-+ lR is said to have bounded
variation on the interval [a, b] C lR if
n-l
sup L !q;(Xi+l) - q;(Xi)! < 00, (45)
'D i=O
where the supremum is over all subdivisions D = {a = Xo < Xl < ... < Xn = b}.
We quote without proof the fundamental result on the strueture of bounded
variation funetions.
THEOREM A3.5. A real-valued function q; has bounded variation over [a, b] if and
only ifthere exist two nondecreasing real-valuedfunctions q;l, q;2 such that,for all
tE [a, b],
q;(t) = q;l (t) - q;2(t). (46)
In partieular, for all X E [a, b), q; has a limit to the right q;(x + 0); for all
X E (a, b], it has a limit to the left q;(x - 0); and the diseontinuity points of q;(t)
in [a, b] form a denumerable set, and therefore a set ofLebesgue measure zero.
THEOREM A3.6. Let f be a 2n-periodic locally integrable real-valuedfunction

of bounded variation in a neighborhood of a given X E lR. Then
lim st (x) = f(x + 0) + f(x - 0) (47)

ntoo 2

The proof is omitted.
EXERCISE A3.2. Let f E L~(lR). Show that, for any B > 0,
I-B B
+ j(v)e2irrvt dv = 2B { f(t
JR
+ s)sine (2Bs) ds,
and use this to study the pointwise convergence of the left-hand side as B tends to
infinity, along the lines of the current chapter.
The funetion
2B sine (2Bt)
is also ealled Dirichlet's kernei.
A31 Dini's and Jordan's Theorems 37
EXERCISE A3.3. Let!t and h be the 2rr -periodic functions defined on (-rr, +rr]
by
!t(x) = x,
Compute their Fourier coefficients, and use this to compute
L (_l)n,
n~l n
Integration of Fourier Series
Let f(t) be a real-valued 2rr-periodic locally integrable function. Denoting by

Cn the nth Fourier coefficient of f(t), we have C n = c~ because f(t) is real.
Therefore, the Fourier series of f(t) can be written as
00
!ao + L{an cos(nx) + bn sin(nx)}, (48)

n=l
where, for n ::: 1,
an = -1
rr
10
2Jr
f(t)cos(nt)dt, bn = -1
rr
1
0
2Jr
f(t)sin(nt)dt.
Of course, the series in (48) is purely formal when no additional constraints are
put on f(t) in order to guarantee its convergence. Now, the function F(t) defined
for tE [0, 2rr) by
F(t) = Iat(f(X) - !ao)dx (49)
is 2rr-periodic, is continuous (observe that F(O) = F(I) = 0), and has bounded
variation on finite intervals.
Therefore, by Jordan's theorem its Fourier series converges everywhere, and for
all x E lR,
00
F(x) = !A o + L{A n cos(nx) + Rn sin(nx)},

n=l
where, for n ::: 1,
An = -1
rr
10
2Jr
F(t) cos(nt) dt
1 [
rr
sin(nx)
=- F(x)--
n
]2Jr
0
-1
nrr
1 0
2Jr
(f(t) - !ao) sin(nt) dt
= - -1
nrr 0
1 2Jr
b
f(t)sin(nt)dt = _..!:,
n
and, with a similar computation,
B n= ..!.. (2n F(t) sin(nt) dt = an

n 10 n
Therefore, for all x E lR,
1
F(x) = zA ~
o+ ~ {an-;; sm(nx) bncos(nx) }.
. - -;; (50)
The constant A o is identified by setting x = 0 in (50):
1 ~bn (51)
zA o = L...- - .
n=l n
Since A o is finite we have shown, in particular, that L~l bn/n converges for any
sequence {b n }n2:1 of the form
bn = ..!.. (2n J(t) sin(nt) dt,

n 10
where, J(t) is areal function integrable over [0, 2n].
Gibbs' Overshoot Phenomenon
We dose this section by mentioning a phenomenon typical of the behavior of a
Fourier series at a discontinuity of the function. Gibbs' overshoot phenomenon
has nothing to do with the failure of the Fourier series to converge at a point of
discontinuity of the corresponding function. It concems the overshoot of the partial
sums at such a point of discontinuity. An example will demonstrate this effect.
I~ ;
Consider the 2n -periodic function defined in the interval ( - n, + n] by
ifx > 0,
J(x) = if x < 0,
n x
---- if x < O.
2 2
The partial sum of its Fourier series is
f _ ~ sin(nx)
Sn (x) - L...- - - .
k=l n
By Dini's theorem, the partial sum sI

(0) converges pointwise to (1/2)(/(0+) +
J(O- = n /2. However, we shall see that for some A > n /2 and sufficiently
large n,
S!(~) ::: A. (52)
Therefore, there exist a constant c > 0 and a neighborhood No of 0 such that

IsI (x) - sI (0)1::: c whenever x E No - {O}. This constitutes Gibbs' overshoot
A32 Fejer's Theorem 39
phenomenon, which can be observed whenever the function has a point of discon-
tinuity. The proof of (52) for this special case keeps most of the features of the
general proof, which is left for the reader. In this special case,
f
Sn (x) = l
o
x sin((n + !)t) dt -
2 sin(!t)
x
-.
2
Now,
l o
x sin((n
2 sin('it)
+ !)t)
1 dt
-_lX -----;----"-- + -21 cos(nt))

(sin(nt)Cos(!t)
dt
o 2sin(!t)
= l
o
x sin(nt)
- - dt
t
+ l 0
x ,
sm(nt)
( cos(!t)
1
2 sin('it)
-
1)
-t dt
+ - ll
2 0
X
cos(nt)dt.
The last two integrals converge uniformly to zero (by the uniform version of the
Riemann-Lebesgue lemma). Also,
1 o
~ sin(nt)
- - dt
tot
= l n sin(t)
- - dt ::::: 1.18 -:rr:rr
> -
2 2
.

A32 Fejer's Theorem
sI
The Fourier (t) series of a 2:rr-periodic locally integrable function I converges
to I(t) for a given t only under certain conditions (see the previous section).
However, Cesaro convergence of the series requires much milder conditions. For
a 2:rr -periodic locally stable function I, Fejer's sum
(53)
behaves more nicely than the Fourier series itself. In particular, for continuous
functions, it converges pointwise to the function itself. Therefore, Fejer's theorem
is a kind of inversion formula, in that it shows that for a large dass of periodic
functions (see the precise statement in Theorem A3.11 ahead), the function can be
recovered from its Fourier coefficients.
Fejer's Kernel
Take the imaginary part of the identity
n-l 1_ inu
' " ei(k+l/2)u = e iu / 2 e .
~ 1- e lU
k=O
to obtain
sI
Starting from Dirichlet's integral expression for (t) [cf, Eq. (37)], we obtain, in
view of the identity just proven, Fejer's integral representation of a! (x),
{+Jr (+Jr
a!(x) = LJr Kn(u)f(x-u)du= LJr Kn(x-u)f(x)du, (54)
where
I sin2(~nt)
Kn(t) = 2 1 (55)
2n:rr sin ('it)
is, by definition, Fejer's kernel. It has the following properties:
(56)
i:
and [letting f(t) = 1 in (54)],
Jr Kn(u) du = 1. (57)
Also (the proofs are left as an exercise),

lim Kn(t) = 1, (58)
ntoo
and, for all e S :rr,
lim
ntoo
j -6
+C
Kn(u) du = 1. (59)
The last four properties make ofFejer's kernel a regularization kerneion [-:rr, +:rr]
(by definition of a regularization kernel).
Cesaro Convergence for Fourier Series of Continuous Functions
We first treat the case of continuous functions, because the result can be ob-
tained from the basic principles of analysis, in particular, without recourse to
the Riemann-Lebesgue lemma.
THEOREM A3.7. Let f(t) be a 2:rr-periodic continuousfunction. Then
lim sup la! (x) - f(x)1 = O. (60)
ntoo XE[-Jr,+Jr]
A32 Fejer's Theorem 41
i:
Proof' From (54) and (56), we have
n
la! (x) - l(x)1 ::s KnCu) I/(x - u) - l(x)1 du
= 1+ +8
-8
[
[-n,+n]\[-8,+8]
=A+B. (61)
For a given 8 > 0, ehoose 8 sueh that I/(x - u) - l(x)1 ::s 8/2 when lul ::s 8.
Note that I is uniformly eontinuous and uniformly bounded (being aperiodie and
eontinuous funetion), and therefore 8 ean be ehosen independently of x. We have
A::s
81+8 Kn(u)du ::s 2'8
2 -8
and, ealling M the uniform bound of I,
B ::s 2M [ Kn(u) du.

J[-n,+n]\[ -8,+8]
By (57) and (59), B ::s 8/2 for n sufficiently large. Therefore, for n suffieiently
large, A + B ::s 8.
Fejer's theorem for eontinuous periodie funetions is the key to important ap-
proximation theorems. The first one is for free. We eall a trigonometrie polynomial
any finite trigonometrie sum of the form
L
+n
p(x) = Ck eikx .
-n
THEOREM A3.8. Let I (t) be a 2n -periodic continuous function. Select an arbitrary

8 > O. Then there exists a trigonometric polynomial p(x) such that
sup I/(x) - p(x)1 ::s 8.

tE[-n,+n]
Proof' Use Theorem A3.7 and observe that a! (x) is a trigonometrie polynom-
iill.
From this, we obtain the Weierstrass approximation theorem.
THEOREM A3.9. Let I : [a, b] 1-+ C be a continuousfunction. Select an arbitrary
8 > O. There exists a polynomial P(x) such that
sup I/(x) - P(x)1 ::s 8.

tE[a,b]
Jf, moreover, I is real-valued, then P can be chosen with real coefficients.
Proof' First, suppose that a = 0, b = 1. One ean then extend I : [0, 1]] 1-+ C
to a funetion still denoted by I, I : [-n, +n]] 1-+ C, that is eontinuous and
sueh that IHn) = I( -n) = O. By Theorem A3.8, there exists a trigonometrie
polynomial p(x) such that

e
sup If(x) - p(x)l:s sup If(x) - p(x)1 :s -.
tE[D, I] tEl -rr,+rr] 2
Now replace each term e ikx in p(x) by a sufficiently large portion of its Taylor
series expansion, to obtain a polynomial P(x) such that
e
sup IP(x) - p(x)1 :s -.
tE[D,I] 2
Then
e
sup If(x) - P(x)1 :s -.
tE[D, I] 2
To treat the general case f : [a, b] f-+ C, apply the result just proven to cp
[0, 1] f-+ ce defined by cp(t) = f(a + (b - a)t) to obtain the approximating
polynomial rr(x), and take P(x) = rrx - a)j(b - a.
Finally, to prove the last statement of the theorem, observe that
If(x) - Re P(x)1 :s If(x) - P(x)l
Fejer's Theorem
We shall first obtain for the Fejer's sum the result analogous to Theorem A3.3.
First, from (54), we obtain
a!(x) = -1 irr sin. 222(!nu) {f(x+u)-f(x-u)}du; (62)

1
2nrr D sm Czu)
therefore, for any number A,
a!(x)-A= -
1 irr sin. 222( !nu) {f(x+u)+f(x-u)-2A}du. (63)
1
2nrr D sm (:zu)
THEOREM A3.10. For any x E IR. and any constant A,
lima!(x) = A (64)
ntoo
if, for some 8 > 0,
. 1
11m
ntoo
-
n
1 D
8 . 4J(u)
sm 2(!nu) - 2 -
u
du = 0, (65)
where
4J(u) = f(x + u) + f(x - u) - 2A. (66)

Proof The quantity
!
In 18
r sin 2(!nu) 4J(U) du l < ! r 14J(u)1 du
sin 2 (!u) - n 18 sin 2 (!u)
t
1
tends to 0 as n 00. We must therefore show that
1 8 sin 2( !nu)
_ 2 fjJ(u) du
n 0 sin 2 (!u)
tends to 0 as n t 00. However, (65) guarantees this because
:'S -
n
11 (10
8
. 2( 1
SIll ZU
) - 1I
12 IfjJ(u)1 du
ZU
tends to 0 as n t 00 (the expression in curly brackets is bounded in [0, 8], and

therefore the integral is finite).
THEOREM A3.11. Let f (t) be a 2JT -periodic locally integrable function and assume
that, for some x E ~, the limits to the right and to the left (respectively, f(x + 0)
and f(x - 0), exist. Then
lim u! (x) = - 0) . f(x + 0) + f(x (67)

2
Proof" Fix 8 > O. In view of the last result, it suffices to prove (65) with
fjJ(u) = {f(x + u) - f(x + O)} + {f(x - u) - f(x - O)}.
Since fjJ(u) tends to 0 as n ~ 00, for any given c > 0 there exists rJ = rJ(c),
o< rJ :'S 8, such that IfjJ(u)1 :'S c when 0 < u :'S rJ. Now,
1
11
-
n 0
8 sin2(!nu)
u
; fjJ(u) du
1
< -
c 1ry sin 2(!nu)
du + -1 1 -IfjJ(u)1-
8
du.
n 0 u2 n ~ u2
The last integral is bounded; and therefore, the last term goes to 0 as n t 00. As
for the penultimate term, it is bounded by Ac, where
A = 1o
00 sin2(!v)
v
2 dv < 00.
A33 The Poisson Formula
The following corollary of F6jer's theorem will play the key role for the proof of
the Poisson sum formula (Theorem A3.l2).
COROLLARY A3.1. Let f be a 2rc -periodie locally integrable function and suppose
that, for some x E lR.,
(a) thefunction f is continuous at x, and

(b) its Fourier se ries sI (x) converges to some number A.
Then A = fex).
Proof' From (b) we see that
lim
ntoo
0'1 (x) = A.
From F6jer's theorem and (a),
lim
ntoo
0'1 (x) = fex).
We have already given a weak version of the Poisson sum formula in Section
A22. A most interesting situation is when the function cI>(t) defined by (31) is
equal to its Fourier series for all t E lR., that is,
Ls(t + nT) = ~ LS(f) e2in !ft for all tE R (68)

nEZ nEZ
The next theorem extends the result in Exercise A2.8.
THEOREM A3.12. Let set) be a stable complex signal, and let 0 < T < 00 be
fixed. Assume in addition that
(1) LnEZ set + n T) converges everywhere to some continuous function,

(2) LnEZ s( f) e2in !ft converges for all t.
Then the strang Poissonformula (68) holds.
Proof' The result is an immediate consequence of both the weak Poisson sum-
mation result (Theorem A2.3) and the corollary of F6jer's theorem in Section
A32.
Here are two important cases for which the strong Poisson sum formula holds.
COROLLARY A3.1. Let set) be a stable complex signal, and let 0 < T < 00 be
fixed. If, in addition, L set + nT) converges everywhere to a continuousfunction
that has bounded variation, then the Poissonformula (68) holds.
Praof' We must verify conditions (1) and (2) ofTheorem A3.12. Condition (1)
is part of the hypothesis. Condition (2) is a consequence of Iordan's theorem
A3.6.
EXAMPLE A3.1. If set) is continuous, has bounded support, and has bounded
variation, the Poisson sumformula (68) holds.
COROLLARY A3.2. If a stable continuous signal s(t) satisfies
s(t) = 0(1 +\tl"') as Itl ~ 00,

s(v) = oe +llvl"') as lvi ~ 00,
(69)
for some a > 1, then the Poisson formula (68) holds for alt < T < 00.
Proof The result is an immediate corollary of Theorem A3.12.

Convergence Improvement
The Poisson formula can be used to replace aseries with slow convergence by one
with rapid convergence, or to obtain some remarkable formulas. Here is a typical
example. For a > 0,
s(t) = e-2naltl ~ s(v) = a

n(a 2 + v2 ) .
Since
Ls(t +n) = Le-2nalt+nl
nEZ nEZ
is a continuous function with bounded variation, we have the Poisson formula, that
is,
The left-hand side is equal to

2
-1,
1 - e- 2na
and the right-hand side can be written as
Therefore,
1 n 1 + e- 2na 1
L a 2 + n 2 = 2a 1 - e- 2na 2a 2 .
nO": 1
Letting a ~ 0, we have
The general feature of the above example is the following. We have aseries
that is obtained by sampling a very regular function (in fact, C OO ) but also slowly
h(t-2T)
~ 1
[jU[jU[j[j[j~
-3T -2T -T 0 T 2T 3T
-3T -2T -T O T 2T 3T
Figure A3.1. Radar return signal
decreasing. However, because of its strong regularity, its Fr has a fast decay. The
series obtained by sampling the Fr is therefore quickly converging.
Radar Return Signal
Let s(t) be a signal ofthe form
s(t) = (I>(t -
nE'L
nT) f(t). (70)
(We may interpret h(t - nT) as a return signal of the nth pulse of a radar after
reftection on the target, and f (t) as a modulation due to the rotation of the antenna. )
The Fr ofthis signal is (see Fig. A3.1)
s(v) = ~ Lh(!!.-) f(v - !!.-). (71)

T nE'L T T
EXERCISE A3.4. Show that if(1) f (t)
is integrable, (2) LnE'L h(t -n T) is integrable
and continuous, and (3) LnE'L h(n/T) < 00, then (71) holds true. Find other
conditions.
References
[Al] Ablowitz, M.J. and Jokas, A.S. (1997). Complex Variables, Cambridge University
Press.
[A2] Bracewell, R.N. (1991). The Fourier Transform and Its Applieations, 2nd rev. ed.,
McGraw-Hil1; New York.
[A3] Gasquet, C. and Witomski, P. (1991). Analyse de Fourier et Applieations, Masson:
Paris.
[A4] Helson, H. (1983). Harmonie Analysis, Addison-Wesley: Reading, MA.
[A5] Katznelson, Y. (1976). An Introduetion to Harmonie Analysis, Dover: New York.
[A6] Kodaira, K. (1984). Introduetion to Complex Analysis, Cambridge University Press.
[A7] Krner, T.W. (1988). Fourier Analysis, Cambridge University Press.
References 47
[A8] Rudin, W. (1966). Real and Complex Analysis, McGraw-Hill: New York.
[A9] Titchmarsh, E.C. (1986). The Theory of Funetions, Oxford University Press.
[AlO] Tolstov, G. (1962). Fourier Series, Prentice-Hall (Dover edition, 1976).
[All] Zygmund, A. (1959). Trigonometrie Series, (2nd ed., Cambridge University Press.
Part B
Signal Processing
Introduction
The Fourier transform derives its importance in physics and in electrical engineer-
ing from the fact that many devices mapping an input signal x(t) into an output
signal y(t) have the following property: If the input is a complex sinusoid e2invt,
the output is T(v)e2invt, where T(v) is a complex function characterizing the de-
vice. For example, when x(t) and y(t) are, respectively, the voltage observed at the
input and the steady-state voltage observed at the output of an Re circuit (see Fig.
BO.I), the input-output mapping takes the form of a linear differential equation:
y(t) + RCy(t) = x(t),

and it can be readily checked that
I
T(v) =
1+ 2irrvRC
The Re circuit is one of the physical devices that transform a signal into another
signal, that satisfy the superposition principle, and that are time-invariant. More
precisely:
1'~1'
x(t)
C y(t)
l""" ,1,,)
Figure BO.I. The Re circuit
52 Part B Signal Processing
(a) If Yl (t) and Y2(t) are the outputs corresponding to the inputs Xl (t) and X2(t),
then AlYl (t) + A2Y2(t) is the output corresponding to the input AlXl (t) + A2X2(t);
(b) If y(t) is the output corresponding to x(t), then y(t - -r) is the output
corresponding to x(t - -r).
Such physical devices are caIled (homogeneous linear) filters.
A basic example is the convolutional filter, for which the input-output mapping
takes, in the time domain, the form
y(t) = 1 h(t - s)x(s)ds,
where h(t) is caIled the impulse response, because it is the response of the filter
when the Dirac pulse 8(t) is applied at the input. Indeed,
h(t) = 1 h(t - s)8(s) ds.
If the impulse response is integrable, the output is weIl defined and integrable, as
long as the input is integrable. Then, by the convolution-multiplication rule, the
expression of the input-output mapping in the frequency domain is
y(v) = T(v)x(v),
where T (v) is the frequency response, that is, the Fr of the impulse response:
T(v) = 1 h(t)e-2i1Cvt dt.
Observe that if the input is x(t) = e-2i1Cvt, the output is weIl defined and equal to
1 h(s)x(t - s)ds = 1 h(s)e- 2i1CV (t-s)ds = T(v)e-2i1Cvt,

in accordance with what was said in the beginning of this introduction.
In the particular case of the RC filter, the solution of the differential equation
with arbitrary initial condition at -00 is indeed of the convolution type, with the
impulse response
The RC filter is a convolutional filter, and it contains the typical features of

the more general filters. In the general case, since a filter is a mapping, we shall
have to define its domain of application. Depending on this domain, the input-
output mapping takes different forms. In the above informal discussion of the
RC circuit, there are a frequency-domain and a time-domain representation and
also a representation in terms of a linear homogeneous differential equation. The
latter is not general. In fact, when it is available, the transmittance is a rational
function ofthe frequency v. The corresponding filters, caIled rational filters, form
an important class, and Chapter BI gives the basic concepts concerning analog
(that is, continuous-time) filters.
Introduction 53
In addition to filtering, there are two fundamental operations of interest

in communications systems: frequency transposition and sampling. Frequency
transposition is a basic technique of analog communications. It has two main ap-
plications, the first of which is transmission. Indeed, the Hertzian channels are in
the high-frequency bands-in fact, much higher than the one ofbrute signals such
as the electric signals carrying voice, for instance-and consequently, the latter
have to be frequency-shifted. The second reason is resource utilization and is re-
lated to frequency multiplexing, a technique by which signals initially occupying
the same frequency band are shifted to nonoverlapping bands and can then be sim-
ultaneously transmitted without mutual interference. From a mathematical point of
view the theory of frequency transposition (or, equivalently, of band-pass signals,
to be defined in Chapter BI) is not difficult. It remains interesting because of the
special phenomena associated with this technique, such as cross-talk in quadrature
multiplexing and dispersion phenomena.
In digital communications systems, an analog signal s(t) is first sampled, and the
result is a sequence of sampies s(n T), n E Z. It is important to identify conditions
under which the sampie sequence faithfully represents the original signal. The
central result of Chapter B2 is the so-called Shannon-Nyquist theorem, which
says that this is true if the signal s(t) is stable and continuous and if the support
of its Fr s( v) is contained in the interval [-1/ T, + I/Tl. The original signal can
then be recovered by the reconstruction formula:
s(t) = L s (nT) sinc (f - n) .

nEZ
The theory of sampling is an application of the results obtained in Part A, and

in particular of the Poisson sum formula. The above reconstruction formula has
many sourees, 1 and its importance in communications was fully realized by Claude
Shannon and Harald Nyquist.
The Shannon-Nyquist sampling theorem is the bridge between the analog
(physical) world and the discrete-time (computational) world of digital signal pro-
cessing. The reader will find in the main text abrief discussion of the interest of
digital communications systems. Therefore, a large portion of this Part B is devoted
to discrete-time signals (Chapters B2-B4).
As we have already mentioned, the Poisson sum formula is the key to the sam-
pling theorem. It also plays a very important role in the numerical analysis of
the discrete Fourier transform considered as an approximation of the continu-
ous Fourier transform (see Chapter B3) and also in the intersymbol interference
problem (see Chapter B2).
The study of the interaction between discrete time and continuous time is not
limited to the sampling theorem. For instance, we prove that filtering and sampling
ISee J.R. Higgins, Five short stories about the cardinal series, Bult. Amer. Math. Soc.,
12, 1985,45-89.
54 Part B Signal Processing
cmumute for base-band signals. This is not a difficult result, but it is of course a
fundamental one because in signal processing, one first sampies and then performs
the filtering operation in the sampled domain, since one of the advantages of digital
processing comes precisely from the difficulty of making analog filters.
One advantage of analog processing is that it is instantaneous. To maintain
competitivity, the signal processing algorithms have to be fast. For instance, the
discrete Fourier transform is implemented by the so-called fast Fourier transform,
an algorithm whose principle we briefty explain in this Part. Subband coding also
has a fast algorithm associated with it. It is a data compression technique. The
signal is not directly quantized, but instead, it is first analyzed by a filter bank, and
the output of each filter bank is quantized separately. This allows one to dispatch
the compression resources unequally, with fewer bits allocated to the subbands
that are less informative (see the discussion in Chapter B4). Subband coding is
the last topic of Part Band introduces the sections on multiresolution analysis in
Part D.
BI
Filtering
B 11 Impulse Response and Frequency Response

Convolutional Filter
We introduce a particular and very important dass of filters.
DEFINITION Bl.l. The transformation fram the stable signal x(t) to the stable
signal y(t) defined by the convolution
y(t) = 1h(t - s)x(s)ds, (1)
where h(t) is stable, is ca lied a convolutional filter. This filter is called a causal
filter if h(t) = 0 for t < O.
The signal y(t) is the output, whereas the signalx(t) is the inputofthe linear filter
with impulse response h(t). Informally, if x(t) is the Dirac generalized function
8(t) (an impulse at time 0), the output is (see Fig. Bl.1)
1 h(t - s)8(s) ds = h(t),
whence the terminology.

A causal filter responds only after it has been stimulated. For this reason, it
is sometimes also called a realizable filter (Fig. B 1.1. features a causal impulse
response). For such filters, the input-output relationship (1) becomes (note the

56 BI. Filtering
8(t) I f\h(t~
> >
0 0 V
impulse impulse response
Figure B 1.1. Impulse and impulse response
upper limit of integration)
y(t) = [t
oo
h(t - s)x(s) ds. (2)
B1.2. The Fourier transform ofthe (stable) impulse response h(t),
L
DEFINITION
T(v) = h(t)e-2i1Cvt dt, (3)
is ca lied the frequency response.

If the input is the complex sinusoid x(t) = e2i1Cvt, by (1), the output is
y(t) = T(v)e2i1Cvt. (4)
(Note that the output is weH defined by the convolution formula, even though in
this particular case the input is not integrable.)
EXERCISE Bl.1. Let y(t) be the output of a stable and causal convolutional filter
with impulse response h(t) [see (2)]. Let
z(t) = 1 t
h(t - s)x(s)ds, t ~ 0,
be the output of the same filter, when the input x(t) is applied only from time t = 0
on. Show that
lim Iz(t) - y(t)1
tt+oo
= O.
A More General Definition

Convolutional filters are only a special dass of filters. A more general definition
is as foHows. Denote by C ~ the set of functions of lR. into C.
DEFINmON Bl.3. Let D(12) be a set of functions from lR. into C with the two
following properties:
(a) It is closed under linear operations;
() it is closed under translation.
12 : D(12) 1-+ C ~ is ca lied a homogeneous linear filter with domain D(12) if:
(i) 12 is linear, and
(ii) 12 is time-invariant.
The meaning ofproperties (a) and () is the following: (a) XI (t), X2(t) E D('c),
AI, A2 E C ===} AIXI (t) + A2X2(t) E D('c); and () x(t) E D('c), T E lR ===}
x(t - T) E D('c).
The meaning of properties (i) and (ii) is the following: (i) XI (t), X2(t) E D('c),
c c C
AI, A2 E C,XI(t)~ YI(t),X2(t) ~ Y2(t) ===} AIXI(tHA2X2(t) ~ AI(t)YI(tH
A2Y2(t); (ii) x(t) E D('c), T E lR, x(t) ~ y(t) ===} x(t - T) ~ y(t - T).
EXERCISE B1.2. Show that if e 2i :rcvt E D('c), then

e 2mvt C
~ T(v)e 2mvt (5)
for some complex number T(v).

The function T(v) is called thefrequency response ofthe filter. Every frequency
response is of the form
T(v) = G(v)ei(v), (6)
where G(v) = IT(v)1 is the amplitude gain and (v) = Arg T(v) is the phase.
EXAMPLE Bl.l. Let
D('c) = {x(t) : L Ix(t)1 dt < oo},
or
D('c) = {x(t) + e(t) : L Ix(t)1 dt < 00 and e(t) E E},
where E is the set of complex finite linear combinations of complex exponentials.

For any signal in D('c), the right-hand side of (1) is welt defined, and we can
therefore define the filter ,C with domain D('c) by the input-output relationship
(1). Thefrequency response, as defined by (5), is then the FT ofh(t).
EXAMPLE B1.2. Let h(t) E L~(lR). We shalt see in Part C that the FT h(v) = T(v)
of h(t) can be defined and that it is in L~(lR). We take D('c) = LUlR) and define
,C by the input-output relationship
y(t) = L T(v)x(v)e 2i :rcvt dv, (7)
where xCv) is the FT ofthe input x(t) E L~(lR). The right-hand side of(7) has a
meaning since T(v) and xCv) being in L~(lR) implies that T(v)x(v) is in Lt(lR)
(see Theorem 20 of the appendix).
EXAMPLE B 1.3. If T ( v) is an arbitrary function, not necessarily in L~ (lR), one can
always define a filter ,C by the input-output relation (7), provided one chooses for
domain D('c) the set of signals x(t) such that the right-hand side has a meaning.
58 B 1. Filtering
-B o +B
Low-pass (B)
~2B ----7 ~2B ----7
-VQ o +VQ
Band-pass (vQ, B)
Figure B 1.2. Low-pass and band-pass frequency responses
Low-Pass, Band-Pass, and Hilbert Filters

The low-pass and band-pass filters (see Fig. B1.2) that we now define belong to
the category of Example B 1.2.
One calls low-pass (B) a filter with frequency response
T(v) = 1[-B,+B](v), (8)
where B is the cut-offfrequency. One calls band-pass (B, va), where 0< B < va,
a filter with frequency response
T(v) = 1[-vo-B,-vo+Bj(v) + l[vo-B,vo+Bj(v), (9)

where Va is the center frequency, and 2B is the bandwidth.
Hilbert's filter (see Fig. B 1.3) belongs to the category of Example B 1.3. It is the
filter with frequency response
T(v) = i sgn (v), where T(O) = O. (10)
One possible domain for Hilbert's filter is the set of stable (resp., finite-energy)
signals whose FT has compact support. The amplitude gain of Hilbert's filter is 1
I
(except for v = 0, where the gain is zero), and its phase is
Jr /2 if v > 0,
(v) = 0 if v = 0, (11)
-Jr /2 if v < O.
+i r - ,- - - - - -
,0
- - - - - - - ' , -i
Hilbert filter
Figure B1.3. Hilbert frequency response

There is no function admitting the frequency response (10). There is, in fact,
a generalized function (in the sense of the theory of distributions) with Fr equal
to T(v). However, in signal theory, the Hilbert filter is used only in the theory of
band-pass signals (see Section BI2). For such signals the Hilbert filter coincides
with a bona fide convolutional filter:
EXERCISE Bl.3. Show that the output y(t) ofthe Hilbertfilter, corresponding to
a stable signal x(t) having an FT x(v) that is null outside the frequency band
[- B, + B], can be expressed as
y(t) =- 1R.
x(t - s)
2 sin 2 (n Bs)
ns
ds.
Differentiation and Integration as Filters

Let D('c) be the set of signals
x(t) = L x(v)e2iJrvt dv, (12)
where
L Ix(v)1 dv < 00 and L Ivllx(v)1 < 00.
Such signals are continuous and differentiable with derivative
~ x(t) =[(2inv)x(v)e2iJrvt dv. (13)

dt JR.
(Apply the theorem of differentiation under the integral sign; 15 ofthe appendix).
The mapping x(t) ~ dx(t)/dt is a linear filter, called the differentiating filter, or
differentiator, with frequency response
T(v) = 2inv. (14)
Let D('c) be the set of signals of the form (12), where
~ Ix(v)1 dv < 00 and [lx(v)1 dv < 00.

JIR. JIR lvi
The signal
y(t) = [
x.(v) e2iJrvt dv
2mv JIR
is in the domain of the preceding filter (the differentiator), and therefore,
~ y(t) = [ x(v)e2iJrvt dv = x(t).

dt JIR
The transformation x(t) ~ y(t) is a homogeneous linear filter, which is called the
integrating filter, or integrator, with frequency response
1
T(v) = - . - . (15)
2mv
60 BI. Filtering
y(t) .c2 *.c1 series
x(t) y(t) .c 2 +.c l parallel
x(t)
~----,-----:~ y (t)
Figure BlA. Series, parallel, and feedback configurations
Series, Parallel, and Feedback Configurations

We now describe the basic operations on filters (see Fig. B1.4). Let C] and C2
be two convolutional filters with (stable) impulses responses h](t) and h 2 (t) and
frequency responses T](v) and T2(V), respectively.
The series filter C = C 2 * C] is, by definition, the convolutional filter with
impulse response h(t) = (h] * h 2)(t) and frequency response T(v) = T](v)T2(V).
It operates as folIows: The input x(t) is first filtered by Cl, and the output of C] is
then filtered by C 2 , to produce the final output y(t).
The parallel filter C = C] + C 2 is, by definition, the convolutional filter with
impulse response h(t) = h](t) + h 2(t) and frequency response T(v) = T](v) +
T2(V). It operates as folIows: The input x(t) is filtered by Cl, and "in parallel," it
is filtered by C 2 , and the two outputs are added to produce the final output y(t).
The feedback filter C = CI/(1 - C] * C 2 ) is, by definition, the convolutional
filter with impulse response frequency response
T(v) = T](v)
1 - T](v)T2 (v)
This filter will be a convolutional filter if and only if this frequency response is the
PT of a stable impulse response. 1fthis is not the case, one may define the feedback
filter by the input-output relation
A T](v) A
y(v) = 1 _ T](v)T2 (v)x(v)

with, for instance, a definition along the lines of Example B 1.2.
The filter 'cl is the forward loop filter, whereas 'cl is the feedback loop filter.
The forward loop processes the total input, which consists ofthe input x(t) plus
the feedback input, that is, the output y(t) processed by the feedback loop filter.
EXERCISE B1.4. Consider the function

1
T(v) = 1 + 4n2v2'
Give the impulse response of the convolutional filter with the above jrequency
response T (v). Interpret the filter as a feedback filter.
Filtering of Decomposable Signals
We introduce the notion of a decomposable signal, because it allows one to rewrite

the results conceming Fourier transforms and Fourier series in a unified man-
ner, without recourse to symbolic expressions in terms of the Dirac generalized
functions or, more generally, to the theory of distributions.
DEFINITION Bl.4. The signal set) is called decomposable ifit can be put into the
form
set) = l e2i :rr:vt p,(dv), (16)
where p, is a complex measure on IR. whose total variation 1p,1 isfinite.

We recall that a complex measure of finite total variation is, by definition, a
mapping /L : B(IR.) ~ <C (B(IR.) is the Borel sigma-field on IR.; see the appendix)
ofthe form
where /LI and /L2 are signed measures of finite total variation. A signed measure
of finite total variation is a mapping /L : B(IR.) ~ IR. of the form
/L(C) = /L1(C n A) - /L2(C n ),
for some A E B(IR.) and all C E B(IR.), where /LI and /L2 are measures on (IR., B(IR.))
of finite total mass.
EXAMPLE Bl.4. Let set) be a periodic signal with period T that is stable over its
period and whose Fourier coefficients satisfy the condition
L ISnl < 00.

nEZ
Denote the Dirac measure at the point a by 8a (dv) and set
p,(dv) = LSn8q,(dv).
nEZ
62 BI. Filtering
This measure is signed, has total variation LnEZ ISn I < 00, and since
LSne2inyt = ( e2invt fl(dv),

nEZ lIR
we seefrom the inversionformula that (16) holds; therefore, set) is decomposable.
The measure fl appearing in (16) will be called the spectral decomposition of
the signal set).
THEOREM BI.I. Let x(t) be a decomposable signal with spectral decomposition
L
/Lx:
x(t) = e2invt flxCdv), (17)
and let he,) be the impulse response, assumed stable, of a convolutionalfilter:F.

The integral on the right-hand side of
y(t) = L h(t - s)x(s)ds
is weil defined, and the spectral decomposition ofy(t) is

fly(dv) = T(v)flx(dv). (18)
Proof
LL Ih(t - s)lle2invsllflxl (dv)ds ~ (L Ih(t)1 dt) Iflxl(lR) < 00,
and hence
L Ih(t - s)llx(s)1 ds < 00.
On the other hand,
y(t) = L
(h(t - s) L e2invs flxCdV )) ds
= L
e 2inv(t-s) (L e- 2inv(t-s)h(t - s) dS) flxCdv),
andhence
(19)

EXAMPLE BI.5. In light of(18), one can interpret Eq. (22) ofTheorem A2.I:
Yn = h
A A(n)A
T x n,
where h(t) is a stable impulse response and x(t) is a locally stable periodic signal
withperiod T: IfLnEZ l.inl < 00, then LnEZ IYnl < 00, since h(v) is bounded.
The two signals x(t) and y(t) are therefore decomposable, and (19) can be written
lLy(dv) = LYneq,(dv) = LXnh(f) eq,(dv)

nEZ nEZ
= h(v)ILAdv) = T(v)iLAdv).
Rational Filters as Differential Equations

The RC and LRC circuits are well-known filters, and they belong to the class of
analog rational filters, which we proceed to define fonnally.
Let
p q
P(z) = ao + Latzt, Q(z) = bo + Lb1z l (20)
1=1 1=1
be two polynornials in the complex variable z. The coefficients ap and bq are

nonzero, so that the degree of P is p, and the degree of Q is q. Moreover, we
assurne that P(z) does not have purely imaginary roots:
P(z) has no roots in iR (21)
For all v E IR, define

Q(2inv)
T (v) = -P-(2-in-v-) . (22)
We define a linear time-invariant filter CC, D(I: as follows. First, we define the
domain
D(I:) = {x(t);x(t), xCv), and vqx(v) E L~(IR)}. (23)
We first observe that any function x(t) in the domain is differentiable up to order
q and that its jth derivative is
x(j)(t) = L (2inv)j x(v)e2irrvt dv. (24)
We now define the application itself:
I: : x(t) ~ y(t) = L T(v)x(v)e2irrvt dv. (25)
One has to verify that the integral in (25) is weIl defined. Indeed, I/I P (2i n v) I is
bounded because P(z) has no imaginary root. In particular,
IT(v)llx(v)1 .:::: KIQ(2inv)llx(v)1 for some K < 00.
Therefore, IT(v)x(v)1 is integrable for all x(t) E D(I:), and the integral in (25) is
weIl defined for all such x(t).
In fact, T(v)x(v)v k is integrable for all k, 0 .:::: k .:::: p. To check this, observe
that Ivl k /IP(2inv)1 is bounded for all k .:::: p because P(2inv) is bounded away
64 BI. Filtering
from zero (P(z) has no imaginary root) and Ivl k /IP(2iJTv)1 behaves as Ivl k - p at
00. Therefore, the output y(t) is differentiable up to order p, and for j :::s p,
y<j)(t) = L (2iJTv)jT(v)x(v)e2invt dv. (26)
From (24), (26), and (22), it follows that the input x(t) and the output y(t) are
linked by the differential equation
+L = box(t) + L
p q
aoy(t) aly(I)(t) b1x(l)(t),
1=1 1=1
that is, symbolically,
p(:t ) y(t) = Q(:t ) x(t). (27)
This is the time-domain relation corresponding to the frequency-domain relation

P(2iJTv).Y(v) = Q(2iJTv)x(v).
Differential Equations as Rational Filters

We now consider the inverse problem: Let x(t) E D(L:)-in particular, x(t) is
differentiable up to order q -and let y(t) be a solution ofthe differential equation
(27). Is it possible to express this solution as
( Q(2iJTv) x(v)e2invt dv ?
Ai P(2iJTv)
The answer is "no, in general" and "yes, asymptotically" if we impose the following
condition:
P(z) is strict1y stable, (28)
that is, the real parts of all the roots of P(z) are strict1y negative. Then, y(t), t ~ to,
the solution of (27) with arbitrary initial conditions y(to) = Yo, Y6j)(to) = Y6 j )
(1 :::s j :::s p - 1), satisfies
ttoo
( l
lim y(t) -
IR
Q(2iJTv)
.
P(2m v)
x(v)e 2lJrvt dv
A )
= O. (29)
Proof" The general solution of (27) is the sum of a particular solution of (27) and
of the general solution of the differential equation without a right-hand side,
p(:t ) y(t) = O. (30)
Therefore, since
( Q(2iJTv) x(v)e2invt
JIR P(2iJTv)
is a particular solution of (27), we have to show that limttoo z(t) = 0 for the gen-
eral solution of (30). This follows from the theory of linear differential equations
because the characteristic polynomial P(z) of (30) has all its roots in the open left
half complex plane (see [B5]).
If P(z) is not strictly stable, there are initial conditions such that
Q(2inv) ~ .
y(t) - ImIR .
P(2mv)
x(v)e217fvt dv
does not tend to 0 as t -+ 00.
EXAMPLE Bl.6. Consider the LRC circuit (see Fig. BI.5). Its input and output
are related through the differential equation
LCy(T) + RCj(t) + y(t) = x(t),
where jet) and y(t) are thefirst and second derivatives ofy(t). The roots ofthe
characteristic polynomial
P(z) = 1 + RCz + LCz 2

are given by the formula
-R JR2 - 4L/C
z = ----'--,----'--
2L
and their real parts are always strictly negative. Therefore, the system is strictly
stable, and the permanent regime when the input is x(t) E D(C) is
( 1 2'
y(t) = JlR 1 + RC(2inv) + LC(2inv)2 x(v)e !7fvt dv.
We note that Q(z) == 1 in this example, and therefore,
D(C) = {x(t) : x(t), xCv) E L~(lR)}.
Rational Filters as Convolutional Filters
Going back to the general case described by Eqs. (20)-(25), we pose the problem:
Is the filter of convolutional type? The answer is yes if and only if q < p. Indeed,
consider the factorization of P(z):
P(z) = ap n
k=l
r
(z - Zk)m k ,
R L
1'~1'
x(t) c 1- y(t)
l111111111111I11111
Figure B 1.5. The LRC circuit

66 BI. Filtering
where Zk is a root of order mk. We have the decomposition
and therefore,
1 -00
0 tj- 1 .
- - - eZkte-217rvt
(j - I)!
dt =-
1
(2inv - Zk)1
..
1 o
00 tj-l
_ _ _ ezkte-217rvt
(j - I)!
.
dt =+
(2inv - Zk)1
..
(Remember that the case Re (Zk) = 0 has been exc1uded.) Defining
'""' ~
k tj - e Zkt
1 j
for t < 0,
~ ~('-1)'
k;Re(zkl>O j=l } .
h(t) = (31)
we have
Fr
r mk
h(t) -+
L L (2inv
k=l j=l
k~ Zk)j .
The input-output mapping is therefore
q-p (
y(t) = LCXkX(k)(t) + J.IR h(t - s)x(s)ds. (32)
k=O lR
In particular,
y(t) = 1
h(t - s)x(s) ds (33)
if and only if q < p. If, moreover, P(z) is strict1y stable (no roots in the c10sed
right half-plane), then, as the expression (31) of the impulse response shows, the
impulse response is causal, and the filter is then called realizable.
EXERCISE BI.5. Give the impulse response of the LRC filter in the case R 2 <
4LjC.
We observe that the input-output relationship (32) is meaningful for all x(t) E
D(,C'), where D(,C') consists of the stable complex signals that are differentiable
up to order max(O, q - p). Therefore, one can consider that the filter (C', D(C',
where L' is described by (32), is an extension ofthe original filter (C, D(C. We
may consider that (32) is an extension of the differential equation (27). For some
functions of the extended domain, the input-output relationship is not a differential
equation.
EXERCISE Bl.6. Give the extendedfiltercorresponding to the differential equation
y" - ~y
2
= x" - ~x
3 .
Butterworth Filters
We consider the problem of implementing an approximation of the ideal low-

pass (B) filter (with cut-off frequency B) by means of a stable and realizable
filter with real impulse response h(t). Let T(v) be the frequency response of the
approximating filter. The following family of filters, called Butterworth filters, has
been proposed:
1
(ir'
(34)
1+
(As n -+ 00, the filter looks more and more like an ideallow-pass filter.) One
seeks T (v) of the form
K
T(v) = P(2inv)'
where P(z) has all its roots strictly to the left of the imaginary axis in order to
guarantee stability and causality.
The roots of the polynomiall + (v / B)2n are
O::;l::;2n+1.
We reorder these roots in such a way that VI, vi, ... , vn , v~, are the 2n roots, where
VI, .. , Vn have strictly positive imaginary parts. We shall allocate VI, . , Vn to
T(v), thus proposing
T(v) = (35)
This is the frequency response of a real filter (i.e., T*(v) = T( -v because any
root among VI, ... , Vn is purely imaginary or it can be associated with another root
symmetrie with respect to the imaginary axis (see Fig. B 1.6).
In the case n = 2, we find
68 BI. Filtering
1/1 1/0
,, /
,, /
/
\ /
, / ........... \ I .........
,,
/ ... /
B .... I \ ....
,,
/
/ ...
/ / \
,
/
/
/
/
/
1/* 1/*
1 o
I/i
n=2 n=3
Figure B1.6. Roots of 1 + (vi B)2n for n = 2 and n = 3
and in the case n = 3,
T(v) = -Vi V2 V3
(v - vd(v - V2)(V - V3)
EXERCISE B1.7. Show that the Butterworth filter 0/ order n = 2 can be
implemented by an LRC circuit.
Bl2 Band-Pass Signals

In this section, we give the basic facts conceming frequency transposition and study
the phenomena associated with it, such as cross-talk in quadrature multiplexing,
and channel dispersion.
Complex Envelope
The first relevant notion is that of a base-band signal.
DEFINITION B1.1. A band-pass (vo, B) signal, where B < VO, is a stable signal
whose FT is null if Iv I f/. [- B + vo, Vo + B]. A base-band (B) signal is a stable
signal s(t) whose FT is null outside the interval [- B, +B].
It will be assumed, moreover, that set) is real and hence that its Fr is Hermitian
even:
sC-v) = s(v)*.
We are going to show that a real band-pass signal set) has the representation
set) = met) cos(2nvot) - n(t) sin(2nvot), (36)
wherem(t) andn(t) aretwo signals thatare real, andbase-band (B). The base-band
signals met) and n(t) are the quadrature components ofthe band-pass signal set).
Bl2 Band-Pass Signals 69
/\/\ I
0
/\1'\ 8(1/)
0
/\1'\ ~S:(I/)
d\ 0
~u(1/ )
Figure BI. 7. Complex envelope in the Fourier domain
One way ofproving (36) is to form the analytic signal of s(t)
sa(t) = 2 10 00
s(v)e2iJrvt dv, (37)
and then its complex envelope u(t) (see Fig. B1.7)
u(t) = 1 sa(v + vo)e2iJrvt dv. (38)
EXERCISE BI.S. Show that the FT ofthe signal Re {u(t)e2iJrvot} is s(v), and thus
s(t) = Re {u(t)e2iJrvot}. (39)
Let m(t) and n(t) be the real and imaginary parts of u(t):
u(t) = m(t) + in(t). (40)
The quadrature decomposition (36) follows from (39) and (40).
EXERCISE BI.9. Show that
~() u(v) + u(-v)* (41a)
mv=
2 '
A u(v) - u(-v)*
n(v) = 2i (41b)
and that
m(v) = {s(v + vo) + s(v - VO)}l[-B,+Bl(v), (42a)
n(v) = - i{s(v + vo) - s(v - vo)}l[-B,+B](v). (42b)
Frequency Transposition and Quadrature Multiplexing

Frequency transposition is the operation that transforms a real signal m(t), base-
band (B), into the band-pass (vo, B) signal
s(t) = m(t) cos 27Tvot.
70 BI. Filtering
The frequency Vo is called the carrier frequency.

The original signal met) is recovered by synchronous detection: One first mul-
tiplies the received signal set) (assuming a channel without noise, distortion, or
attenuation) by the carrier cos 2rrvot:
2s(t) cos(2rrvot) = 2m(t) COS2(2rrvot)
= met) + met) cos(4rrvot),
and the signal m(t)cos4rrvot, which is band-pass (2vo, B), is eliminated by the
low-pass (B), which leaves met) intact.
Since a real signal such as met) has a Hermitian symmetrie Fr, the frequency
transposition technique uses a bandwidth 2B, and therefore, there is a waste of
bandwidth: One should be able to transmit two real signals in the base-band (B) on a
bandwidth of 2B. There are several ways of doing this. One ofthem is quadrature
multiplexing (quadrature amplitude modulation, or QAM). In this technique, in
order to transmit two real base-band (B) signals met) and n(t), one sends the
signal set) = met) cos(2rrvot) - n(t) sin(2rrvot).
EXERCISE B1.10. Let set) be a base-band (B) signaloffinite energy. What is the
support ofthe FTofthe signal s(t)2?
EXERCISEB1.11. Showthat, in ordertorecoverm(t)(resp., n(t), onecanmultiply
set) by 2 cos(2rrvot) (resp., 2 sin(2rrvot) and thenpass the resulting signal through
a low-pass (B) (see Fig. Bl.8).
Band-Pass Filtering
When the band-pass signal (36) is passed through a filter with frequency response
T(v), we may, without loss of generality, consider that T(v) = 0 if lvi fj [vo -
B, Vo + B], since filtering is expressed in the frequency domain by multiplication
of the Frs. Hence it will be assumed that the impulse response h(t) of the filter is
also a band-pass (vo, B) function.
The output signal y(t) has as Fr
y(v) = T(v)s(v), (43)
cos(27rl/ot) 2 cos(27rl/ot)
m(t) -+-oo~ m(t)
n (t) -+-OO---Y n(t)
sin(27rl/ot) 2 sin(27rl/ot)
Figure B 1.8. Quadrature multiplexing

B12 Band-Pass Signals 71
and it is therefore also band-pass (va, B).

EXERCISE B1.12. Show that ifwe denote by v(t) and u(t) the complex envelopes
ofy(t) and s(t), respectively, then
v(v) = T(v + va)u(v). (44)

The equality (36) is a base-band representation of the band-pass filtering equality
(43).
We shall describe two effects that are specific of frequency transposition. The
first one is the phenomenon of cross-talk in quadrature multiplexed channels.
Cross-Talk
Suppose we use quadrature multiplexing; we thus send two band-pass messages
m(t) and n(t) in the form
s(t) = m(t)cos(2nvat) - n(t)sin(2nvat).
Ideal reception (without distortion in the channel) is performed by synchronous

detection whereby m(t) and n(t) are recovered. Their Frs are given by (42a) and
(42b), respectively.
Suppose there is distortion in the channel and that, consequently, the received
signal is s'(t). We then obtain, after synchronous detection, m'(t) and n'(t) with
respective Frs
m'(v) = {s'(v + va) + s'(v - Va)}l[-B,+Bl(v),
n'(v) =- i{s'(v + va) - s'(v - Va)}l[-B,+Bl(v).
Let us assume that the distortion s(t) -+ s'(t) is a linear filtering with frequency
response T(v). Let us note that T(v) is Hermitian symmetric, as it is the Fr of a
real impulse response h(t). We have s'(v) = s(v)T(v), and therefore,
m'(v) = {T(v + va)s(v + va) + T(v - va)s(v - va)}l[-B,+Bl(v),
n'(v) = - i{T(v + va)s(v + va) - T(v - va)s(v - Va)}l[-B,+Bl(v).

It appears that m' (v) in general cannot be expressed as a function of m( v) alone.
It depends on both m(v) and n(v), and therefore, in general, there is interference
between the two paths. However, under the condition that T(v) be Hermitian
symmetric about Va in the band of width 2B centered on Va, that is,
T(v + va) = T*(va - v) forall v E [-B, +B], (45)
or, again, in view of the Hermitian symmetry of T (v) about 0,

T(v + va) = T(v - va) forall v E [-B, +B], (46)
then
m'(v) = G(v)m(v), n'(v) = G(v)n(v), (47)
72 B 1. Filtering
T(v)
~+13_
~:::: !-~=~
v
-va-B -Va -va+B va-B Va va+B
Figure B1.9. The frequency response in Exercise B1.13
where
G(v) = T(v + vo). (48)
In this case there is only a linear distortion, represented by independent filtering of

m(t) and n(t). After identification ofthe channel (that is, identification of T(v)),
we can therefore recover the signals in the two paths.
EXERCISEBl.13. Suppose thatin the band [vo- B, vo+ B], T(v) has thefollowing
form (see Fig. Bl.9):
T(v + Vo) = A + v, T(v - Vo) =A- v.

Show that m'(t) is a linear combination ofm(t) and (dfdt)n(t).
We shall now study another phenomenon associated with frequency transposi-

tion, that of group delay.
Dispersive Channels
A dispersive channel is a homogeneous linear filter with frequency response
T(v) = Kei(v), (49)
where K is a complex constant that will be taken equal to unity. This channel trans-
forms the complex sinusoid e2irrvt into the delayed complex sinusoid e i (2rrvt+ (v)),
where (v) is the phase ofthe filter at the frequency v.
Let s(t) be a real signal, band-pass (vo, B), of the form s(t) = m(t) cos 2:rrvot.
Let y(t) be the signal obtained by passing s(t) through the dispersive channel. The
corresponding base-band equivalent filter has the frequency representation
v(v) = T(v + vo)m(v),
where v(v) is the Fr of the complex envelope v(t) of y(t) [see (43)].
Suppose that in the band [vo - B, Vo + B], the dispersion has a first-order
expansion
(v + vo) ~ (vo) + v -a/ , v E [-B, +B];

av V=Vo
Bl2 Band-Pass Signals 73
then (approximately)
where
(vo)
T ---- (50)
P - 2:rrvo
and
(51)
Therefore, we have
Now,
y(t) = Re {v(t)e2invot}.
Hence we have
y(t) = m(t - Tg ) cos 2:rrvo(t - Tp ). (52)
The constants Tp and Tg are the phase delay and group delay, respectively.
B2
Sampling
B21 Reconstruction and Aliasing

In a digital communication system, an analog signal {S(t)}tE~ must be transformed
into a sequence of binary symbols, 0 and 1. This binary sequence is generated
by first sampling the analog signal, that is, extracting a sequence of sampies
{s(n)}nEZ, and then quantizing which means converting each sampie into a block
ofO and 1.
The first question that arises is: To what extent does the sampie sequence re-
present the original signal? This cannot be true without further assumptions since
obviously an infinite number of signals fit a given sequence of sampies.
The second question is: How do we efficiendy reconstruct the signal from its
sampies?
The Shannon-Nyquist Theorem
We begin with a general result that will then be applied to the study of
undersampling and both oversampling.
THEOREM B2.1. Let s(t) be a stable and continuous complex signal with Fourier
transform s( v) E LJ::OR.), and assume in addition that, jJr some 0 < B < 00,
Lls(~)1 <00. (53)

nEZ 2B

76 B2. Sampling
Then
LS(v + j2B) = _1 L S( ~) e-2iJrvfs, a.e. (54)

ja 2B na 2B
Let h(t) be a complex signal oftheform
h(t) = L T(v)e2iJrvt dv, (55)
where T(v) E LU~). The signal
set) = _1 LS(~) h(t - ~) (56)

2B nEZ 2B 2B
then admits the representation
set) = [ !LS(V + j2B)! T(v)e2iJrvt dv. (57)

J~ JEZ
Proof" By Theorem A2.3, the 2B-periodic function <P(v) = LjEZ s(v + j2B)
is locally integrable, and its nth Fourier coefficient is
1 [ 2 n
2B J~ s(v)e- "'2li V dv,
that is, since the Fourier inversion formula for set) holds (s(v) is integrable) and
it holds everywhere (s(t) is continuous), the nth Fourier coefficient of <p(v) is in
fact equal to
The formal Fourier series of <P( v) is therefore
2~ ~s(2~)e-2iJrfsv.
nE",
In view of condition (53), the Fourierinversion formula holds a.e. (Theorem A2.2),
that is, <p(v) is almost everywhere equal to its Fourier series. This proves (54).
Since the frequency response T(v) E Lb(~), the impulse response h(t) given
by (55) is bounded and uniformly continuous, and therefore set) is bounded and
continuous (the right-hand side of (56) is a normally convergent series-by (53)-
of bounded and continuous functions). Also, upon substituting (55) in (56), we
obtain
set) = _1 L S ( ~) [ T(v)e2iJrv(t-fsl dv
2B nEZ 2B J~
=[ ! L _1 s( ~)
e-2iJrvfs! T(v)e2iJrvt dv.
J~ nEZ 2B 2B
B21 Reconstruction and Aliasing 77
(The interehange of integration and summation is justified by Fubini's theorem

beeause
L
krna Is( ~
2B
)IIT(V)I dv = (L Is( ~ )1) (rk
na 2B
IT(v)1 dV) < 00.)
Therefore,
set) = L g(v)e2inVI dv,
!
where
g(v) = 2~ {~S(2~) e-2inviB T(v).
The result (57) then follows from (54).

We now state the Shannon-Nyquist sampling theorem.
THEOREM B2.2. Let s(t) be a stable and continuous signal whose FT s( v) vanishes
outside [- B, + B], and assume condition (53) is satisfied. We can then recover
s(t)from its sampies s(nj2B), n E Z, by theformula
set) = LS(~) sine(2Bt - n), a.e. (58)

nEZ 2B
Proof' This is a direet eonsequenee of the previous theorem, with T ( v) the

frequeney response ofthe low-pass (B). Indeed,
{ LS(V + j2B)! T(v) = s(v)I[-B,+Bl(v) = s(v),

JEZ
and therefore, by (57),
set) = L s(v)e2invI dv = set).

The seeond equality is an almost everywhere equality; it holds everywhere when
set) is a eontinuous signal (see Corollary Al.2).
If we interpret s(nj2B)h(t - nj2B) as the response of the low-pass (B) when

a Dirae impulse ofheight s(nj2B) is applied at time nj2B, the right-hand side of
Eq. (58) is the response of the low-pass (B) to the Dirae eomb (see Figs. B2.1 and
B2.2)
s.(t) = _1
1 2B
"s(~)
L;,2B
8(t - ~).
2B
(59)
nEa-
78 B2. Sampling
!21 s (t)
_~~~f~f~f~f'f'f'f",
Figure B2.l. The Dirae eomb of (59)
1/2B
s (t) --7j)(}--+---l s(t)
Figure B2.2. Sampling and reeonstruetion
Sampie and Hold
In praetiee, the Dirae eomb is replaeed by a train of "true" funetions. Instead of

the above train of impulses one of the teehniques of reeonstruetion (ealled sampie
and hold) uses the train of rectangles
1,T(t ) --
S -
2B 2B gT (t - -2B'
lL:(n) S n)
-
nEZ
where gT(t) is a rectangle of base r and unit area,

I
gT(t) = -r 1[0
'
Tl(t),
an approximation of the Dirae impulse as r beeomes large. This signal is then

filtered by a low-pass (B), to produee the signal ST(t). We show that the result is a
smoothed version of the original signal:
ST(t) = - I1
r 0
T
s(t - u) du.
(Observe that we eannot use Theorem B2.1 as such; why?). Condition (53) implies
that Si,T(t) is integrable and has an Fr given by
= ;(v) (2~ ~S (2~) e- 2irrfB ) .
The signal ST(t) is obtained by low-pass (B) filtering of the stable signal Si,T(t).
Sinee the impulse response of a low-pass is not integrable, we eannot use the
eurrent version of the eonvolution-multiplieation rule as it iso However, we shall
proeeed formally beeause the result is justified by a more appropriate version of
the convolution-multiplication rule (Theorem C3.4). We therefore have
~ ~
s,(v) = g,(v) (1"
2B L;s (n)
2B e- 2in 2Bn l[-B,+Bl(v) ) = g,(v)s(v).
nE",
~ A
The result then follows by the inversion formula and the convolution-multiplication
formula (the current version, this time).
Aliasing
What happens in the Shannon-Nyquist sampling theorem if one supposes that the
signal is base-band (B), although it is not the case in reality?
Suppose that a stable signal s(t) is sampled at frequency 2B and that the
resulting impulse train is applied to the low-pass (B) with impulse response
h(t) = 2Bsinc (2Bt), to obtain, after division by 2B, the signal
s(t) = LS(~) sinc(2Bt - n).

nEZ 2B
What is the Fr of this signal? The answer is given by the following theorem,
which is a direct consequence of Theorem B2.l.
THEOREM B2.3. Let s(t) be a stable and continuous signal such that condition
(53) is satisfied. The signal
s(t) = LS(~) sinc(2Bt - n) (60)

nEZ 2B
admits the representation
s(t) = l J(v)e2inVf dv,
where
J(V) = !LS(V + j2B)!1[-B,+Bl(V), (61)

kEZ
~
If s(t) is integrable, then s(v) is its Fr, by the Fourier inversion theorem. This
Fr is obtained by superposing, in the frequency band [- B, +B], the translates
by multiples of 2B of the initial spectrum s( v). This superposition constitutes the
phenomenon of spectrum folding, and the distortion that it creates is called aliasing
(see Fig. B2.3).
EXERCISE B2.1. Show that if the signal
s(t) = ( Sin(2Jl' Bt)2

Jl't
is sampled at rate 1/2B and if the resulting train of impulses is filtered by a
low-pass (B) and divided by 2B, the result is the signal
sin(2Jl' Bt)
Jl't
80 B2. Sampling
-w
~
-B +B +W s(v)
s(v + 4B) s(v + 2B) s(v) i s(v - 2B) s(v - 4B)
o B 2B 3B 4B
-W -B
<--~,
+B +W
~(v)
Figure B2.3. Aliasing
EXERCISE B2.2. Let Vo and B be such that
0< 2B < vo,
and let s(t) be a stable and continuous base-band (B) signal such that
LkEZ Is(kjvo)1 < 00. Consider the train ofimpulses
Si(t) = ~ L s (~) 8 (t - ~) .
Vo nEZ Vo Vo
Passing this train through a low-pass (vo + B), one obtains a signal a(t). Passing
this train through a low-pass (B), one obtains a signal b(t).
Show that
a(t) - b(t) = 2 s(t) cos(2Jl'vot).

(We have therefore effected the frequency transposition of the original signal.)
The following exercise gives aversion of the sampling theorem for band-pass
signals.
EXERCISE B2.3. Let Vo = 2K B for some integer K 2: 1, and let m(t) be a stable
base-band (B) signal. Consider the jrequency-transposed version of this signal,
that is, s(t) = m(t) cos(2Jl'vot). Suppose that
Show that if the impulse train
_1
2B
L s (~)
nEZ 2B
8 (t - ~)
2B
isfiltered by a low-pass (B), we then recover the original signal m(t).

Oversampling
We have seen the effeets of inadapted sampling, that is, sampling at a too slow
rate (undersampling) that results in aliasing, or speetrum folding. We now show
that oversampling ean be exploited to obtain faster rates of eonvergenee in the
reeonstruetion formula.
Assurne that the situation of the Shannon-Nyquist theorem prevails; in partie-
ular, we have the reeonstruetion formula (58). The quantity sine (2Bt - n) therein
is of the order of 1/ n in absolute value and of altemating sign. Therefore, the speed
of eonvergenee of the series on the right-hand side is, roughly, eomparable to that
of
L (_1)n S (..!!:.-).
nEZ n 2B
In order to aeeelerate eonvergenee, one ean use oversampling in the following way.
Assurne that supp(s(v)) is eontained in the frequeney interval [- W, + W] for
some 0< W < 00. In formula (57) ofTheorem B2.1, ehoose
B = (1 +a)W (62)
for some a > 0 and take any integrable function T(v) such that
T(v) =1 ifv E [-W, +W]. (63)
The resulting signal is then a perfect replica of s(t) since
! ?=S(V + j2B)) T(v)

JEZ
= s(v).
Therefore, ifwe sampIe at arate 2B largerthan the Nyquistrate 2W, and then filter
the resulting train of impulses with a filter of impulse response h(t), we obtain,
after division by 2B, the signal
_1 '"' s(..!!:.-) h(t _ ..!!:.-)

2B L::,
nEtL.
2B 2B '
which is a replica of the original signal s(t) provided the frequency response of
the filter verifies condition (63).
EXERCISE B2.4. Suppose that
v+B -v+B
T(v) = B _ W 1[-B,-w] + 1[-w,+w] + B _ W 1[+B,+w].
Give the corresponding impulse response, and study the rate 0/ convergence 0/ the
series on the right-hand side 0/(60).
The series in the reconstruetion formula can decay faster by choosing a smoother
frequency response T(v), since increasing the smoothness of a function increases
the decay of its Fourier transform.
82 B2. Sampling
B22 Another Approach to Sampling

This section presents another approach, more direct and with a broader scope,
to sampling. It acknowledges the fact that a signal is a combination of complex
sinusoids and therefore starts by obtaining the sampling theorem for this type of
elementary signals.
Sampling a Single Sinusoid
Consider the signal
set) = e2iTrAt ,
where A E R This signal is neither stable nor of finite energy, and therefore it
does not fit into the framework ofthe L'- and L 2 -versions of Shannon's sampling
theorem. However, the Shannon-Nyquist formula remains essentially true.
THEOREM B2.4. For all t E ~ and all A E (-1/2T, + 1/2T),

. (rr
- (t - nT) )
= L e2iTrnT
sm
e2iTrAt rr T . (64)
nEZ - (t - nT)
T
For all B < 1/2T, the convergence is uniform in t E [ - B, +B].
Proo!, We first prove that for all A E ~ and all t E (-1/2T, + 1/2T),
. (rr )
e2iTrAt = L e2iTrnTt sm rr-T (A - nT) , (65)
nEZ - (A - nT)
T
where the series converges uniformly for all t E [-B, +B] for any B < 1/2T.
The result then follows by exchanging the roles of t and A.
Let g(t) be the 1/2T-periodic function equal to e2irrt on (-1/2T, +1/2T].
The series in (65) is the Fourier series of g(t). We must therefore show uniform
pointwise convergence of this Fourier series to the original function.
Without loss of generality, we do this for the Fourier series of the 2rr-periodic
function equal to e iat on (-rr, +rr], where the convergence is uniform on any
interval [-c, +c] C (-rr, +rr). By (39) ofSectionA31, it suffices to show that
lim
ntoo
I-Tr
+Tr
leia(t-s) - eiat 1
sin((n + .! )s)
sin(s /2)
2 ds =0
uniformlyon [-c, +c]. Equivalently,
lim
ntoo
I +Tr 1 sin(as /2)1
-Tr
.
sm(s /2)
sin((n + !)s) ds = O.
This is true, for instance, by the extended Riemann-Lebesgue lemma A1.3.

B22 Another Approach to Sampling 83
In partieular, if set) is a trigonometrie signal,

M
set) =L Yke2ilrVkf, (66)
k=!
where Yk E C, Vk E lR, and if T satisfies

1
2T > SUp{IVkl : 1 ~ k ~ M}, (67)
we have the Shannon-Nyquist reconstruction formula
+N sin (~(t - nT)

set) = lim Ls(nT)
Ntoo -N
7/(t -
- nT)
. (68)
T
EXERCISE B2.1. With a single sinusoid check that you realty need the strict
inequality in (67).
Sampling a Decomposable Signal

The following extension ofthe sampling theorem for sinusoids is now straightfor-
ward:
THEOREM B2.5. Let J.L be a nonnegative, finite measure on [- B, + B], where
o< B < 00,and define the signal
set) = [ e2ilrVf J.L(dv). (69)

[-B.+B]
Then for any T < 1/2B and for alt t E lR,
set) =L s(nT)
sin (!f
7T:
(t - nT)
. (70)
nEZ T (t - nT)
Proof" Sinee ,Lis finite and the eonvergenee in (64) is uniform in A E [-B, +B],
set) =[ {L e2ilrvnT
sm - (t - nT
. (7T:
7T: T
)} J.L(dv)
[-B,+B] nEZ - (t - nT)
T
- (t - nT))
. (7T:
sm
=L {[ e2ilrvnT J.L(dv) }
nEZ [-B,+B]
7T: T
- (t - nT)
T
.

This c10ses for the moment our study of the Shannon-Nyquist sampling theory.
It will be eompleted in Seetion B32 by the theorem of equivalenee of analog and
digital filtering, and in Seetion C22 by the L 2 -version of the sampling theorem.
84 H2. Sampling
B23 Intersymbol Interference

As a further illustration of the weak Poisson sum formula, we consider the problem
of intersymbol interference in digital communication. It does not belong to the
Shannon-Nyquist sampling theory, however, it does concern sampling.
Pulse Amplitude Modulation and the Nyquist Condition
In a certain type of digital communication system one transmits "discrete" infor-
mation consisting of a sequence {an}nEZ, of real or complex numbers, in the form
of an analog signal
s(t) = I>n g(t - nT), (71)
nEZ
where g(t) is areal or complex function (the "pulse"). Such a "coding" of the
information sequence is referred to as pulse amplitude modulation. Here, T > 0
determines the rate of transmission of information and also the rate at which the
information is extracted at the receiver.
EXERCISE B2.2. Assume that g(t) is a stable signal with FT g(v) and that an is
stable, with transferfunction A(z). Show that s(t) is stable, andgive its FTin terms
of g(v) and A(z).
At time kT the receiver extracts the sampie

s(kT) = L ang(kT - nT),
nEZ
that is,
akg(O) + L ak-j g(jT).
jEZ
NO
If one only wants to obtain ak from the sampie s(kT), the term
L ak_jg(jT)
jEZ
NO
is parasitic. This term disappears for every sequence ak if and only if
g(jT) =0 for all j i= O. (72)
It turns out that this is equivalent to
(73)
The weak version of the Poisson sum formula of Section A22 is actually all
that is needed to prove the result. Indeed,
THEOREM B2.6. Let g(t) be a continuous and integrable function, and assume
that its FT g( v) is in LUIR). The following two conditions are equivalent:
B23 Intersymbol Interference 85
(a) g(jT)= Oforall j E Z, j f. 0;

(b) LnE:d{v + !f) = const. almost everywhere.
Proof By the weak version of the Poisson sum formula of Section A22,
Tg( - nT) is the nth Fourier coefficient ofL g(v +n/T) (Note that the continuity
condition on g(t) is used here.) Therefore, if (b) is true, then (a) is necessarily
true. Conversely, if (a) is true, then the sequence {Tg( - nT)}nEZ is the sequence
of Fourier coefficients of two functions, the constant function equal to T g(O),
and LnEZ g( v + n / T), and therefore the two functions must be equal almost
everywhere.
Condition (73) is the Nyquist condition for the absence of intersymbol
interference. 2
The pulses g(t) used in communications always, for reasons both techno-
logical and operational (bandwidth resources), have a restricted frequency band
[- W, + W]. The Nyquist condition (73) can be satisfied only if
I
2W ~ T' (74)
Therefore, if transmission without interference between symbols is required, the

minimal bandwidth is
2W=2B~ ~
T
In this case, there is no other choice for the corresponding pulse than
A I
g(v) = 2B 1[-B,+Bj(v),
that is,
sin(27r Bt)
g(t) = . (75)
27r Bt
One dis advantage of such a pulse is linked to questions of numerical stability.
Indeed, let us assurne that the sampling of s(t) is not carried out at the time kT but
at the time kT + ~, where ~ > O. We obtain
sin(27r B~)" sin(27r B~)
s(kT +~) = ak 27rB~ + f;;;oa k- j 27rB(~ _ jT)'
We see that the error
Is(kT +~) - akl (76)
does not stay bounded for all bounded sequences {ad, because
L I . I
NO I~ - JT
=00. (77)
2S ee H. Nyquist, (1928), Certain Topics ofTelegraph Transmission Theory, Trans. Amer.

Inst. Elec. Eng., 47, 617-644.
86 B2. Sampling
A better pulse from this point of view is the "raised eosine"

eos(27f Bt)
g(t) = sine (2Bt) 1 _ 16B 2 t 2 ' (78)
whose Fr is
g(v) = eos 2(7fV)

A
4B 1[-2B,+2Bj(v). (79)
In fact,
sine (4BD.)
sekT + D.) = ak 1 _ 16B 2D. 2
'" sin(47fBD.)
+ f#oa n- j 47f B(D. - jT)(1 - 16B2(D. - jT)2)'
and the error (76) is seen to remain bounded whatever the bounded sequenee {ad.
Partial Response Signaling
Another disadvantage of the pulse (75) is that one eannot realize signals with an
Fr that has an "infinite slope" (at - B and + B).
We shall see that, with clever encoding, we ean attain the Nyquist limit (74)
(which says that in order to transmit a "symbol" an every T seeonds without
intersymbol interferenee, a bandwidth of at least 2W = 2B ~ 1fT is needed),
without resorting to an unrealizable pulse (with a very large slope).
For example, in the duobinary encoding teehnique, instead of transmitting (7),
one transmits
S'(t) = L(an + an+l)g(t - nT), (80)
nEZ
that is,
S'(t) = Lang'(t - nT), (81)
nEZ
where
g'(t) = g(t) + g(t + T). (82)
With the pulse (75) ofminimal bandwidth 2B, starting from (80) we obtain
s'(kT) = ak + ak-l = Cb
and from the sequence {Ck} and the initial datum ao we recover the sequenee {ak}'
The interest of this teehnique is that we do not seek to implement Si (t) in the form
(80) using the unrealizable pulse g(t), but rather in the form (81) with a realizable
pulse g'(t). Indeed,
8'(V) = (1 + e-2iJrvT)g(v)
= 2T eos(7fvT)e-2iJrvT 1[-B,+Bj(v).
B23 Intersymbol Interference 87
This pulse has minimal bandwidth 2B, and, furthermore, it is easier to realize, not
having an infinite slope.
The above is a particular case of the technique of partial response signaling. 3
The general principle is the following: We pretend to use the unrealizable pulse
g(t) given by (75), but in (71) we replace the symbol an by an encoding Cn , say, a
linear encoding
(83)
which gives
S'(t) = I>ng(t - nT).
nEZ
In order to realize S'(t) it is rewritten in the form
S'(t) = .~:::>ng'(t - nT),

nEZ
where
g'(t) = g(t) + y,g(t + T) + ... + Ykg(t + kT) (84)
is a base-band (B) pulse, in general easily realizable, with FT
8'(v) = T(v)g(v), (85)
where
k
T(v) = 1 + LYje-2inVjT = P(e-2invT) (86)
j=!
and
k
P(z) = 1 + LYjz j . (87)
j=!
By sampling at the time t = kT we obtain

s'(kT) = p(z)ak = Ck.
We shall see in Seetion B32 that the sequence fad is deduced from the sequence
{ Ck} by inverse filtering
1
ak = P(z) Ck (88)
(we assume that 1/ P(z) is stable and therefore that the corresponding filter is
causal; these notions are discussed in detail in Section B32).
3See A. Lender (1981), Correlative (Partial Response) Teclmiques and Applications

to Radio Systems, in Feher, K. (ed.), Digital Communications: Microwave Applications
(Prentice-Hall: Englewood Cliffs, Ni), Ch. 7..
88 B2. Sampling
B24 The Dirae Formalism
Do We Need Distributions Theory Here?

In the applied literature, the Dirae formalism of generalized funetions is used
profusely. It eonsists of a small set of symbolie roles that are justified by the
classical Fourier theory of the previous ehapters. In signal proeessing, the Dirae
formalism eulminates in the formula giving the Fr of a Dirae eomb. We shall see
that the Poisson sum formula is, for all praetieal purposes, all that is needed to deal
with such a mathematieal objeet in a rigorous way.
We shall see in the next part that the Fourier theory has in the Hilbert spaee
framework a high degree of formal beauty. There was yet another important step to
be made in this direetion. The physicists had introdueed a very useful tool, the Dirae
generalized funetion, associated with a formal ealeulus that was rather pleasant to
use, but that laeked mathematieal foundations. These were established by Laurent
Schwartz, with the elegant theory of distributions (or generalized funetions) and
the equally elegant Fourier theory of tempered distributions. 4
Most engineers are familiar with the so-ealled Dirae funetion o(t), whieh is
"defined" by the property
L qJ(t)o(t) dt = qJ(O),
for all funetions qJ(t). They are aware that there exists no such funetion in the
usual sense with such property, and they take the above formula as a symbolie
way of dealing with a limit situation. In the "prelimit," o(t) is replaeed by a proper
funetion, depending on a parameter, say, n. There are many ehoiees for this proper
funetion on(t), the simplest one being
on(t) = nree~(t).
Then for sufficiently regular funetion qJ(t) (say, eontinuous),
lim ( qJ(t)on(t) dt = qJ(O).

ntoo JIR
Thus, in this point of view, the Dirae funetion is the limit of proper funetions
beeoming more and more eoneentrated around the origin of times while their
integral remains equal to 1. Another eandidate with these properties is the Gaussian
pulse that we have already eneountered in the proof of the inverse Fourier formula:
1 ,2
ha(t) = - - e--,;;z,
a"fEi
where this time the positive parameter a tends to zero. Observe that the Fr ofboth
on(t) and ha(t) (whieh we have previously eomputed) eonverge pointwise, as the
4Theorie des Distributions, Vols. 1 and 2,1950-1, Hermann, Paris.

B24 The Dirac Formalism 89
eorresponding parameters tend to the appropriate limits, to 1. This is eonsistent

with the formal eomputation of the Fr of the Dirae funetion
8(v) = L 8(t)e-2irrvI dt = e-2irrvO = 1.

Another generalized funetion that is omnipresent in the signal proeessing literature
is the Dirae eomb (indeed a eosmetie too1!), also ealled the Dirae pulse train. It is
the T -periodie generalized funetion
Il T (t) = L 8(t - nT).
neZ
If we formally eompute its nth Fourier eoeffieient, we obtain
-1
T
l
0
T
8(t)e- 2''" TI
n dt = 1.
Now if we write the eorresponding formal Fourier series
-1
T
Le 2irr!!.v
T
'
neZ
we observe that its eonvergenee is rather problematic. We ean, however, pursue

the heuristies, and eonsider that the latter sum is the limit as N ~ 00 of the
1/ T -periodie funetion
1 +N
-
T
Le 2irr!!.v
T
'
-N
whieh is the Diriehlet kernel
1 sin(2rr(N + T) !
T sin(rrT)
GraphieaIly, up to a multiplieative faetor 1/ T, such a funetion looks in the vicinity
of 0 like a Dirae funetion: As N ~ 00, it becomes more and more coneentrated
around 0, and its integral in a neighborhood of 0 tends to 1. Therefore, at the limit
we have, invoking the 1/ T -periodicity, the Fourier transform of the Dirac comb
~T(V) = ~ L 8 (v - !!..) .
T neZ T
This overdose of heuristics may weIl be fatal for the more critical mind. However,
in most basic courses in signal analysis, it is administered with the best intentions,
with the exeuse that it saves the student from a painful exposition to distributions
theory. This apology of mathematical euthanasy is founded on wrong premiees.
The first question that one should ask is: Do we need the Dirae comb in signal
analysis? Looking back at the previous chapters, we ean immediately answer NO.1t
is not needed to derive the Shannon-Nyquist theorem, because the Poisson formula
is all that is needed there. Is the Poisson formula harder than distributions theory?
Again, the answer is NO, without surprise, because the distributions theory version
90 B2. Sampling
of the Poisson sum formula is only a small ehapter of distributions theory. (I shall
add that the heuristie derivation of the Poisson sum formula-see the eomment
following the statement of Theorem A2.3 of Chapter l-is mueh more eonvineing
than the usual heuristie derivation of the Fr of the Dirae eomb.)
In fact, the reader may skip this ehapter and proeeed to Chapters 3 and 4 without
damage. On the other hand, the Fourier transform of the Dirae eomb is part of a
well-established tradition in signal analysis that is bound to be etemal due to its
aesthetie appeal. I have therefore devoted the next seetion to the expression of the
classical results of Fourier analysis in the Dirae formalism. It is, however, a purely
symbolie analysis.
The Dirac Generalized Function
The principal formal objeet of the Dirae formalism is the Dirae generalized funetion
8(t), and the first formal rule is the symbolie formula
L qy(t)8(t - a)dt = qy(a). (Dl)
B2.1. By the first symbolie rule,
L
EXAMPLE
e-2iJrvt8(t - a)dt = e-2iJrva,

that is,
Fr
8(t ...,.. a) -+ e- 2IJrva.
In partieular, the Fouriertransform ofthe Dirae generalizedjunetion is the eonstant

junetion equal to l.
EXAMPLE B2.2. Let x(t) be a T -periodie signal, and let {xn}, n E Z, be the
sequenee of its Fourier eoeffieients. In Seetion A2I we defi-ned the FT x(t)
symbolieally, by
Using the symbolieformula
x(t) = L e2iJrvt xCv) dv
and the symbolie rule (Dl), we then have
x(t) =L
nEZ
r
J[{
e2iJrvtxn8(V - -f) dv
= LXne2iJrft,
nEZ
and we recover the inversionformula of Seetion A2I.

B24 The Dirae Fonnalism 91
EXAMPLE B2.3. Let x(t) be as in the previous example. /fit is the input of afilter
with (stable) impulse response h(t) and withfrequency response T(v) = h(v),
symbolic calculations give for the output
y(t) = 1 h(v)x(v)e2iJrvt dv
= LXn JJ[{[ h(v)e2iJrvt8(v - -f) dv,

nEZ
that is,
The sequence LYn} of Fourier coefficients ofy(t) is thus

A(n)A
Yn = h
A
T xn,
a result that we already know.
The FT of the Dirac Comb

Consider the Dirae eomb
I:lT(t) = L 8(t - nT).
nEZ
The seeond symbolie fomula, that we now introduee, gives the FT of this
generalized funetion:
(D2)
EXAMPLE B2.4. The Poisson sum formula. The formal Plancherel-Parseval

equality
1 cp(t)I:lT(t) dt = 1 qJ'(v)Lr;:.(v) dv
gives, upon substituting into it
I:lT(v) = ~L 8 (v - -f)'
nEZ
the Poisson sum formula
Multiplication Rule
The third symbolie formula of the Dirae formalism eoneerns the multiplieation of
a Dirae generalized funetion by a funetion in the usual sense:
s(t)8(t - a) == s(a)8(t - a). (D3)
92 B2. Sampling
This rule is consistent with the first rule, in that
l s(t)8(t - a)cp(t)dt = s(a)cp(a) = l s(a)8(t - a)cp(t)dt.
EXAMPLE B2.5. Sampling and Spectrum Folding. The train oJ sampled pulses
Si(t) = Ls(nT)8(t - nT)
netz:
may, in view oJ(D3), beJormally written
Si(t) = s(t)Llr(t).
Its symbolic FT is thereJore
Si(V) = s(v) * K;(v)
that is,
~
Sie v) "~(
= -1 ~ s v- n) .
-
T netz: T
Ifwe input TSi(t) into a low-pass [-I/T, + l/T]filter, we thereJore obtain at the
output the signal set) with FT
s(v) = LS(V -
netz:
-f) l[-t,+tl(v).
This is the equation describing spectrum Jolding (see Theorem B2.3).

EXAMPLE B2.6. The FT oJ a Radar Return Signal. Let us consider the signal
set) = ( L h(t - nT)) J(t)

netz:
= (h(t) * Llr(t))J(t) = v(t)J(t).

Its FT is (by the convolution-multiplicationJormula)
s(v) = l V(fL)J(V - fL)dfL
Now, on using the rule (D3):

v(v) = h(v)~r(v)
=~
T
L h(v)8(v -
netz:
!!.-)
T
B24 The Dirac Formalism 93
On the other hand,
Thuswe have
s(v) = ~ Lh(~) i(v - ~),

nEZ
that is, Eq. (71) 0/ Section B2.3.

The examples above show how the Dirae symbolie ealeulus formally aeeounts
for ea1culations of Fourier transforms. This symbolie ea1culus retrieves formulas
already proven in the framework of the classical Fourier theory in LI, formulas
that have been proved under eertain eonditions of regularity, and of integrability
or summability. The symbolie ea1culus does not say under what eonditions the
final symbolie formulas have a meaning, nor in what sense they must be inter-
preted (equalities almost everywhere? in LI?). For this reason, the Dirae symbolie
ea1culus must be used with preeaution. From a mnemonic point of view, it ean
be useful, as it allows one to obtain some formulas very quiekly, and "generally"
these formulas are eorreet under eonditions that are "almost always" satisfied in
praetiee.
However, let us emphasize onee more the fact that these formulas have been
obtained rigorously within the framework of Fourier transforms in LI.
B3
Digital Signal Processing
B31 The DFf and the FFT Algorithm

TheDFT
Suppose we need to compute numerically the Fr of a stable signal s(t). In practice
only a finite vector of sampies is available,
s = (so, ... , SN-t>,
where Sn = s(nb.). The Fourier sum of this vector evaluated at pulsations Wk =

2k1t / N is the discrete Fourier transform (DFr).
DEFINITION B3.1. The DFT 0/ s = (so, . .. ,SN-i) is the vector S =
(So, ... , SN-d, where
L
N-i
Sk = sn e - i (21rkn/N).
n=O
The DFr is an approximation of the Fr, the quality of which depends on the
parameters N and .1.. The first question to ask is: How to choose these parameters
to attain a given precision? As we shall see, the answer is given by the Poisson
sum formula. For the time being, we shall give the basic properties of the DFr
without reference to a sampled signal.
Let a = (ao, ... , aN-i) be a finite sequence of complex numbers. For the Nth
root of unity, we adopt the following notation:

96 B3. Digital Signal Processing
The finite sequence A = (A o, ... , AN-I) defined by

N-I
Am = Lanw';rn (89)
n=O
is the DFT of a = (ao, ... , aN-d.
THEOREM B3.1. We have the inversion formula
1 N-I
an = N LAmw'Nmn . (90)
m=O
Proof'
N-I N-I
~ ~ m(k-n)
= L...t ak L...t w N .
k=O m=O
But if k =1= n,
N-I WN(k-n) _ 1
~ m(k-n) N =0
L...t w N = 1
m=O w Nk-n -
since wZr = 1 when r =1= 0; on the other hand, for k = n,

N-I N-I
L w~(k-n) = L I = N.
m=O m=O

If we consider the periodic extensions of the finite sequences a = (ao, ... , aN - d
and A = (Ao, ... , AN-d, defined by
an+kN = an, (91)
Eqs. (89) and (90) remain valid since w<;:+kN)n = w';rn.
The sequences an and Am being N -periodie, the domains of the sums (89) and
(90) can be shifted arbitrarily. In particular, with N = 2M + 1,
+M 1
L A mWN-mn .
+M
Am = L an w';rn , an = (92)
n=-M 2M + 1 m=-M
In the sequel, we use the above periodie extensions. The relation between the
sequences {an} and {Am} will be symbolized by
(93)
We observe that
(94)
B31 The DFf and the FFf Algorithm 97
THEOREM B3.2. If an ~ Am and b n ~ B m, we have the convolution-

N N
multiplication rule
(95)
ProoJ- The proof of (95) consists of a simple verification. In fact,
The change of variable n - k =r gives

N-I N-I-k
"~ b n-kwN
mn = w mk"
N ~ b rWn
mr
n=O r=-k
But because of the N -periodicity of the sequences {b r } and {w:zn, the last sum
can be taken from 0 to N - 1. Therefore,
~ (~akbn_k)w~n = ~ak (w~k %:brw:r) = AmBm.
Equation (95) and the inversion formula (90) give
(96)
Making n = 0 in (96) and taking (94) into account, we obtain the Plancherel-
Parseval equation for the OFf
N-I 1 N-I
LakbZ = N L AmB~. (97)
k=O m=O
With an == bn we obtain the energy conservation formula
N-I 1 N-I
L
k=O
lall = N L IA
m=O
I
m 2 (98)
The Fast Fourier Transform Aigorithm
The calculation of the OFf of the sequence {an} by formula
requires N - 1 multiplications for m ~ 1 (none for m = 0). If we consider that

the cost of an addition is negligible compared with that of a multiplication, the
calculation of A = (Ao, ... , AN-I) thus requires (N - 1)2 computational units,
98 B3. Digital Signal Proeessing
where one unit corresponds to one multiplication. The fast Fourier algorithm, 5 , also
called the fast Fourier transform (FFT), considerably reduces the computational
complexity. It is based on the following remark.
Let an ~ Am be a DFT pair (note that we are considering a DFT of order 2N
2N
with 2N terms an and 2N terms Am). Define
(0 ::S n ::s N - 1),
and
(the latter DFTs are of order N). A direct calculation shows that
(0 ::s m ::s 2N - 1). (99)
. that Bm+N
Ob servmg = B m, Cm+N = Cm, and m+N
W 2N = 2N , we can sp lt
-wm 1
Eq. (99) in two parts:
(O::s m ::s N - 1) (100)
and
(0 ::s m ::s N - 1). (101)
In order to calculate B m and C m for 0 < m ::s N - 1, we need 2(N - 1)2
computational units. When (100) is used we need N -1 additional multiplications.
The multiplications in (101) are for free since they were done in (100). In total,
the method requires
2(N - 1)2 +N - 1 = (N - 1)(2N - 3)
units instead of (2N - 1)2 for the direct method. If we have to calculate a DFT of
order N such that
N = 2s , (102)
!
the FFT will take F(N) ::S N 10g2 N computational units. The result is obtained
by induction. Indeed, F(2) = 1, and the considerations above show that
F(2N) = 2F(N) + N - 1 ::S 2F(N) + N.
But if F(N) ::S !N 10g2 N, then 2F(N) + N ::S N(log2 N + 1) = !2N log2 2N.
The gain in computational complexity with respect to the direct method is thus
of the order of
1 10g2 N
---
2 N
5Cooley, J.w., Lewis, P.A.w., and Welch, P.D., The Fast Fourier Transform Algor-
ithm, eonsiderations in the ealeulation of sine, eosine, and Laplaee transforms, 1. Sound
Vibrations, 1970, 12(3),315-337.
B31 The DFT and the FFT Algorithm 99
The above discussion just gives the basic idea of the FFT. For a detailed account
of the algorithmic aspects of the discrete-time Fourier transform, see, for instance,
[B8]. We now turn to the numerical issues behind the DFf.
Numerical Analysis of the DFT
The Poisson sum formula is useful in numerical analysis when approximating a
Fourier integral by a Darboux sum, and this is of course related to the finite Fourier
transform.
Let us recall the Poisson sum formula, assuming that the conditions of validity
are satisfied:
(103)
The expression (103) elucidates the relation between the Fr s(v) of the signal
s(t) and the DFr ofits sampled and truncated version (s( - M ll), ... , s( +M ll,
+M
L s(nll)e-2irrn2.J+,.
n=-M
In fact, letting v = kj[(2M + l)ll] in (103),

Ls(nll)e-i2rr2~~' = ~ LS(~ + k ). (104)
nEZ II nEZ II (2M + l)ll
THEOREM B3.3. Let s(t) be a signal with support contained in [-M ll, +M ll].
We then have
~ s(nll)e-2irr 2!:t:, = 1 ' " ~(n k) (105)

n~M II fas II + (2M + l)ll .
Proof: Just apply formula (104).

If the terms corresponding to the indices n =1= 0 in the right-hand side of (105)
were null, only the central term
1 ~( k )
II s (2M + l)ll
would remain. The DFf of (s( -M ll), ... , s( +M ll would then be a sampled
version of the PT, that is,
( II1~s(-Mvd,, II1~)
s(+MvI) ,
where VI = Ij[(2M + l)ll]. But one cannot have a signal s(t) with bounded
support which has FT s(v) also with bounded support. There will thus always be
an error, equal to
This error is the aliasing error. It can be controlled by choosing t:.. small enough for
s(v) to be negligible outside the interval [-B, +B] = [-1/2t:.., 1/2t:..]. But then
M must be adjusted so that s(t) remains zero outside [-M t:.., +M t:..]. Increasing
M increases the computational complexity.
We shall retain the approximate relation linking the effective bandwidth 2B =
1/ t:.., the effective temporal extension T = 2M t:.., and the complexity N = 2M + 1:
2BT :::::: N. (106)
Band T are chosen such that s(t) is negligible outside [-T /2, +T /2] and s(v) is
negligible outside [- B, + B]. Precision requires large T and large B, in order to
capture a large amount of the time-frequency content of the signal. This results in
large complexity (measured by N) of the DFT. This in turn requires sophisticated
algorithms such as the FFT in order to reduce the computationalload.
B32 The Z-Transform

Discrete-Time Fourier Transform
A discrete-time signal is, in signal processing, a sampled signal. This section gives
the basic tools of digital signal processing: the Fourier transform (reducing to a
Fourier sum) and the z-transform.
DEFINITIONB3.1. A stable discrete-time signal is a sequence {Xn}nEZ of complex

numbers such that
(107)
Its Fourier sum is the function

i(w) = LXke-ikW. (108)
kEZ
We observe that it is a 2JT-periodic function. Also, with the same arguments as

for the FT of a stable signal (see Section All), we observe that it is continuous
and bounded by LnEZ Ixnl
An inversion formula is available:
THEOREM B3.1. Xn is the nth Fourier coefficient of i(w):
Xn = -1 1+11" i(w)e . lnW dw. (109)

2JT -11"
Proof" MultipIy (108) by einwand integrate from -JT to +JT.

EXERCISE B3.1. Let Xn = s(n/2B), where s(t) is a continuous base-band (B)
signal with FT s(v). Show that the Fourier sum associated with {x n } is
i(w) = 2Bs~(W 2B)

2JT . (110)
B32 The Z-Transform 101
EXERCISE B3.2. Give the impulse response of the filter with frequency response
exp(cos(w))ei sin(w).
DEFINmoN B3.2. The operation that associates to astahle discrete-time signal

Xn the discrete-time signal
(111)
where h n is a stahle signal, is called convolutional filtering. The signal Yn is the

output of the convolutional filter :F with impulse response h n, and Xn is the input.
When the input signal is the unit impulse at 0,
I ifn = 0,
On = { (112)
o otherwise,
the output is Yn = h n, whence the terminology.
When X n and h n are stable, the right-hand side of (111) has a meaning. In fact,
and, in particu1ar,
L IXkllhn-kl < 00 for al1 n E Z.
kE71
This also shows that Yn is stable.

DEFINmON B3.3. A causal, or physically realizable, filter is one such that
for all n < O. (113)
The filter is called causal because if the input Xn is zero for n .::: no the output
Yn is zero for n .::: no. The input-output relation (111) takes, for a causal filter, the
form
n
Yn = L Xkhn-k. (114)
k=-oo
DEFINITION B3.4. The Fourier sum

h(w) = Lhne-inW (115)
nE71
is the frequency response of the convolutional filter with stahle impulse response
hn.
Ifwe write i(w) and y(w), respectively, for the Fourier sums ofthe input X n and
the output Yn, the input-output relation (6) reads
y(w) = h(w)i(w). (116)
Indeed,
y(w) = LYne-inW
nEZ
= " " h n-ke -inw

~~Xk
nEZ kEZ
_" 1
- ~
kEZ
Xk e -ikw " h - -i(n-k)w
~ n-k e
nEZ
I

EXAMPLE B3.1 (The pure delay). The input-output relation X n -+ Yn defined by
Yn = Xn-k
is a homogeneous filtering with impulse response
hn = 1o 1 ifn
ifn
= k,
=1= k,
and jrequency response
h(w) = e- ikw .
EXERCISE B3.3 (The smoothing filter). This is the filter defined by the input-
output relation
1 +N
Yn = 2N + 1 L
k=-N
Xn-k
Show that its jrequency response is

- 1 sin{(N + !)w}
hN(w) = - - - ------'=-----
2N + 1 sin{w/2}
where h(O) = 1. What is limNtoo hN(w)?
Equivalence of Analog and Digital Filtering
It is important to understand how the operation of analog filtering followed by
sampling can be performed if one chooses first to sample and then to operate in
the sampled (digital) domain. The precise statement and the precise answer are
contained in the theorem below.
Let s(t) be a stable continuous signal, base-band (B), sampled at the Nyquist
frequency 2B. We obtain the sampled signal
S
. n s(v)dv
( - n) = j+B e 2lJrV2Jj
2B -B
= _1 j+Jt eiJtW2BS(!!.. w) dw
2Jr -Jt Jr
(the inversion formula can be applied because s( v) is integrable, having a bounded

support; on the other hand, the equality of s(t) and flR s( v )e2iJtvt dt holds for all t,
since both quantities are continuous). It is further assumed that
~ Is (2~) I < 00, (117)
and therefore, the Fourier sum of s(nj2B) is 2BsB jJr )W).

THEOREM B3.2. Let x(t) and h(t) be stable continuous signals, base-band (B),
both satisfying the condition 0/ type (117). Then
(118)
This is the theorem of the equivalence of analog and digital filtering.

Prao!" The discrete-time signal
Yn = 2~ t; h (2~ )x(n 2~ k )
is stable, and its Fourier sum is
Hence we have
Yn = -1 j+Jt 2Bh~ (B
- W) - w ) e lnW
x (B . dw
2Jr -Jt Jr Jr
= j+B h(v)x(v)e2iJtv it; dv.

-B
On the other hand, for the analog signal
y(t) = L h(t - s)x(s)ds, (119)
we have the inversion formula
y(t) = j +B
e2iJtvtx(v)h(v)dv,
-B
and therefore Yn = y(nj2B).
Transfer Functions
To every discrete-time signal Xn is associated itsformal z-transform, which is the
formal series
(120)
The formal z-transform of the impulse response h n of a convolutional filter is the

formal transfer function of the filter considered:
(121)
The input-output relation (111) reads as a function of the z-transforms of Xn, Yn,
andh n
Y(z) = H(z)X(z). (122)
Note, however, that the z-transform of a signal only takes a meaning as a function
of z E C if one gives the domain of convergence of the series defining it.
We use the unit delay operation z defined symbolically by
iXn = Xn-k
With this notation the relation (6) is written
Yn = Lhk(ixn)
nEZ
that is, symbolically,

Yn = H(z)x n . (123)
In some cases (see the examples below) a function H(z) holomorphic in a ring
{rl < Izl < r2} containing the unit circle {Izl = I} is given. This function defines
a convolutional filter whose impulse response h n is given by the Laurent expansion
(see [B6], Theorem 1.22, p. 53)
H(z) = L hnz n (rl < Izl < r2). (124)
nEZ
In particular, the Laurent expansion at z = 1 is absolutely convergent, and thus

the impulse response h n is stable. The frequency response of the filter is
(125)
i
Recall that the Laurent expansion is explicitly given by the Cauchy formula
hn = - 1 -H(z) dz, (126)

2irr c zn+l
where C is a c10sed path without multiple points that lies within the interior of the
ring of convergence, for example the unit circ1e, taken in the anti-c1ockwise sense.
The method of residues can be used to compute the right-hand side of formula
(126). This equality also takes the form
(127)
The integral in (126) can also, have been computed by the method of residues:
If C is a simple c10sed contour on which f is analytic, except for a finite number
of isolated singular points Z I, ... , ZN, then
1. f(z)dz = 2irr
~
tab
k=1
where ak is the residue of f at Z = Zk (see [B 1], Chapter 4, pp. 207 and following).
This is the Cauchy residue theorem. In the case where f has a pole of order m at
Z = Zk, the residue at this point is given by formula
1 dm- I
ak = (m _ I)! dz m- I [f(z)(z - Zk)m-I]lz=Zk
EXERCISE B3.4. Compute

1. 3z + 1 d
~ z(z - 1)3 Z.
Series, Parallel, and Feedback Configurations

We now describe the basic operations on digital filters (see Fig. B3.1).
Let'ci and'c2 be two convolutional filters with (stable) impulse responses hi and
h~ and transfer functions H I(z) and H2(Z), respectively.
The series filter ,C = 'c2 * 'cl is, by definition, the convolutional filter with
impulse response h n = (h I * h 2)n and transfer function H(z) = H I(Z)H2(Z). It
operates as follows: The input X n is first filtered by ,cl. and the output of'ci is then
filtered by 'c2, to produce the final output Yn.
The parallel filter ,C = 'cl + 'c2 is, by definition, the convolutional filter with
impulse response hi + h~ and frequency response H(z) = HI(z) + H2(Z). It
operates as follows: The input X n is filtered by 'cl, and "in parallel," it is filtered
by 'c2, and the two outputs are added to produce the final output Yn.
The feedback filter ,C = ,CJ/(l - 'cl * 'c2) is, by definition, the convolutional
filter with impulse response frequency response
H(z) = HI(Z)
1 - H I (Z)H2(Z)
This filter will be a convolutional filter if and only if this frequency response is the
FT of a stable impulse response.
Xn Yn
Xn
Figure B3.1. Series, parallel, and feedback configurations
The filter .cl

is the forward loop filter, whereas .cl
is the feedback loop filter.
The forward loop processes the total input, which consists of the input X n plus the
fed-back input, that is, the output Yn processed by the feedback loop filter.
EXERCISE B3.1. Consider the filter with impulse response
hn = (~r l{n:::O).
Give a feedback representation of this filter.
Rational Transfer Functions

Let
p q
P(z) = 1 + L:>jzj, Q(z) = 1 + Lbel (128)
j=l e=l
be two polynomials with complex coefficients of the complex variable z. We shall
assurne that P(z) has no roots on the unit circle {Izl = I}.
Let rl = max{lzl : P(z) = 0 and Izl < I}; rl = 0 if there is no root of P(z)
with modulus strictly smaller than 1.
Let r2 = inf{lzl : P(z) = 0 and Izl > I}; r2 = +00 ifthere is no root of P(z)
with modulus strict1y larger than I.
The function
H( ) = Q(z) (129)
z P(z)
is holomorphic in the ring Cr j,r2 = {rl < Izl < r2} (in the open disk {Izl < r2}
if rl = 0) which contains the unit circle since r2 > 1. We thus have a Laurent
expansion in Crj ,r2
H(z) = I>nzn, (130)
nEZ
which defines a filter with stable impulse response h n and frequency response
Q(e- iW )
H(e iW ) =
P(e-' W )
(see [BI], Section 3.3, or [B6], Theorem 1.22, p. 153).
EXAMPLE B3.1. An integer r 2: 1 and y E C are given. We set

1
H(z) = .
(z - yy
First Case: lyl > 1. The ring 0/ convergence is defined by r2 = Iyl and rl = 0
(thus, in/act we have a disk 0/ convergence {Izl < lylJ that contains the unit
circle). The Laurent expansion is in this case a power-series expansion in the
neighborhood 0/ zero
(Izl < Iyl)
n;:::O
To find the impulse response h n we must expand H(z) as apower series. Rut
(_1)r-l(r - I)!(z - y)-r is the (r - I)st derivative 0/
_1_ = _ 2. (1 + ~ + ~ + ... + ~ + ... ),

z- y y Y y2 yn
Izl < y,
and there/ore,for Izl < y,

(-Ir-l(r - I)!(z _ y)-r-l
1 zn-r+l
L n(n -
00
=- - 1) ... (n - r + 2) - - ,
y n=r-l yn
= _ 2. ~ (j + ~ - I)! zj. _1_ .

yko }! yJyr-1
Finally,
(- IY ~ (j + r - I)! j
1 ~
(z - yY ( 1)
yr r- . j=O '1'
}.yJ Z,
and, identifying this expression with L}:o h j zj, we obtain

hn = (-Ir (n + r -1)! (2.)r+n, n 2: 0,
n!(r - I)! Y
with hn = 0 if n < O.
Second Case: Ir I < 1. The Laurent expansion is then a power-series expansion

in the neighborhood 0100:
1
--- = h_nz- n
(z - r)'
L
nO":O
Changing Z into l/ s,
(-s1- r )-r = (-1)r-rrsr ( s - -r1)-r Isl< - .
Irl
1
We can use the previous calculations to obtain
and we obtain the anticausal filter

(n - I)! -n-r
h_ n = (r - 1)!(n - r)!
r ,n ~ r,
where h n = 0 ifn > -r.
Linear Recurrence Equations
If Yn is the output of the filter with transfer function (129) corresponding to the
stable input signal X n, we have y(w) = H(e-iw)x(w), that is,
P(e-iw)ji(w) = Q(e-iw)x(w).
Now P(e-iw)y(w) is the Fourier surn of the signal Yn + L:~=l ajYn-j, and
Q(e-iw)x(w) is the Fourier surn of X n + L:i=l bexn-e. Therefore,
p q
Yn + LajYn-j = X n + Lbexn-e, (131)

j=l e=l
or, syrnbolically,
P(Z)Yn = Q(z)xn.
The general solution of the recurrence equation (131) is the surn of an arbitrary
solution and of the general solution of the equation without right-hand side
p
Yn + LajYn-j = O.
j=l
This latter equation has for a general solution a weighted surn of terms of the form
r(n)p-n,
where p is aroot of P(z) and r(n) is a polynornial of degree equal to the multiplicity
of this root minus one. If we are given X n , n E Z, and the initial conditions
Yo, Y-l, ... , Y-p+l, the solution of (131) is cornpletely deterrnined.
In order that the general solution never blows up (it is said to blow up if
limlnltoo /Yn/ = (0) whatever the stable input X n, n E Z, and for any initial con-
ditions Y-p+l, ... , Y-l, Yo, it is necessary and sufficient that all the roots of P(z)
have modulus strict1y greater than unity.
A particular solution of (131) is
Yn =L hkxn-k .
k::::O
The output Yn is stable when the input X n is stable since the impulse response h n
is itself stable, and therefore Yn does not blow up.
Therefore, we see that in order for the general solution of (131) with stable input
Xn to be stable, it is necessary and sufficient that the polynomial P(z) has all its
roots with modulus strict1y greater than 1.
DEFINITION B3.1. The rational filter Q(z)/ P(z) is said to be stable and causal if
P(z) has all its roots outside the closed unit disk {/z/ ::: I}.
Causality arises from the property that if P(z) has roots with modulus strictly
greater than unity Q(z)/ P(z) = H(z) is analytic inside {/z/ < rz} where rz > 1.
The LaUfent expansion of H(z) is then an expansion as an entire series H(z) =
Lk::::O hkz k, and this means that the filter is causal (hk = 0 when k < 0).
DEFINITION B3.2. The stable rational filter Q(z)/ P(z) is said to be causally
invertible if Q(z) has all its roots outside the closed unit disk {/z/ :::
I}.
In fact, writing the analytic expansion of P(z)/ Q(z) in the neighborhood of zero
as Lk::::O WkZ k, we have
that is,
X n= L WkYn-k . (132)
k::::O
B33 All-Pass and Spectral Factorization

All-Pass Filters
A particular case of a rational filter is the all-pass filter.
THEOREM B3.3. Let Zi (l ::: i ::: L) be complex numbers with modulus strictly
greater than 1. Then the transfer function
L *_ 1
H(z)=n~ (133)
i=l Z - Zi
satisfies
 1 iflzl > 1.

Proof" Let
Hi() ZZi* - 1
Z=---
Z - Zi
be an arbitrary factor of H(z). If Izl 1, we observe that IHi(z)1 1, using
Fejer's identity
(z - )(z -~) = -~zlz - I 2 (135)

* * '
which is true for Izl = 1, E <C, =I- O. On the other hand, Hi(Z) is holomorphic
on Izl < Iz;J and IHi(O)1 = Iz;J-l < 1. Therefore, we must have IHi(z)1 <
1 on {Izl < Iz;J}' otherwise the maximum modulus theorem for holomorphic
functions would be contradicted. (Recall the maximum modulus theorem: If I is
analytic in a bounded region D and III is continuous in the closure of D, then III
takes its maximum on the boundary of D; see [BI], Theorem 2.66, p. 97, or [B6],
Theorem 1.21, p. 51.) Observing that
IHiCI* ) 1= IHiI(z) I
we see that the resultjust obtained implies that IHi(z)1 > 1 if Izl > 1.
A filter with frequency response H(e- iw ) is a pure phase filter, or all-pass filter,
by definition.1t is called all-pass because its gain is unity: IH(e-iW)1 = 1.
Consider a signal X n such that

O-::;.n-::;.N,
otherwise.
It can be represented by its polynomial z-transform
N
A(z) = L anZn .
n=O
Let Zl, Z2, ... , ZN be its roots. In particular,
A(z) = aN n(z -
N
j=l
Zj).
The effect offiltering X n with an all-pass filter (zrz - I)/(z - Zl) is to replace the
factor z - Zl in A(z) by zrz - 1, but it does not change the energy of the signal.
Indeed, the z-transforrn of the resulting signal,
B(z)
* -
= A(z) zlz 1
,
Z - Zl
is such that
and therefore,
Thus,
N N
L la n l2 = L Ib l n 2 (136)
n=O n=O
At a time 0 ::::: k ::::: N the two signals (ao, ... , aN) and (b o, ... , b N) have already
dissipated the energies
k k
Ea(k) = L lajl2 and Eb(k) =L Ibj l2 .
j=O j=O
There is an interesting relation between these partial energies. Writing
A(z) = (z - zl)F(z), B(z) = (zrz - I)F(z),
where
F(z) = 10 + fIz + ... + IN-I ZN-I ,

we have
an= In-I - zdn, bn = ZUn-1 - In (0::::: n ::::: N),
where I-I = IN = 0 by convention. Taking the square of the modulus and

subtracting yields
and therefore,
(137)
This shows that if Izii < 1, then (ao, ... , an) is always late with respect to
(b o, ... , bN ) in dissipating its energy.
Fejer's Lemma
EXERCISE B3.2. Let X n be a stable signal with z-transJorm X(z). Define its
autocorrelationfunction Cn by
cn = L Xn+k X;
kEZ
Show that {cn}nEZ E .e~(.Z) and that its Fourier sum is

c(w) = li(w)1 2 = R(e- iUJ ),
where
R(z) = X(z)X(z)*.
The 2:rr-periodic function R(e- iUJ ) in the above exercise has the following
properties:
i: n
Moreover, if X(z) is a rational fraction,

R(e- iUJ ) < 00.
(138)
(139)
R(e- iUJ ) is a rational fraction in e- iUJ (140)

The next result is Fejer's lemma, which is also called the spectralJactorization
theorem.
'THEOREM B3.1. Let R(z) be a rational Jraction in z with complex coefficients
such that (138) and (139) are satisfied. Then there exist two polynomials in Z with
complex coefficients, P(z) and Q(z), and a constant c 2: 0, such that P(O)
Q(O) = 1 and
2
R(e- iUJ ) = c 1 Q(e- iUJ ) 1 (141)
P(e- 1UJ )
Moreover, one can choose P(z) to be without roots inside the closed unit disk, and
Q(z) to be without roots inside the open unit disko
Proof" R(z) can be factored as
R(z) = az mo T1 (z - Zk)m k ,
kEK
where a E C, the Zk are nonnull distinct complex numbers, and the mk E Z. If
Izl = 1, R(z) is real, and therefore,
R(z) = R(z)* = a*(z*)mo T1 (z* - zDm k = a*(z-I)m o T1 (Z-I - ZDmk.
kEK kEK
Therefore, when Izl = 1, there exist b E C and ro E Z such that
R(z) = bz ro T1 ( z - --;Zk1 )m
kEK
k
Therefore, if Izi = 1,
Two rational fractions that coincide when Izi = 1 coincide for all Z E <C. In
particular, a = b, and whenever we have in R(z) the factor (z - zd with IZk I i= 1,
then we also have the factor (z - 1.). We therefore have
Zk
wherelZtl = lforallf E L,andlzjl i= lforallj E J.Weshowthatrt = 2se E N

for all f E L. For this, we write Ze = e- i"'" and observe that in the neighborhood of
We, R(e- i "') is equivalent to a constant times (w - weP and therefore can remain
nonnegative if and only if re = 2st . Since R(e- i"') is locally integrable, then
necessarily Se E N. Therefore,
R(z) = bzro n(z - ZjYj (z _ ~)Sj n(z _ Ze)2s,.

jE] Zj eEL
Using Fejer's identity (135), we therefore find that R(z) can be put under the form
R(z) = ciIG(z)1 2 ,
where
G(z) = n(z - Zj)Sj n(z - ze)".
jE] tEL
The function R(e- iw ) can remain real and nonnegative if and only if c ~ 0 and
d = O. Finally, we can always suppose that IZj I < 1 for all j E J (a root Zj is
paired with another root l/zj).
EXERCISE B3.3. Find a constant c and polynomials P(z) and Q(z) as in Theorem
B3.1, such that
2
5 - 2cos(w) 1 Q(e- i "') 1
----)=c .
3 - cos(w) P(e-'''')
The proof ofTheorem B3.1 can be specialized to obtain that for any polynomial
p(z) such that p(e- i"') ~ 0 for all w E lR, there exists a polynomial A(z) with
A(O) = 1 and no root inside the closed unit disk, and a constant c ~ 0, such that
p(e- i"') = cIA(e- i"')1 2

Looking at the proof of B3.1, we see if there exist another polynomial B(z) with
B(O) = 1 and a constant c' ~ 0, such that p(e- iw ) = eil B(e- i "')1 2 , then c = c' and
B(z) = H(z)A(z)
for some all-pass filter H(z).

B4
Subband Coding
B41 Band Splitting with Perfect Reconstruction

Smooth Filter Banks
Let x(t) be a stable base-band (B) real signal that we seek to analyze in the
following sense. For fixed N = 2k we wish to obtain for all I ::::: i ::::: 2k the signals
Xi(t) with Fourier transforms
where Bi is the frequency band
i- I i ]
Bi = [T B '2 kB .
From a theoretical point of view the problem is stated with its solution: For each
i, do no more than filter x(t) with a pass-band filter offrequency response IB/v)!
From the practical point of view of digital processing, in the sampIe domain, an
ideal band-pass filter has an infinite impulse response-actually one with rather
slow decay-and this makes the above pure band-pass filters of poor value from a
numerical point of view.
A solution consists of replacing the pure band-pass filters by approximations

with "good" impulse responses, and if possible finite impulse responses (FIR).
However, FIR filters with short impulse response have in general a poor frequency
resolution, and therefore the analysis will not be satisfactory without a careful
choice of the approximate band-pass filters. One also requires perfect synthesis,

116 B4. Subband Coding
that is,
2k
x(t) = LXi(t),
i=l
where Xi (t) is obtained from x(t) by approximate band-pass filtering on the band Bi.
This means that leakage between contiguous bands must be mutually compensated.
The above is a summary of the numerical problem associated with subband
decomposition of a signal by a filter bank. The second problem is algorithmic:
How to perform efficiently analysis and synthesis? The standard example of an
efficient algorithm is the FFf, which involves successive splitting, and subband
decomposition is another avatar of this idea: The basic block of the algorithm con-
sists of splitting a given band in two, that is, of solving the subband decomposition
problem for N = 2.
Subband coding is one way of performing data compression. Instead of sam-
pling the original signal and then quantifying the resulting sampies with a view
of digitizing them, one performs the sampling and quantifying operations on each
of the outputs Xi(t). If a subband Bi is deemed unimportant it will be allocated
fewer compression resources, that is, only coarsely quantified. The appraisal of
the importance of each subband is generally based on psychological experiments.
The subjective difference between subbands is very marked in two-dimensional
signal processing, where it has been observed that low-frequency components are
the most important from a subjective point of view.
The Basic Algorithm
Since all signals and filters considered in the present chapter are real, we need only
consider positive frequencies, those in the frequency band [0, B]. Ideal splitting of
the frequency band [0, B] uses two ideal band-pass filters, one for the band [0, B 12]
and the other for the band [BI2, B]. We call To(v) and Tl(V) their frequency
responses. Then, as the Shannon-Nyquist theorem suggests, we sampie each output
at rate B, and reconstruction is perfo,Pled by t~o ideal band-pass filters, [0, B 12]
and [B 12, B], respectively. We call To(V and Tl (v) their fr~quency responses (of
course, ifweuse ideal pass-band filters, To(v) = To(v), and Tl (v) = Tl (v); wekeep
different notations because in the nonideal case, the analysis and reconstruction
filters need not be the same).
Consider Fig. B4.1. In the ideal case (ideal pass-band filters), the signals in the
upper branch at levels (){ (Xl (t and Y (Yl (t are identical and equal to the original
signal x(t) filtered by the band-pass [0, B 12]. This follows from the theory of
sampling of Chapter B2, and the details of the operations in the lower branch are
shown in Figure B4.2. Similarly, in the lower branch of Fig. B4.1, the signals at
levels ()( (X2(t and Y (Y2(t are identical and equal to the original signal x(t)
filtered by the band-pass [B 12, B].
As we explained ~efore, the ideal band-pass filters will be replaced by approx-
imations To( v) and To( v) that have most of their energy inside the band [0, B 12],
B41 Band Splitting with Perfect Reconstruction 117
Figure B4.1. Block diagram of subband coding
original signal x(t), base-band (B)

J3 -B/2 0 B/2 B
after the band-pass (B/2, B] (To(lI

-13 -B/2 o B/2 B
13 //\f'......//\f'......//\f'.....
~~ ~~ ~
,
~ after sampling at rate B
- 0 +B +2B
after the reconstruction filter (B/2, B] (Ta(lI

o B/2 B
Figure B4.2. Subband coding in the frequency domain
and Tl (v) and Tl (v) that have most of theirs inside [B /2, B]. We insist once more
on the fact that we do not require that To(v) = To(v) nor that Tl(v) = Tl(V),
because we need some freedom in the choice of To(v) and Tl(v) to guarantee
perfect reconstruction. Analysis of the original signal yields the decomposition
(Xl (t), xz(t, whereas synthesis reconstructs y(t) = Xl (t) + xz(t). Synthesis is
called perfect when y(t) = x(t).
The signal at level a is
Xl(t) = _1 L>(~)
. ~
2B JEa.
2B
ho(t - ~),
2B
where ho(t), h l (t), ho(t), h l (t) are the r~spective imp~lse responses correspond-
ing to the frequency responses To(v), To(v), Tl(V), Tl(V). Sirnilarly the signal
Yl (t) at level y in Fig. B4.1 is
Yl(t) = ~Xl(~)ho(t- ;).

Sampling at the rate 2B gives the sampie sequence
YI(n2B) = Lk _1
2B j
LX(~)
2B
ho(~B - ~)
2B
iio(!!.-
2B
- ~).
B
If we set 2B = 1 (this condition can be forced upon the system by a change of

time scale), we find
YI(n) =L LX(j)h o(2k - j)iio(n - 2k), (142)
k j
with a similar expression for the output Y2(t) ofthe lower branch ofFig. B4.1.
Down- and Up-sampling
We shall now express the resuIts in terms of the operations of down-sampling and
up-sampling, and then go back to (142).
Let {xnlnez be a sequence of complex numbers and let m E N. Consider the
sequences {Ynlnez and {znlnez defined by
Yn = Xnm ' nEZ

and
{ Znm = Xn, nE Z,
Zj =0 if j is not divisible by m.
For example, with m = 2,
Xo Xl X2 X3 X4 Xs
Yo YI Y2
and
Xo 0 Xl 0 X2 0 X3 0 X4 0 Xs
Zo Zl Z2 Z3 Z4 Zs Z6 Z7 Zg Z9 ZIO
The sequence {YnlneZ is said to be obtained from the original sequence {xnlnez
by down-sampling by a factor m. The corresponding operation is denoted as m,/...
Up-sampling by a factor m, denoted as mt, is the operation that transforms {xnlnez
into {znlnez.
In this chapter, we are concemed with the case m = 2. For future use, we shall
express the operation of down-sampling by 2 followed by up-sampling by 2 in
terms of z-transforms (see Figure B4.3).
Denote X(z) and R(z) thez-transforms ofthe sequences {x(n)lnez and {r(n)}neZ,
respectively. The sequence {r(n)}neZ is therefore obtained from {x(n)}neZ by
B41 Band Splitting with Perfect Reconstruction 119
x( n)) @1--_Y--'l(~_)_--1CWI-_~)_r_(n_)
XW RW
Figure B4.3. Down-sarnpling and up-sarnpling
1-------------- r--------------I
1
1
" - - - - - - - - ______ 1
ANALYSIS SYNTHESIS
Figure B4.4. Subband coding in the Z-domain (1 split)
replacing all the entries with an odd index by a zero. Therefore,
R(z) = LX(2n)z2n = ~ !LX(n)zn + Lx(n)(-zt) ,

nEZ nEZ nEZ
that is,
R(z) = !{X(z) + X( - z)}. (143)
Going back to (142) and the similar expression for the lower branch of Fig. B4.1,
we see that the whole system is equivalent in the z-domain to Fig. B4.4.
From (143) we see that
y(z) = ! {X(z)Ho(z) + X( - z)Ho( - z)} Ho(z)
+ !{X(z)H1(z) + X(Z)Hl( - Z)}Hl(Z).
Separating the aliasing terms from the rest,
y(z) = ! X(z){Ho(z)Ho(z) + H 1(z)H1(z))
Therefore, aliasing is eliminated if
Ho( - z)Ho(z) + H 1(- Z)Hl (z) = 0, (145)
and perfect reconstruction is obtained provided that

~
Ho(z)Ho(z)
- (z) = 2.
+ H 1(Z)Hl (146)
B4 2 FIR subband filters

Quadrature Mirrors Filters
In or~r t'2., find a solution of (145) and (146), one ean first fix Ho and then find
H I , Ho, HI in terms of Ho in order to satisfy the no-aliasing eondition (145). Then
one ean determine Ho so that the perfeet reeonstruetion eonditions (146) ean be
satisfied.
Given Ho(z), one possible solution of (145) is 6
HI(z) = Ho(- z),
{
~o(z) = Ho(z), (147)
H I (z) = - Ho( - z).
Assume that the filter Ho is symmetrie, that is, it has a symmetrie impulse response
= ho(n), n E Z). Then
(h o( - n)
HI(z) = Ho(-z) = L(-lrho(n)zn

nEZ
= L (- Ir ho( - n)zn (symmetry of Ho)

nEZ
= L ( - l tho(n) (~)n
nEZ Z
Therefore, if Ho is symmetrie,
that is, in terms of pulsations,

HI(e- iw ) = Ho(e-i(Jr-w).
This means that the pulsation speetrum of HI is symmetrie with respeet to that of
Ho with respeet to the frequeney n /2. This is why in this ease Ho and H I are said
to be quadrature mirror filters (QMFs).
Going back to (147)-and without assurning that Ho is symmetrie-the perfeet
reeonstruetion eondition (146) beeomes, in terms of Ho:
Ho(d - Ho( - Z)2 = 2. (148)
One drawback of the solution (145) is the nonexistenee of a finite impulse response
filter Ho satisfying it. However, we ean relax eondition (146) to
-
Ho(z)Ho(z) -
+ HI(z)HI(z) = 2z k (149)
6Esteban, D., and Galand, C. (1977), Applications of quadrature mirror filters to split-
band voice-coding schemes, Proc. IEEE Inf. Conf ASSP, Hartford, Connecticut, 191-195.
for some K ::: 1, which means that we accept a delay of K time units to recover
the input, and in this case FIR filters do exist.
EXAMPLE B4.1. Taking the no-aliasing condition (147) into account, the relaxed
condition (149) with K = 1 gives
Ho(z)2 - Ho( - d = 2z. (150)
Afamous solution is the Haar filter

1
Ho(z) = -J2 (1 + z). (151)
The relaxed condition (149) allows a "linear phase" corresponding to a delay

K. For K ::: 2, we shall just mention that (149) does not have an exact solution
with a FIR filter.
If in Fig. B4.I, the input signal x(t) is assumed to be band-pass [jB, (j +

I)B], for some j ::: 1, the resulting output of the analyzer is the same as if the
input had been frequency-shifted by j B, to obtain a base-band (B) signal. In fact,
immediately after the sampler at rate B, at level y, we have the same signal, for
both inputs (the pass-band signal x(t), or its base-band version).
Therefore, the analyzer of Fig. B4.1 performs band splitting on the base-band
(B) version of any band-pass [j B, (j + I)B] signal. Consequently, the analyzerof
Fig. B4.4 behaves in the same way, with the additional feature ofbeing independent
of B!
This remark shows that the full program of subband coding can be achieved by
a cascade ofthe analysis (resp., synthesis) structures ofFig. B4.4 (by anticipation,
let us mention that this is similar to the structure of Mallat's algorithm in multires-
olution analysis). Fig. B4.5 shows the analysis synthesis of the band [0, B] into
four subbands [0, BI4], [BI4, B12], [BI2, 3BI4], [3BI4, B].
---------------------
1 1----------------------
1 1
I ____________________ J
~---------------------~
ANALYSIS SYNTHESIS
Figure B4.5. Subband coding in the Z-domain (2 splits)

stage 1
stage 2
stage J
stage 1
stage 2
stage J
Figure B4.6. Octave band filtering
In constant Q-filtering one decides not to split the high-frequency component.

Thus at each stage of analysis only the "coarse" component (corresponding to low
frequency) is further analyzed (see Fig. B4.6). Such a structure is also called a
logarithmicjilter or an octave bandjilter.
EXERCISE B4.2. (a): Verify that filtering by H(z) followed by up-sampling by 2
is equivalent to up-sampling by 2 followed by filtering by H(Z2). Show that the
synthesis part in Fig. B4.6 with J = 3 is equivalent to Fig. B4.7. (b): With the
Haar filter we have
- 1
Ho(z) = -J2 (1 + z),
Figure B4.7. Equivalent octave band filtering
Give the impulse response of each of the fOUf filters in Fig. B4.7.
Another Solution
Another class of solutions7 for the no-aliasing condition (145) is
~l (z)= Z-l Ho( - Z-l),
Ho(z) = Ho(C 1),
{
(152)
H 1(z) = zHo( - z).
The perfect reconstruction condition (146) then becomes
Ho(z)Ho(Z-l) + H o(- z)Ho(- Z-l) = 2. (153)
Since Ho is a real filter,
HO(Z-l) = H(z)* for z = e-ia>,
and (153) takes the form
Ho(e-ia 12
1 + IHo(- e-i a1 2 = 2. (154)
We shall now exhibit a general solution of (152). We perform a change of

notation that will be convenient in the chapter on multiresolution analysis:
Ho(z) = H(z) = L hnzn,
neZ
1 ja>
mo (w) = ,J2Ho(e- ),
1 ja>
ml (w) = ,J2Hl(e- ).
7Smith, M.J.T., and Barnwell, m T.P. (1986), Exact reconstruction techniques for tree-
structured subband coders, IEEE Transactions ASSP, 34, 434-441.
In view of (152), we have

ml(w) = ei"'mo(w + JT)*,
and the perfect reconstruction condition (154) becomes
Imo(w)12 + Imo(w + JT)1 2 = 1. (155)
The solution (152) is in terms of Ho(z), and therefore it suffices to obtain mo(w)
satisfying (155). We seek a finite impulse response filter Ho(z), in which case
mo(w) is a polynomial in e- i "'. We shall in fact look for a solution in the form
1 + ei"')N
mo(w) = ( - 2 - L(w),
where N :::: 1, and L(w) is a polynomial in e- i",. Letting

Mo(w) = Imo(w)12,
+
we have
1+ 2N N
= (cos2 (~))
i", 1
Mo(w) = 1 IL(w)1 2 IL(w)1 2.
But IL(w)1 2 is a real-valued polynomial in e- i "', and therefore it is a polynomial

in cos(w). Since cos(w) = 1 - 2 sin 2(wj2),
Mo(w) = (COS2(~))N P(sin2(~)),

for some polynomial P. Condition (155) must be satisfied for all w, and therefore
it is equivalent to
(156)
for all y E [0, 1]. Since two polynomials identical on [0, 1] are identical
everywhere, the latter equality is for all y E IR.
The polynomials yN and (1 - y)N have no common roots, and therefore, by
Bezout's theorem, there exist two unique polynomials a and b, of degree::: N -1,
such that
(1 - y)N a(y) + yNb(y) = 1. (157)
This is true for all y E IR, and in particular, replacing y by 1 - y,
(1 - y)Nb(1 - y) + yN a(1 - y) = 1.
By the uniqueness of a and b, it follows that
b(y) = a(1 - y).
Therefore, (157) is
(1 - y)N a(y) + yN a(1 - y) = 1.
Therefore, P(y) = a(y) is a solution of (156). We have thuse proven that (156)
admits at least one solution, and by the uniqueness in Bezout's theorem, this
solution is the only one of degree :s N - 1. We have
L
N-l
a(y) = (1 - y)-N[l - yN a(1 - y)] = (f+k-l) l + G(yN).
k=O
Since a is a polynomial of degree :s N - 1, it is equal to its Taylor series truncated

at N - 1, and therefore,
= L (f+k-l) l
N-l
a(y)
k=O
This solution is the unique one with degree :s N - 1. Observe that it is nonnegative
for aH y E [0, 1], and therefore a solution to the initial problem. Call it PN and let
P be the general solution. We have
(1 - y)N (P(y) - PN(y + yN (P(1 - y) - P N(1 - y = 0.

This implies that P - PN is divisible by yN, that is, P(y) - PN(y) = yN Q(y),
and
(1 - y)N yN Q(y) + yN (1 - y)N Q(l - y = 0,

which implies Q(y) + Q(1- y = 0. That is, Q is symmetrie with respect to 1/2,
and therefore ofthe form Q(y) = R(1/2 - y) for an odd polynomial R.
In summary, the general solution of (156) is
P(y) L (f+k-l) l + yN R
= N-l (1 )
-- , (158)
k=O 2
where R(y) is any odd polynomial such that P(y) so defined remains nonnegative
for all y E [0, 1].
Having obtained Mo(w), it remains to extract its square root mo(w). But this can
be done by spectral factorization, using Fejer's lemma.
We shall elose this chapter on the basic principles of subband coding. Note, how-
ever, that other solutions were proposed, most notably "biorthogonal solutions,"8
which are more versatile and yield finite impulse response subband filters with
better properties (of symmetry, for instance). We refer to the monograph [B12],
where the reader will find a full and detailed treatment of this topic, as weH as
additional references.
8Vetterli, M. Filter banks allowing peifect reconstruction, Signal Processing, 10 (3),

1986,219-244.
References
[BI] Ablowitz, M.J. andJokas, A.S. (1997). Complex Variables, Cambridge University
Press.
[B2] Daubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf.
Series in Applied Mathematics, SIAM: Philadelphia, PA.
[B3] Gasquet, C. and Witomski, P. (1991). Analyse de FourieretApplications, Masson:
Paris.
[B4] Haykin, S. (1989). An Introduction to Analog and Digital Communications, Wiley:
New York.
[B5] Hirsch, M.W. and Smale, S. (1974). Differential Equations, Dynamical Systems,
and Linear Algebra, Academic Press: San Diego.
[B6] Kodaira, K. (1984). Introduction to Complex Analysis, Cambridge University
Press.
[B7] Lighthill, MJ. (1980). An Introduction to Fourier Analysis and Generalized
Functions, Cambridge University Press.
[B8] Nussbaumer, H.J. (1981). Fast Fourier Trans/orm and Convolution Algorithms,
Springer-Verlag: New York.
[B9] Orfanidis, S. (1985). Optimal Signal Processing, McMillan: New York.
[BIO] Papoulis, A. (1984). Signal Analysis, McGraw-Hill: New York.
[B 11] Rudin, W. (1966) Real and Complex Analysis, McGraw-Hill: New York.
[BI2] Vetterli, M. and Kovacevic, J. (1995). Wavelets and Sub-Band Coding, Prentice-
Hall: Englewood Cliffs, NJ.
Part C
Fourier Analysis in L2
Introduction
The modem era of Fourier theory started when the tools of functional analysis-
in particular, Lebesgue's integral and Hilbert spaces-became available. Fourier
theory then seemed to have reached the promised land, which is called L 2, the
space of square-integrable complex functions, indeed a Hilbert space.
F. Riesz and E. Fischer were the first to study Fourier series in the L 2 framework. 1
Many ideas of the modem theory of Hilbert spaces were already contained in the
work of these two mathematicians, and they had a c1ear view of the geometrie
aspect ofthe L 2-spaces. They were inspired by aseries of articles by David Hilbert
written after 1904 on the theme of integral equations and in which he gives the
properties of 4(Z). Note, however, that the notion of abstract Hilbert spaces made
its appearance much later than one usually believes, in the years 1927-1930, with
the work of John von Neumann, who was motivated by quantum mechanics. 2
In short, a Hilbert space is a vector space H on the field <C (or lR.), with a
Hermitian (or scalar) product, denoted (., .) or (., .) H, and a special topological
property that we shall now briefly introduce. The Hermitian product induces a
norm, the norm IIxll, or IIxIlH, ofthe vector x E H being
IIxll = (x,x}'1.
I
IF. Riesz, Sur les systemes orthonormaux de fonctions, CRAS Paris, 144, 1907,615-
619; and E. Fischer, Sur la convergence en moyenne, CRAS Paris, 144, 1907, 1022-1024;
Applications d'un theoreme sur la convergence en moyenne, CRAS Paris, 144, 1907, 1148-
1151.
2His theory was published in the reference text Mathematische Grundlagen der Quantum
Mechanik in 1932.
130 Part C Fourier Analysis in L 2
This allows us to define a limit in H: We say that lim n-4oo x n = x iflimn-4oo IIxn -
xII = O. Having this notion of a limit, we have the notion of a Cauchy sequence:
A sequence {xnlnEN in H is called a Cauchy sequence if
lim IIxm - xnll

m,n~oo
= O.
To be called a Hilbert space, H must-besides being a vector space on C with a
Hermitian product-be complete with respect to the induced norm. This means
that any Cauchy sequence {xnlnEN in H converges, that is, there exists an x E H
such that
lim IIxn - xII

n-4OO
= O.
Note that for any positive integer k, Ck , considered as a vector space on C with the
usual Hermitian product, is indeed a Hilbert space. B ut there are more sophisticated
Hilbert spaces. For instance, L~(lR), the space of functions f : lR --+ C that are
square-integrable:
L If(t)1 2 dt < 00.
In L~(lR), one does not distinguish two functions that are almost everywhere equal.
The Hermitian product is
(f, g) = (f, gh 2 (ITt) = ( f(t)g(t)* dt.

c lITt
Saying that limntoo fn =f in L~(lR) means that
lim ( Ifn(t) - f(t)1 2 dt = O.

ntoo lITt
Another example of a Hilbert space is the space of functions f : lR --+ C that are
2rr-periodic, and in L~([ -rr, +rr]), that is, square-integrable on [-rr, +rr):
j +Jr
-Jr If(t)1 2 dt < 00,
with the Hermitian product
(f, g) = (f, g) L~([-Jr,+Jr]) = L: Jr

f(t)g(t)* dt.
In L~([-rr, +rr]), one also does not distinguish two functions that are almost
everywhere equal.
A third example is .e~(Z), the set of complex sequences a = {xnlnEz such that
L Ix l n 2 < 00,
nEZ
Introduction 131
with the Hermitian product
(a, b) = (a, b) e~('z.) = L

neZ
anb~.
The Hilbert space LUlR.) is a paradise ofFourier transforms, since every function
thereof admits a Fourier transform, and moreover the mapping that associates to a
function its Fourier transform is a bijection from L~(lR.) to itself, and the inversion
formula for Fourier transforms, which gives the latter in terms of the former is
f(t) = L j(v)e+ 2i :n:vt dv.
This is not apreeise statement. In particular, the integrals appearing in the definition
of the transform and in the inversion formula are in some extended sense, and the
equality in the inversion formula is "almost everywhere." To be exact,
f(t) = lim jb j(v)e+ 2i :n:vt dv,

a,btoo -a
where the limit is in the sense of L~(lR.). A similar interpretation is needed for the
integral defining the Fourier transform.
The beautiful formula of the L 2- theory is the Plancherel-Parseval's formula
L j(v)g(v)* dv = L f(t)g(t)* dt,
in other words,
(f, g) L~(IR) = (j, g) L~(IR)'

where f, g E L~(lR.).
The above results are stated for the Fourier integral transform, but similar results
hold for the Fourier series of periodic functions: Let f be a 2n -periodic function
square-integrable on [0, 2n]; then it admits the representation
f(t) =L cn(f)e int .
neZ
This is the inversion formula for Fourier series. Similarly to the Fourier transform
in L~ (lR.), this equality is only almost everywhere, and the sum has to be interpreted
in an extended sense:
where the limit is in the sense of L~([ -n, +n]). This result is in fact a particular
case of the Hilbert basis theorem, which gives the orthonormal expansion
x = L(x, en)en
neZ
132 Part C Fourier Analysis in L 2
of any vector x in a Hilbert space H, when {en}nE!\! is a complete orthonormal

system. "Orthonormal" means that
(ek, en) = lk=n,
and "complete" means that the closure of the vector space consisting of the finite
linear combination of elements of {en }nE!\! is H. (See Chapter C2 for precise def-
initions.) In this case, the above orthonormal expansion is valid (the series in the
right-hand side converging with respect to the distance induced by the Hermitian
product of H), and moreover, we have Plancherel-Parseval's identity
IIxll 2 = L 11 (x, e }1I
nEZ
n 2.
The Fourier series development is a particular case of the above very general resuIt,
where H == L~([-x, +x]), and
1 .
en(t) == ~elnt.
",2x
The Plancherel-Parseval formula for Fourier series reads
-
1 j+Jr f(t)g(t)* dt = L cn(f)cn(g)*,
2x -Jr nEZ
where f and gare 2x-periodic functions in L~([ -x, +x]. In terms ofHermitian
products,
Cl
Hilbert Spaces
CII Basic Definitions

Hilbert space theory is the fundamental tool in Fourier analysis of finite-energy
signals. It is a huge chapter of functional analysis, but we shall only give the
definitions and prove the basic facts used in this book, in particular, the projection
theorem and the theorem of extension of isometries.
Pre-Hilbert Spaces
DEFINITIONCl.l. Let E beavectorspaceoverC(resp., lR)andlet(x, y) -+ (x, y)

be a mappingjrom Ex E to C (resp., IR) such that,forall x, y E E and all a E C
(resp., IR),
(a) (x + z, y) = (x, y) + (z, y),

(b) (ax, y) = a(x, y),
(c) (x, y) = (y, x)* (resp., (x, y) = (y, x)),
(d) (x, x) ::: 0, and (x, x) = 0 if and only ifx = O.
It is then said that Eis a complex (resp., real) pre-Hilbert space with the Her-
mitian product (resp., scalar product) (., .). The complex (resp., real) number
(x, y) is the Hermitian (resp., scalar) product ojx and y.
In the above definition and in the sequel, 0 represents the zero of IR or C, or the
neutral element of addition in E. The context will remove ambiguity.
From now on, we shall consider complex pre-Hilbert (and later Hilbert) spaces.
The other choice for the scalar field, IR, leads to formally analogous results.

134 Cl. Hilbert Spaces
For any x E E, denote

IIxll 2 = (x, x). (1)
Elementary computations yield

IIx+YII 2 = IIxIl 2 +IIYIl2+2Re{(x,y)} (2)
for any x, y E E. The parallelogram identity

IIxll 2 + IIYll2 = !(lIx + Yll2 + IIx _ Y1l2) (3)
is obtained by expanding the right-hand side of (3) using (2).
EXERCISE CI.I. Prove the polarization identity
I
(x, y) = 4{lIx + Yll2 - IIx - Yll2 + i IIx + iYll2 - i IIx - iYIl2}. (4)
Consequently, two Hermitian products (., h and (., h on E such that 11 . 111 =
11 . 112 are identical.
THEOREM CI.I. For all x, y E E, we have the Schwarz inequality
I(x, y)1 :s IIxll x lIyll, (5)
with equality holding if and only if there exist a, E C such that ax + y = O.
Proof" We do the proof for the real case and leave the complex case to the reader.
We may assume that (x, y) =1= 0; otherwise, the result is trivial. For all E lR,
IIxll 2 + 2(x, y)2 + 2(x, y)211Y1l2 = IIx + (x, Y)YIl2 ~ O.
This second-degree polynomial in E lR therefore cannot have two distinct real
roots, and this implies a nonpositive discriminant:
I(x, y)1 4 - IIxIl 2 1(x, y)1 211y1l2 :s 0,
and thus the inequality (5) holds. Equality in (5) corresponds to a null discriminant,
and this in turn implies a double root of the polynomial. For such a root, IIx +
(x, Y)YIl2 = 0, that is, by Property (d) in Definition CU,
x + (x, y)y = O.
DEFINITION CI.2. Two elements x, y E E are said to be orthogonal if (x, y) = O.
Let Xl, ... , X n E E be pairwise orthogonal. We have Pythagoras' theorem:
(6)
which follows from (2).

THEOREM CI.2. The mapping x -+ IIx 11 is a norm on E, that is, for all x, y E E,
all a E C,
(a) IIxll ~ 0, and IIxll = 0 if and only ifx = 0,
Cll Basic Definitions 135
(b) lIaxll = lalllxII,

(c) IIx + ylI ::: IIxll + lIyll
Proof The proof of (a) and (b) is immediate. For (c) write
IIx + Yll2 = IIxll 2+ IIYll2 + (x, y) + (y, x)
and
It therefore suffices to prove

(x, y) + (y, x) ::: 2l1xllllyll,
and this follows from the Schwarz inequality.

The norm 11 . 11 induces a distance d(, .) on E by
d(x, y) = IIx - ylI. (7)
Recall that a mapping d : E x E -+ lR+ is called a distance on E if, for all

x, y, Z E E,
(a') d(x, y) ~ 0, and d(x, y) = 0 if and only ifx = y,
(bi) d(x, y) = d(y, x),
(c') d(x, y) ~ d(x, z) + d(z, y).
The above properties are immediate consequences of (a), (b ), and (c ) of Theorem
C1.2.
Hilbert Spaces
A metric space is a set E endowed with a distance d. One then says: the metric
space (E, d), or, for short and when the context is sufficiently explicit as to the
choice of the distance, the metric space E. A pre-Hilbert space E is therefore a
metric space for the distance d induced by the Hermitian product.
DEFINffiON C1.3. A Hilbert space is a pre-Hilbert space that is complete with
respect to the distance d defined above.
Recall that a metric space (E, d) is called complete if any Cauchy sequence in E
converges; that is, if {Xn}n:o:l is a sequence in E such that limm.ntoo d(x n , x m ) = 0,
then there exists x E E such that limntoo d(xn , x) = O.
When considered as a Hilbert space relative to the norm 11 . 11, E will be denoted
H.1f necessary, the notation for the Hermitian product and the norm will explicitly
refer to the space H: We then write (., ')H and 11 . IIH.
EXAMPLE C1.1. Let (X, X, f.L) be a measure space. It is proven in the appendix
(Theorem 26) that LUf.L) is a Hilbert space relative to the Hermitian product
(j, g)L 2 (/-L)

c
= Jx[ f(x)g(x)* f.L(dx).
EXAMPLE Cl.2. A particular case of Example 3.1 is that in which X = Z, X =

P(Z), and J-t is the counting measure on Z. The corresponding Hilbert space L~(J-t)
is then denoted lUZ). Therefore,
4(Z) = {{Xn}nEZ : X n E Cforall nE Z and L Ix ln 2 < oo}

nEZ
is a Hilbert space with Hermitian product
(x, Y}e~(z) = LXnY;.

nEZ
EXERCISE C1.2. Show that if h(t) and x(t) are both in L~(lR), then Y = h * x is
weil defined. Find h E L~(lR) such that IIh 11 = 1 and maximizing y(T)for a given
time T. What is the corresponding maximum?
Cl 2 Continuity Properties
Closed Subspaces
A subset G is said to be c10sed in H if every convergent sequence of G has a limit

in G.
THEOREM Cl.3. Let GeH be a vector subspace of the Hilbert space H. Endow
G with the Hermitian product that is the restriction to Gof the Hermitian product
on H. Then G is a Hilbert space if and only if G is closed in H.
In this case, Gis called a Hilbert subspace of H.
Proof Assume that Gis c1osed. Let {Xn}nEN be a Cauchy sequence in G. It is a

fortiori a Cauchy sequence in H, and therefore it converges in H to some x, and
this x must be in G, because it is a limit of elements of G and G is c1osed.
Assume that G is a Hilbert space with the Hermitian product induced by the Her-
mitian product of H. In particular, every convergent sequence {x n }nEN of elements
of G converges to some element of G. Therefore, G is c1osed.
EXERCISE Cl.3. Let (X, X, J-t) be a measure space. For somefixed constant K, let
G = L~(J-t) n {f; sup If(t)1 :::: K, J-t - a.e.}. Is G a Hilbert subspace of L~(J-t)?
Answer the same question, with G = L~(J-t) n {f; sup If(t)1 < 00, J-t - a.e.}.
WhataboutLUJ-t)n{f;suplf(t)l:::: K(f) < 00, J-t-a.e.}?
DEFINITION C1.4. Let {Xt }tET be an arbitrary collection of elements of H. The

smallest Hilbert subspace of H containing all the vectors Xt, t E T, is called the
Hilbert subspace generated by {Xt }tET, or the Hilbert span of {Xt }tET.
EXERCISE Cl.4. Call G be the Hilbert subspace generated by {XrJtET. Let ,c, the
vector space formed by all finite linear combinations of elements of {Xt }tET. Show
that G = l (the closure of ,c).
Cl2 Continuity Properties 137
The following notation is eonvenient:

12 = span {XI, t E Tl.
G = span {XI, t E T}.
Paraphrasing the above result, we see that X E span {XI> t E T} if and only if X is
the limit in H of a sequenee of finite linear eombinations of elements of {XI, t E T}.
Continuity of the Hermitian product
THEOREM CI.4. Let H be a Hilbert space over C with the Hermitian product
( " .). The mappingfrom H x H into C defined by (x, y) t-+ (x, y) is bicontinuous.
Proof: We have
I(x + h l , Y + h 2 ) - (x, y)1 = I(x, h 2 ) + (h l , y) + (h l , h 2 )1
By Sehwarz's inequality, l(x,h 2 )1 ::: IIxllllh 2 11, l(hl,y)1 ::: lIyllllhlll, and
l(h l ,h 2 )1::: Ilh 1 1l1lh 2 11. Therefore,
lim
II h l II.IIh2 11-1-0
I(x + h l , Y + h 2 ) - (X, y)1 = O.
In partieular, the norm X t-+ IIx 11 is a eontinuous funetion from H to lR+
EXERCISE C1.5. Let (X, X, fL) be a measure space, where fL is a finite measure.
Let {fn}n ::: 1 be a sequence of LUfL) converging to f. Apply Theorem Cl.4 to
prove that limntoo fL(fn) = fL(f). Give a counterexample ofthis property when the
hypothesis that fL is finite is dropped. (Hint: f = 1[0, I] , fn = (1-1/ n) 1[0, 1] +- .. .)
Show that when fL is finite,
G = L~(fL) n {f; fL(f) = O}
is a Hilbert subspace of L~(fL).
Note that when fL is not finite, G need not be a Hilbert subspaee of L~(fL).
Wavelet multiresolution analysis will provide a speetacular eounterexample.
Isometry Extension Theorem
DEFINITION CI.5. Let Hand K be two Hilbert spaces with Hermitian products
denoted ( " .) Hand ( " .) K, respectively, and let q; : H t-+ K be a linear mapping
such that,for all x, y E H,
(q;(x), q;(Y)K = (x, y)H. (8)
Then q; is called a linear isometry from H into K. If, moreover, q; isfrom H onto
K, then Hand Kare said to be isomorphie.
Note that a linear isometry is neeessarily injeetive, sinee q;(x) = q;(y) implies
q;(x - y) = 0, and therefore,
0= 1Iq;(x - y)IIK = IIx - yllH,
and this implies x = y. In particular, if the linear isometry is onto, it is neeessarily
bijeetive.
Recall that a subset A E E, where (E, d) is a metric space, is said to be dense

in E if, for all x E E, there exists a sequence {xn}ne:! in A converging to x. The
following result will often be used. It is ca1led the isometry extension theorem of
Hilbert spaces or, for short, the isometry extension theorem.
THEOREM CI.5. Let Hand K be two Hilbert spaces with Hermitian products
( ., .) Hand ( ., .) K, respectively. Let V be a vector subspace of H that is dense
in H, and let cp : V f-+ K be a linear isometry from V to K (cp is linear and (8)
holds for all x, y E V). Then there exists a unique linear isometry (l : H f-+ K
such that the restriction of (l to V is cp.
Proof We sha11 first define (l(x) for x EH. Since V is dense in H, there exists
a sequence {xn}ne:! in V converging to x. Since cp is isometric,
In particular, {cp(xn) }ne:! is a Cauchy sequence in K, and it therefore converges to

some element of K, which we denote (l(x).
The definition of (l(x) is independent ofthe sequence {xn}ne:! converging to x.
Indeed, for another such sequence {Yn}ne:!'
The mapping (l : H f-+ K so constructed is c1early an extension of cp (for x E V,

one can take as the approximating sequence of x the sequence {xn}ne:! such that
Xn == x).
The mapping (l is linear. Indeed, let x, Y E H, a, E C, and let {xn}ne:!
and {Yn}ne:! be two sequences in V converging to x and y, respectively. Then
{axn + Yn}ne:! converges to ax + y. Therefore,
lim cp(axn + Yn) = (l(ax + y).

ntoo
However,
= acp(xn) + CP(Yn) ---+ a(l(x) + (l(y)

cp(axn + Yn)
Therefore, (l(ax + y) = a(l(x) + (l(y)
The mapping (l is isometric since, in view of the bicontinuity of the Hermitian

product and of the isometricity of cp,
where {xn}ne:! and {Yn}ne:! are two sequences in V converging to x and y,

respectively.
Cl3 Projection Theorem 139
Cl 3 Projection Theorem
Let G be a Hilbert subspace of the Hilbert space H. The orthogonal complement
of G in H, denoted G1., is defined by
G1. = {z EH: (Z, x) = 0 for all x E G}. (9)
Clearly, G1. is a vector space over Co Moreover, it is c10sed in H since if {Zn}n~l
is a sequence of elements of G 1. converging to zEH, then, by continuity of the
Hermitian product,
0= lim(Zn,x) = (Z,x) forallx E H.
ntoo
Therefore, G1. is a Hilbert subspace of H.

Observe that a decomposition x = Y + Z where Y E G and Z E G1. is necessarily
unique. Indeed, let x = Y' + z' be another such decomposition. Then, letting
a = Y - y', b = Z - z', we have that 0 = a + b where a E G and b E G1..
Therefore, in particular, 0 = (a, a) + (a, b) = (a, a), which implies that a = O.
Similarly, b = O.
THEOREM Cl.6 (Projection theorem). Let x E H. There exists a unique element
Y E G such that x - Y E G1.. Moreover,
lIy -xII = inf lIu -xII (10)
UEG
Proof" Let d(x, G) = infzEG d(x, z) and let {Yn}n~l be a sequence in G such
that
d(x, Gi :s d(x, Yni
1
:s d(x, G)2 + -. (*)
n
The parallelogram identity gives, for all m, n ::: I,
IIYn - Ym 11 2 = 2(lIx - Yn 11 2 + IIx - Ym 11 2 ) - 411x - ~(Ym + Yn)1I 2
Since ~(Yn + Ym) E G,
therefore,
IIYn - Ymll 2 :s 2 (~ + ~).

The sequence {Yn}n~l is thus a Cauchy sequence in G, and it consequently con-
verges to some Y E G since G is c1osed. Passing to the limit in (*) gives
(10).
Uniqueness of Y satisfying (10): Let y' E G be another such element. Then
IIx - Y'II = IIx - Yll = d(x, G),
140 CL Hilbert Spaces
and from the parallelogram identity

lIy - y'1I 2 = 211Y - xll 2 + 211y' - xll 2 - 411x _ !(y + Y')11 2
= 4d(x, G)2 - 411x _ !(y + y')1I 2.
Since !(y + y') E G,
Therefore,
lIy - y'1I 2 ~ 0,
which implies that lIy - Y'1I 2 = 0 and therefore, y = y'.
It now remains to show that x - y is orthogonal to G, that is,
(x-y,z)=O forallZEG.
This is trivially true if z = 0, and we shall therefore assume z =j::. O. Because
y + AZ E G for all A E IR,
IIx - (y + Az)1I 2 ::: d(x, Gi,
that is,
Since
wehave
- 2A Re {(x - y, z)} + A211z 11 2 ::: 0 .for allA E IR,
which implies Re {(x - y, z)} = O. The same type of calculation with A E ilR
(pure imaginary) leads to
Im{(x - y, z)} = o.
Therefore,
(x - y, z) = O.
That y is the unique element of G such that y - x E G.L follows from the
observation made just before the statement of Theorem C 1.6.
The element y in Theorem C 1.6 is called the orthogonal projection of x on G
(see Fig. CU) and is denoted PG(x).
Projection Principle
The projection theorem states, in particular, that for any x E G there is a unique
decomposition
x = Y +z, Y E G, Z E G.L, (11)
C13 Projection Theorem 141
xeH
Figure C 1.1. Orthogonal projection
and that y = Pa(x), the (unique) element of G closest to x. Therefore, the

orthogonal projection y = Pa (x) is characterized by the following two properties:
(1) y e G;
(2) (y - x, z) = 0 for all z e G.
This characterization is called the projection principle and is useful in
determining projections.
Projection Operator
The next result features two useful properties of the orthogonal projection operator
Pa
'THEOREM CI.7. Let G be a Hilbert subspace ofthe Hilbert space H.
(a) The mapping x ~ Pa(x) is linear and continuous; furthermore,
IlPa(x)1I :::: IIxll forall xe H.
() If Fis a Hilbert subspace of H such that F S;; G, then PF 0 Pa = PF. In

particular, Pa = Pa (Pa is then called idempotent).
Proof: (a) Let Xl, X2 eH. They admit the decomposition
Xj = Pa(Xj) + Wj (i = 1,2),
where Wj e Gl. (i = 1,2). Therefore,
Xl + X2 = Pa(XI) + Pa(X2) + WI + W2
= Pa(xt} + Pa(X2) + W,
where W e Gl.. Now, Xl + X2 admits a unique decomposition of the type
= y+w,
Xl +X2
where W e Gl., y e G: namely, y = Pa(XI + X2). Therefore,
Pa(XI + X2) = Pa(XI) + Pa(X2).
One similarly proves that
Pa(ax) = aPa(x) for all a e G, X e H.
142 Cl. Hi1bert Spaces
Thus PG is linear.
From Pythagoras' theorem applied to x = PG(x) + w,
IIPG(x)1I + IIwII 2 = IIx1I 2 ,

and therefore,
Hence, PG is continuous.
() The unique decompositions of x on G and G.L and of PG(x) on F and F.L
are
x = PG(X) + w,
From these two equalities we obtain

(*)
But (z E G.L) =} (z E F.L) since F ~ G, and therefore v = Z + W E F.L. On the

other hand, PF(PG(X E F. Therefore, (*) is the unique decomposition of x on
Fand F.L; in particular, PF(X) = PF(PG(X.
The next result says that the projection operator PG is "continuous" with respect
to G.
THEOREM CI.S. (i) Let {Gn }n:::l be a nondecreasing sequence ofHilbert subspaces
ofH. ThentheclosureGofUn:::l Gn isaHilbertsubspaceofH and,for all X E H,
lim PGn(x) = PG(x).
ntoo
(ii) Let {G n } be a nonincreasing sequence of Hilbert subspaces of H. Then

nn:::l G n = G is a Hilbert subspace of Hand, for all x E H,
lim PGn(x) = PG(x).
ntoo
Proof: (i) The set Un>l G n is evidently a vector subspace of H (in general,
however, it is not closedflts closure, G, is a Hilbert subspace (Theorem C1.3). To
any Y E Gone can associate a sequence {Yn}n:::h where Yn E G n, and
lim IIY-Ynll=O.
n->oo
Take Y = Pa(x). By the parallelogram identity,
IIPGn(x) - PG(x)1I 2 = lI(x - PG(x - (x - PG n(x1I 2
= 211x - PGn (x)1I 2 + 211x - PG(x)1I 2
- 411x - !(PGn(x) + PG(xII

C13 Projection Theorem 143
But since PGn(x) + PG(x) is a vectorin G,

IIx - !(PGn(x) + PG(x))1I 2 ~ IIx - PG(x)1I 2,
and therefore,
IIPGn(x) - PG(x)1I 2 S 211x - PGn (x)1I 2 - 211x - PG(x)1I 2
S 211x - Ynll 2 - 211x - PG(x)1I 2.

By the continuity of the norm,
lim IIx - Ynll 2 = IIx - PG(x)1I 2,

ntoo
and, finally,
lim IIPGn(x) - PG(x)1I 2 = O.

ntoo
(ii) Devise a direct proof in the spirit of (i) or use the fact
Gl.. = clos (U G;) .

n~1

EXERCISE Cl.6. Prove the lollowing two assertions:
Let {G n } be a nonincreasing sequence 01 Hilbert subspaces 01 H. Then
nn~IGn = 0 ifand only iflimntoo PGn(x) = Olorall xE H .
Let {Gn}n~1 be a nondecreasing sequence 01 Hilbert subspaces 01 H. Then
clOSUn~1 G n = H ifand only iflimNtoo PGn(x) = xlorall XE H.
NOTATION. IIG I and G 2 are orthogonal Hilbert subspaces olthe Hilbert space
H,
GI ffi G2 := {z = XI + X2 : XI E G, X2 E G2}
is called the orthogonal sum 01 GI and G2.
Riesz's Representation Theorem

DEFlNITION C1.6. Let H be a Hilbert space over C and let I : H t-+ C be a
linear mapping; I is then called a (complex) linear form on H. It is said to be
continuous if there exists A ~ 0 such that
I/(xd - I(X2)1 s AlixI - x211 lor all XI, X2 E H. (12)
The infimum olthe constants A satisfying (12) is called the norm 01 I.
EXAMPLE C1.3. Let Y E Hand define I: H t-+ C by
I(x) = (x, y). (13)

It is a linear form and, by Schwarz's inequality,

If(XI) - f(X2) I = If(XI - x2)1
I(XI - X2, y)1 :'S lIyllllxl - x211
=
Therefore, fis continuous. Its norm is lIyll. To prove this it remains to show that
if K is such that I(XI - X2, y)1 :'S K IIXI - x211foralt Xl, x2 E H, then lIylI :'S K.
It suffi,ces to take Xl = X2 = Y above, which gives IIYll2 :'S K lIylI.
We now state and prove Riesz's representation theorem.
THEOREM C1.9. Let f : H 1-+ C be a continuous linear form on the Hilbert space
H. Then there exists a unique y E H such that (13) is true for alt X E H.
Proof" Uniqueness. Let y, y' E H be such that
f(x) = (x, y) = (x, y') for an X EH.
In particular,
(x, y - y') =0 for all x E H.
The choice x =y- y' leads to lIy - y'II 2= 0, that is, y - y',
Existence: Consider the kernel of f, N = {u EH: f (u) = O}. It is a Hilbert
subspace of H. We may suppose that f is not identically zero (otherwise, if f == 0,
take y = 0 in (13. In particular, N is strictly included in H. This implies that N1.
does not reduces to the singleton {O} and, therefore, there exists zEN 1., z =f- O.
Define y by
y = f(z)* IIzzll 2
For an x E N, (x, y) = 0; therefore, in particular, (x, y) = f(x). Also,
(z, y) = (z, f(z)* 11 zZIl 2 ) = f(z).

Therefore, the mappings x --+ f(x) and x --+ (x, y) coincide on the Hilbert
subspace generated by N and z. But this subspace is Hitself. Indeed, for an
xE H,
x = (x - f(x)
f(z)
z) + f(x)
f(z)
z= u + w,
where u E N and w is colinear to z.
C2
Complete Orthonormal Systems
C21 Orthonormal Expansions

The result ofthis section is the pillar ofthe L 2-theory ofFourier series and wavelet
expansions. It concems the possibility of decomposing a vector of a Hilbert space
along an orthonormal base.
The Gram-Schmidt Orthonormalization Procedure
The central notion is that of an orthonormal system:
DEFINITION C2.I. The sequence {en}n~O in a Hilbert space H is called an
orthonormal system of H if it satisfies the following two conditions:
(cx) (en,ek) = Oforall n =j:.k;and
() lien 11 = Iforalln:::: O.
An orthonormal system {en}n~O isfree in the sense that an arbitrary finite subset
of it is linearly independent. For example, taking (eI, ... , ek), the relation
k
Lcxiei =0
i=1
t
implies that
CXiei ) =0 1::: e ::: k.

EXERCISE C2.I. Let {fn}n~O be a sequence of vectors of a Hilbert space H.
Construct {e n }n~O by the Gram-Schmidt orthonormalization procedure:

146 C2. Complete Orthonormal Systems
Set p(O) = 0 and eo = 10111 10 11 (assuming 10 i= 0 without loss 01 generality);

Witheo, ... , en and p(n) defined, let p(n + 1) be thefirst index p > p(n) such
that I p is independent 01 eo, ... , en, and define, with p = p(n + 1),
n
Ip- L(fp, ej}ej
.,,-11-------;;-11
j=!
en +! =
I
p - "t(fp, ej}ej
Show that {en}n~O is an orthonormal system.
Hilbert Basis
The following theorem gives the preliminary results that we shall need for the
proof of the Hilbert basis theorem.
THEOREM C2.1. Let {en}n2:0 be an orthonormal system 01 Hand let G be the
Hilbert subspace 01 H generated by {en}n~!. Then:
(a) For an arbitrary sequence {an }n~O 01complex numbers, the se ries Ln>o anen
is convergent in H if and only if {an}n~! E e~, in which case -
(14)
(b) For alt x E H, Bessel's inequality holds:

L I(x, e }l2 .::; IIx1l
n 2 (15)
n~O
(c) For alt x E H, the series L (x, en}en converges, and

n~O
L(x,en}en = PG(x), (16)

n~O
where PG is the projection on G.

(d) For alt x, y E H, the series Ln>! (x, en}(y, en ) is absolutely convergent,
and -
L(x, en}(y, en}* = (PG(X), PG(y)} (17)

n~O
Prool: (a) From Pythagoras' theorem we have
and, therefore, {LJ=oajej}n~o is a Cauchy sequence in H if and only if

{LJ=o la j 12}n~0 is a Cauchy sequence in IR. In other words, Ln~o anen con-
C21 Orthononnal Expansions 147
verges if and only if Ln>O lan 12 < 00. In this case equality (14) follows from the
continuity of the norm, by letting n tend to 00 in the last display.
(b) Accordingto (a) ofTheorem Cl.7, IIxll :::: 11 PGn (x)lI, where G n is theHilbert
subspace spanned by {el, ... , en }. But
n
PGn(x) = ~)x, ei}ei,
i=O
and by Pythagoras' theorem,
n
IlpGn (x)11 2 = L I(x, ei}1 2.
i=O
Therefore,
n
IIxll 2 :::: L I(x, ei}1 2,
i=O
from which Bessel's inequality follows on letting n -+ 00.
(c) From (15) and result (a), it follows that the series Ln>o (x, en}en converges.
For any m :::: 0 and for all N :::: m, -
and, therefore, by continuity of the Hermitian product,
( X - L(x, en}en, em) =0 for all m :::: O.

n;::O
This implies thatx - Ln>o(X, en}en is orthogonal to G. Also, Ln>o(X, en}en E G.
Therefore, by the projectlon principle, -
PG(x) = L(x, en}en

n;::O
(d) By Schwarz's inequality in .e~, for all N :::: 0,
(~llx, e")(y, e"),r " (~llx, e")I') (~IIM)I')

~ IIx1l 2 11Y1l2.
Therefore, the series L:'o (x, en ) (y, en )* is absolutely convergent. Also, by an
elementary computation,
148 C2. Compiete Orthononnal Systems
Leuing N -+ 00, we obtain (17) (using (16) and the continuity of the Hermitian
product).
DEFINITION C2.2. The sequence {Wn}n~O oJvectors oJ His said to be total in H
ifit gene rates H.
In other words, the finite linear combination of the elements of {w n }n~O forms
a dense subset of H.
EXERCISE C2.2. Prove that a sequence {wn}n~O oJthe Hilbert space H is total in
H if and only if there is no element oJ H orthogonal to alt the Wn, n ::: 0, except
0, that is, if and only if
((Z, w n ) = 0 Jor alt n ::: 0) ==} (z = 0). (18)

We are now ready for the fundamental result: the Hilbert basis theorem.
THEOREM C2.2. Let {en}n~O be an orthonormal system oJ H. The Jollowing
properties are equivalent:
(a) {en}n~O is total in H;

(b) Jor alt x E H, the Plancherel-Parseval identity holds true:
IIxII 2 = L I(x, e }1
n 2; (19)
n~O
(c)Jor alt x E H,
x = L(x, en}en. (20)

n~O
Proof (a)=}(c) Accordingto(c)ofTheoremC2.1,

L(x, en}en = PG(x),
n~O
where G is the Hilbert subspace generated by {en}n>O. Since {en}n>O is total, it

follows by (18) that Gl. = {O}, and therefore PG(x) :: x. -
(c)=::}(b) This follows from (a) ofTheorem C2.I.
(b)=::}(a) From (14) and (16),
L I(x, en}1 2 = IIPG(x)1I 2 ,
n~O
and (19) therefore implies
From Pythagoras ' theorem,

IIxll 2 = IIPG(x)+x - PG(x)1I 2
= IIPG(x)1I 2 + IIx - PG(x)1I 2
= IIxll 2 + IIx - PG(x)1I 2 ;

C21 Orthononnal Expansions 149
therefore,
IIx - PG (x)1I 2 = 0,
which implies
x = PG(x).
Since this is true for an x E H, we must have G == H, that is, {en}n;::O is total in
H.
A sequence {en }n;::O satisfying one (and then an) of the conditions of Theorem
C2.2 is caned a (denumerable) Hilbert basis of H.
EXERCISE C2.3. Let 1/f be a function in L~ (IR) with the FT VI
= 2~ 1/, where
I = [-2rr, -rr] U [+rr, +2rr]. Show that {1/fj,n}jEZ,nEZ is a Hilbert basis of L~(IR),
where 1/fj,n(x) = 2 j / 2 1/f(2 j / 2 x - n).
EXERCISE C2.4. Let {gj}j;::o be a Hilbert basis of L 2 O, 1]). Show that {gA -
n)ICn,n+1l0}j;::O,nEZ is a Hilbert basis of L 2 (1R). (Here, L 2(I) Denotes the Hilbert
space (equivalence classes) of measurable complex-valued functions defined on
I, with the Hermitian product (f, g) = 1/
f(t)g(t)*dt.)
Biorthonormal Expansions
DEFINITION C2.3. Two sequences {en}n;::O and {dn}n;::O of aHilbert space H form
a biorthonormal system if
(0:) (e n , d k ) = Oforall n =f. k,
() (e n , dn ) = Ifor all n ~ O.
This system is ca lIed complete if, in addition, each ofthe sequences {en}n;::O and
{dn}n;::oforms a total subset of H.
Then we have the biorthonormal expansions
x = L(x, en}dn, x = L(x, dn}en
n;::O n;::O
whenever these series converge. Indeed, with the first series, for example, calling
its sum y, we have for any integer m ~ 0,
= L(x, en}(dn , em } = (x, e m ).

n;::O
Therefore,
(x - y, em ) =0 for an m ~ O.
Since {en}n;::O is total in H, this implies x - y = O.
150 C2. Complete Orthononnal Systems
Separable Hilbert Spaces
An interesting theoretical question is: For what type of Hilbert spaces is there a
denumerable Hilbert basis? Here is a first (theoretical) answer.
DEFINITION C2.4. A Hilbert space H is called a separable Hilbert space if it

contains a sequence {fn}n:::O that is dense in H.
THEOREM C2.3. A separable Hilbert space admits at least one denumerable

Hilbert basis.
Proof Let {fn }n:::O be a sequence defined in Definition C2.4. Construct from
it the orthonormal sequence {en}n:::O by the Gram-Schmidt orthonormalization
procedure. It is a Hilbert basis because (a) of Theorem C2.2 is satisfied. Indeed,
forany zEH,
(en , Z) = 0 for all n ~ 0) ==> (fp, z) = 0 for alln ~ 0).

In particular, (y, z) = 0 for any finite linear combination of {fp}p:::o, Because
{fp}p:::o is dense, (y, z) = 0 for all y E H. In particular, (z, z) = 0, that is,
z=Q
C22 Two Important Hilbert Bases
The Fourier Basis
The following theorem is the fundamental result of the theory of Fourier series of
finite-power periodie signals.
THEOREM C2.4. The sequence
=
{en ( )} def { 1 2i Jr
../Te !l. . }
T , n E
'7J
fLj,
is a Hilbert basis of L~([O, T]).
Proof One first observes that {enO, n E Z} is an orthonormal system in

L~([O, T]). It remains to show that the linear space it generates is dense in
L~([O, T]) (Theorem C2.2).
For this, let f(t) E L~([O, T]) and let fN(t) be its projection on the Hilbert sub-
space generated by {enO, -N ::: n ::: N}. The coefficient of en in this projection
is cn(f) = (f, en)L~([O.Tl)' we have
+N {T (T
L Icn(f)1 + Jo
n=-N
2
0
If(t) - fN(t)1 2 dt = Jo
0
If(t)1 2 dt. (21)
(This is Pythagoras' theorem for projections: 11 PG(X) 11 2 + IIx - PG(X) 11 2 = IIx 11 2 .)

In particular, LnEZ Icn (f)1 2 < 00. It remains to show ((b) of Theorem C2.2) that
lim [T If(t) - fN(t)1 2dt = 0.

Ntoo 10
We assume in a first step that f is continuous. For such a function, the formula
qJ(X) = l T
Jex + t)Jet)* dt,
where
J(t) =L f(t + nT)l(o.nCt + nT),

nEZ
defines a T -periodic and continuous function qJ. Its nth Fourier coefficient is
Cn(qJ)
- + t)f(t)*
= T1 10[T( fex - dt ) e- 2.IITTXn dx,
1 {T _ {{T _
= T 10 10 2. n }
f(t)* fex + t)e- IITTX dx dt
=
- 2
T1 10{T f(t)* {ft+T f(s)e-
t
n
IITTS ds e
} 2 n
IITTt dt
Since LnEZ Icn (f)1 2 < 00 and qJ(X) is continuous, it follows from the Fourier
inversion theorem for locally integrable periodic functions that, for all x E IR,
qJ(x) =L Icn(f)1 2e 2i Jl"j-x.

nEZ
In particular, for x = 0,
and therefore, in view of (21),
lim (T If(t) - fN(t) 12 dt = 0.

Ntoo 10
It remains to pass from the continuous functions to the square-integrable func-
tions. Since the space C([O, Tl) of continuous functions from [0, T] into C is dense
in L~([O, Tl) (Theorem 27), with any 8 > 0, one can associate qJ E C([O, Tl) such
that IIf - qJlI ::::: 813. By Bessel's inequality, IIfN - qJNII 2 = 1I(f - qJ)N1I 2 :::::
11 f - cP 11 2 , and therefore,
IIf - fNIl S IIf - cplI + IIcp - CPNII + IIfN - CPNII

S IIcp - CPNII + 211f - cpll
8
S IIcp - CPNII + 2 3,
For N sufficiently large, 11 cP - cP Nil S 8/3. Therefore, for N suffieiently large,
IIf-fNIIS8.
The Cardinal Sine Basis
The LI-version of the Shannon-Nyquist theorem of Seetion B2l eontains a

eondition bearing on the samples themselves, namely,
(22)
The simplest way of removing this unaesthetie condition is given by the L 2 _

version of the Shannon-Nyquist theorem.
THEOREM C2.5. Let set) be a base-band (B) signaloffinite energy. Then
lim ( Iset) - L+N bn sine (2Bt - n

j2 dt = 0, (23)
Ntoo llR n=-N
where
bn = I-B
+B
s(v) e2irrvfB dv,

Proof' Let L~(lR; B) be the Hilbert subspaee of LUlR) consisting of the finite-
energy eomplex signals with a Fourier transform having its support eontained in
[ - B, +B]. The sequenee
(24)
where h(t) == 2B sine (2Bt), is an orthonormal basis of L~(lR; B). Indeed, the
functions of this system are in L~(lR; B), and they form an orthonormal system
sinee, by the Planeherel-Parseval formula,
= I-B
+B
e2irrvkz-; dv = 2B x ln=k.
It remains to prove the totality of the orthonormal system (24) (see Theorem C2.2).
We must show that if g(t) E L~(IR.; B) and
L g(t)h(t - 2~) dt = 0 for all n E Z, (25)
then g(t) == 0 as a function of L~(IR.; B) (or, equivalently, that g(t) = 0 almost

everywhere).
Condition (25) is equivalent (by the Plancherel-Parseval identity) to
[+B g(v)e 2irrv ii;- dv =0 for all n E Z. (26)

LB
But we have proven in the previous section that the system {e2irrvn/2B }nEZ is total in
LUIR.; B); therefore, (26) implies g(v) = 0 almost everywhere, and consequently,
g(t) = 0 almost everywhere.
Expanding s(t) E L~(IR.; B) in the Hilbert basis (24) yields
+N 1 n
s(t) = lim LCn Mnh(t - - ) , (27)
Ntoo -N v2B 2B
where the limit and the equality in (27) are taken in the L 2-sense (as in (23, and
Cn = [s(t) ~h(t - ..!!...-)dt.

Jrw. v2B 2B
By the Plancherel-Parseval identity,
Cn = I -B
+B 1
./fii
. n
s(v) - - e 2l1tv2ij dv.
An Apparent Paradox
Note that since s( v) is in L 2 and of compact support, it is also in LI, and therefore
the Fourier inversion formula is true and the reconstruction formula takes the
farniliar form
s(t) = Ls(..!!...-) sinc(2Bt - n). (28)

nEZ 2B
This is essentially true, but not quite.
Indeed, imagine that someone tells you the following: Look, I have an proof that
aL 2-signal s(t), base-band (B) is almost everywhere zero! Here is my cute proof.
Of course, the reconstruction equality is in the sense of equality of L 2-functions,
and in particular, it holds only for almost all t. Now, let me change the original
signal to obtain a new signal s'(t) differing froms(t) only atthe times n/2B, where
I set s'(n/2B) = O. I now apply the reconstruction formula and obtain that s'(t) is
almost everywhere zero. But s' (t) and s(t) are almost everywhere equal. Therefore,
s(t) is almost everywhere zero! Quod erat demonstrandum.
The flaw in the above "proof' is that the Fourier inversion formula holds only
almost everywhere, and maybe not at the sampling times. Therefore, formula (28)
is true only if the Fourier inversion formula can be applied at all the times of the
form nj2B. This is the case if s(t) is continuous, because the inversion formula
then holds everywhere.
We see that the continuity hypothesis always pops up. We cannot expect a much
better version of the sampling theorem in the LI or L 2 framework. Indeed, since
s(v) is integrable, the right-hand side of
s(t) = Ls(v)e2invtdv
is continuous, and s(t) is therefore almost everywhere continuous.

We have a sampling theorem for sinusoids and for decomposable signals (Theo-
rem B2.5), and those signals are neither in LI nor in L 2 . Note, however, that they
are continuous.
C3
Fourier Transforms of Finite Energy
Signals
C31 Fourier Transform in L 2

A stable signal as simple as the rectangular pulse has a Fourier transform that is not
integrable, and therefore one cannot use the Fourier inversion theorem for stable
signals as it iso However, there is aversion of this inversion formula that applies
to all finite-energy functions (for instance, the rectangular pulse). The analysis
becomes slightly more involved, and we will have to use the framework ofHilbert
spaces. This is largely compensated by the formal beauty of the results, due to the
fact that a square-integrable function and its Fr play symmetrical roles.
The Isometrie Extension

We start with a technical result. We use 1(.) to denote the function I : lR 1-+ C;
in particular, I(a + .) is the function la : lR 1-+ C defined by la(t) = I(a + t).
THEOREM C3.1. Let set) E LUlR). The mapping Irom lR into L~(lR) defined by
t -+ set + .)
is uniformly continuous.
Proof: We have to prove that the quantity
L'S(t + h + u) - set + u)1 2 du = L1S(h + u) - s(u)1 2 du
tends to 0 when h -+ o. When sO is continuous and compactly supported, the

result follows by dominated convergence. The general case where sO E LUlR)

156 C3. Fourier Transforms of Finite-Energy Signals
is obtained by approximating sO in L~(I~) by continuous compact1y supported

functions (see the proof ofTheorem A1.4).
From Schwarz's inequality, we deduce that
t -+ (s(t + .), s( . ) L~(R)

is uniformly continuous on ~ and bounded by the energy of the signal.
The above function is
t -+ LS(t +x)s*(x)dx (29)
and called the autocorrelation function of the finite-energy signal s(t). Note that
it is the convolution s(t) * s(t), where s(t) = s( -t)*.
THEOREM C3.2. lfthe complex signal s(t) lies in L~(~) n L~(~), then its FT s(v)
belongs to L~(~) and
L Is(t)1 2 dt =L Is(v)1 2 dv. (30)
Praof: The signal s(t) admits s(v)* as FT, and thus by the convolution-
multiplication rule,
(31)
Consider the Gaussian density function

I ,2
h,y(t) = --e-2,;2.
a,J2ii
Applying the result in (14) of Chapter Al, with (s * s)(t) instead of s(t), and
observing that ha(t) is an even function, we obtain
L Is(v)1 2 ha(v)dv =L (s * s)(x)ha(x)dx. (32)
Since ha(v) = e- 2:rr 2a2x 2 t I when a .,j.. 0, the left-hand side of (32) tends to
IR Is(v)1 2 dv, by dominated convergence.
On the other hand, since the autocorrelation function (s * s)(t) is continuous and
bounded, the quantity
L (s * s)(x)ha(x)dx = L (s * s)(ay)hj(y)dy
tends when a .,j.. 0 toward
L (s *s)(O)hj(y)dy = (s *s)(O) = L Is(t)1 2 dt,
by dominated convergence.

C31 Fourier Transform in L 2 157
From the last theorem, we have that the mapping cp: s(t) -+ s(v) from LboR.) n
L~(lR) into L~(lR) thus defined is isometrie and linear. Sinee LI n L 2 is dense
in L 2, this linear isometry ean be uniquely extended into a linear isometry from
LUlR) into itself (Theorem C1.5). We will eontinue both to denote by s(v) the
image of s(t) under this isometry and to eall it the FT of s(t).
EXERCISE C3.1. Show thatfor s(t) E L~(lR),
lim ( 1s(v)
Ttoo J~
-1 -T
+T
2
s(t)e-2irrVldt 1 dv. (33)
The above isometry is expressed by the Plancherel-Parseval identity:
THEOREM C3.3. If SI (t) and S2(t) are finite-energy, complex signals, then
L SI(t)S2(t)* dt = L SI(V)S2(V)* dv. (34)
EXERCISE C3.2. Show that
Im ( Sin(JTv))2
~ JTV
dv = 1.
THEOREM C3.4. If h(t) E Lb(lR) and x(t) E L~(lR), then
y(t) = Lh(t - s)x(s) ds (35)
is almost everywhere weil defined and in LUlR). Furthermore, its FT is
y(v) = h(v)x(v). (36)
Proof Letus first show that f~ h(t-s)x(s) ds is welldefined. Forthis weobserve

that on the one hand ..
L1h(t - s)llx(s)1 ds :::: L1h(t - s)l(l + Ix(s)1 2)ds
= L1h(t)1 dt + L1h(t - s)llx(s)1 2 ds,
and on the other, for almost all t,
L1h(t - s)ll(x(s)1 2 ds < 00,
sinee Ih(t)1 and Ix(t)1 2 are in Lb(lR). Therefore, for almost all t,
L1h(t - sllx(s)1 ds < 00,
and y(t) is almost everywhere well defined. We now show that y(t) E LUlR).
158 C3. Fourier Transforrns of Finite-Energy Signals
Using Fubini's theorem and Schwarz's inequality, we have
LIL h(t - S)X(S) dS I

2
dt
= LIL h(u)x(t - U)dUr dt
= LL{L x(t - u)x(t - v)* dt} h(u)h(v)* du dv
~ (L Ix(s)1 dS) (L Ih(u)1 du Y<

2 00.
For future reference, we rewrite this as
IIh * XIlL~(lR) ~ IIhIlLt(lR)lIxIlL~(lR)' (37)

The signal (35) is thus in L~(IR) when h(t) E LUIR) and x(t) E LUIR). If,
furthermore, x(t) is in Lb(IR), then y(t) is in Lb(IR). Therefore,
x(t) E Lb(lR) n L~(IR) -+ y(t) E Lb(lR) n L~(IR). (38)
In this case, we obtain (36) by the convolution-multiplication formula in L I .
We now suppose thatx(t) is in L~(IR) (butnot necessarily in Lb(IR. The !signal
XA(t) = x(t)l[-A,+Aj(t)
isinLb(lR)nL~(IR)andlimxA(t) = x(t)inL~(IR).Inparticular,limxA(v) = x(v)
in L~(IR). Introducing
YA(t) = L h(t - s)xA(s)ds,
we have YA(V) = h(V)XA(V), Also, lim YA(t) = y(t) in L~(IR) [use (37)], and thus
limYA(v) = y(v) in L~(IR). Now, since limxA(v) = x(v) in L~(IR) and h(v) is
bounded, lim h(V)XA(V) = h(v)x(v) in L~(IR). Therefore, we have (36).
EXERCISE C3.3. Use the Plancherel-Parseval identity to prove that
{dt 7t:
llR (t 2 + a 2 )(t 2 + b2 ) - ab(a + b)'

EXERCISE C3.4. Show that
H = {w(t) E L~(IR);tw(t) E L~(IR), vw(v) E L~(IR)}
is a Hilbert space when endowed with the norm
1
IIwllH = (lIwll~2 + IItwll~2 + IIvwll~2P .
Show that the subset 0/ H consisting 0/ the C oo -functions with compact support
is dense in H. Hint: Select any q; in a Coo-function with compact support, with
C32 Inversion Formula in L 2 159
integral equal to 1, and equal to 1 in a neighborhood ofO, andfor any wEH,

*
consider the function w(t)cp(t / n) ncp(nt).
C32 Inversion Fonnula in L 2

So far, we know that the mapping cp : L~(~) 1-+ L~(~) defined in Seetion C31
is linear, isometrie, and into. We shall now show that it is onto, and therefore
bijeetive.
THEOREM C3.5. LetS(v) be the FT of s(t) E L~(~). Then
cp : s( -v) --+ s(t), (39)
that is,
s(t) = lim
Atoo
j-A
+A
s(v)e 2i Jl'vt dv, (40)
where the limit is in LU~), and the equality is almost everywhere.
We shall prepare the way for the proof with the following result.
LEMMA C3.1. Let u(t) and v(t) be two jinite-energy signals. Then
L u(x)v(x)d.x = L u(x)v(x)d.x. (41)
Proof' If (41) is true for u(t), v(t) E L~(~) n L~(~), then it also holds for
u(t), v(t) E L~(~). Indeed, denoting XA(t) = x(t)I[-A,+Aj(t), we have
L u A(x )(v;)(x) d.x = L (l7;)(x)v A(x) d.x,
that is, (UA, VA) = (UA, VA)' Now UA, VA, UA, and VA tend in LU~) to u, V, u,
and V, respeetively, as A t 00, and therefore, by the eontinuity of the Hermitian
produet, (u, v) = (u, v).
The proof of (41) for stable signals is aeeomplished by Fubini's theorem:
L u(x)v(x)d.x = L {L u(x) v(y)e- 2i Jl'xy d Y } d.x
= L {L v(y) u(x)e- 2i JI'XY d Y } dy
=L v(y)u(y)dy.
160 C3. Fourier Transforms ofFinite-Energy Signals
Proof of(39); Let g(t) be a real signal in L~(lR), and define f(t) = (g~)(t), where
g-(t) = g(-t). We have j(v) = g(v)*. Therefore, by (41):
L g(x)j(x)dx = L g(x)f(x)dx
= f (x)(x)* dx.
Therefore,
IIg - f112 = IIgll 2 - 2Re (g, j) + IIfII 2
= IIgll 2 - 211g11 2 + IIfII 2 .
But IIgll 2 = IIgll 2 and IIfll 2 = IIg1l 2 . Therefore, IIg - f112 = 0, thatis,
g(t) = j(t). (42)
In other words, every real (and, therefore, every complex) signal g(t) E L~(lR)
is the Fourier transform of some function of LUIR). Hence, the mapping q; is
onto.
EXERCISE C3.5. Show that if a stable signal is base-band (that is, if its FT has
compact support), then it also has afinite energy.
We dose this seetion by showing how the LI Fourier inversion theorem was
lirnited in scope, since it does not take much for a stable signal not to have an
integrable FT.
EXERCISE C3.6. Show that if a stable signal is discontinuous at a point t = a, its
FT is not integrable.
C4
Fourier Series of Finite Power Periodic
Signals
C41 Fourier Series in Lfoc

Let us consider the Hilbert space e~ of complex sequences a = {an}, n E Z, such
that LnEZ lan l2 < 00, with the Hermitian product
(a, b}e~ = L>nb~ (43)

nEZ
f:
and the Hilbert space L~([O, T], dt/T) of complex signals x = {x(t)}, t E lR.,
such that Ix(t)1 2 dt < 00, with the Hermitian product
Jorx(t)y(t)* T'
T dt
(x, Y}L~([O,T],~) = (44)
THEOREM C4.1. Formula
Sn = -1
T
l
0
T
s(t)e- 2"17r'i/
n dt
(45)
defines a linear isometry sO --+ {Sn} lram L~([O, T], dt / T) onto e~, the inverse
01 which is given by
s(t) = L::Sne2i1l"f/ , (46)
nEZ
where the se ries on the right-hand side converges in L~([O, T], dt / T), and the
equality is almost everywhere. This isometry is summarized by the Plancherel-

162 C4. Fourier Series of Finite-Power Periodic Signals
Parseval identity:
LXnY~ = ~ (Tx(t)y(t)* dt. (47)

nEZ 10
Prao!" The result follows from general results on orthonormal bases of Hilbert
spaces, since the sequence
{e n (-)} ~ {Jre 2i1t T'}' nE Z,
is a complete orthonormal sequence of L~([O, Tl. dt / T) (Theorem C2.4).
The L 2 inversion theorem tells us that if s (t) is a T -periodic complex signal

with finite power, then
+N L~([O, Tl)
L Sn e2i1t (n/T)t ------+ set).
-N Ntoo
In general, for an arbitrary sequence of functions of L~([O, Tl), convergence

in L~([O, Tl) does not imply convergence almost everywhere. However, for
sequences of partial Fourier series, we have the surprising Carleson's theorem:
THEOREM C4.2. The Fourier se ries 01 a T -periodie signal s(t) with finite power
converges almost everywhere to set).
This result shows that the situation for finite-power periodic signals is pleasant,
in contrast with the situation prevailing locally stable periodic signals (remember
Kolmogorov's result, Theorem A3.1). The proof ofCarleson's result is omitted; it
is rather technical. It also shows that the L 2 framework is very adapted to Fourier
series, since everything works "as expected."
Discrete-Time Fourier Transform of Finite-Energy Signals
Let lb be the space of sequences In, n E Z, such that LnEZ Ilnl < 00 (stable
discrete-time signals).
THEOREM C4.3. lb c l~, that is, a discrete-time stahle signal has finite energy.
Prao!" Let A = {n: IX n I ~ I}. Since LnEZ IX n I < 00, then necessarily
card(A) < 00. On the other hand, if Ixnl ::::: 1, Ix n l2 ::::: Ixnl, whence

The situation for discrete-time signals is therefore in contrast with that of
continuous-time signals, for which there exist stable signals with infinite energy
that and finite-energy signals that are not stable.
C42 Orthonormal Systems of Shifted Functions 163
Let L~(2n) be the Hilbert space of functions j: [-n, +n] ~ C such that
f~: Ij(w)1 2 dw < 00 provided with the Hennitian product
(], g)L~(2n) = _1
2n
l+
-n
n
j(w)g(w)* dw.
THEOREM C4.4. There exists a linear isomorphism between LU2n) and .e~
dejined by
fn = l+ -n
n
j(w)e inw dw ,
2n
j(w) =L
nEZ
fn e- inw . (48)
In particular, we have the Plancherel-Parseval identity
L fng: = -2n1 l+
nEZ -n
n
-
f(w)g(w)* dw. (49)
Proof: This is arestatement of Theorem C4.3.

C4 2 Orthonormal Systems of Shifted Functions
We give a necessary and sufficient condition in the frequency domain for a system
of shifted functions to be orthonormal.
THEOREM C4.5. Let g(t) be afunction of L~(lR.) andjix 0 < T < 00. A necessary
and sufficient condition for the family offunctions
{g(. - nT)}nEZ (50)
to fonn an orthononnal system of L~(lR.) is
L
nEZ
Ig(v + -f )1 2
= T almosteverywhere. (51)
Proof: The Fourier transform g( v) of g(t) E L~(lR.) is in L~(lR.) and, in particular,

Ig(v)1 2 is integrable. By Theorem A2.3, LnEZ Ig(v + (n/T1 2 is (I/T)-periodic
and locally integrable, and T In~. g(t) g(t - nT)* dt is its Fourier coefficient (this
L L
follows from the Plancherel-Parseval formula:
T g(t)g(t - nT)* dt =T Ig(v)12e-2invnT dv
The definition of orthonormality of system (50) is
L g(t) g(t - nT)* dt = I n=o.

The proof then follows the argument in the proof of Theorem B2.6.

Riesz's Basis
The following notion will play an important role in multiresolution analysis.
DEFINITION C4.1. A system offunctions of LU~)
{w( - nT)}nEz (52)
is said to form a Riesz basis of some Hilbert subspace Vo of LU~) if

(a) it spans Vo, and
(b)forall sequence, {ckhEZ ofe~(Z),
AL 1ck1 2 ::s
k
(IL
Jffi. kEZ
CkW(t - kT)1
2
dt ::s B L
kEZ
Icd, (53)
where 0 < A ::s B < 00 are independent ofthe Ck.
Thefunction LkEZ Ck w(t-kT) has theFouriertransform LkEZ cke-2inkTvw(v),

and, therefore, by the Plancherel-Parseval identity, the term between the bounds
(IL
in (53) is equal to
Cke-2itrkTVW(V)12 dv
Jffi. kEZ
Also,
Now, any function c(v) E L~([O, I/T]) has the form LkEZcke-2itrkTv, where
LkEZ ICkl 2 < 00, and (53)is therefore equivalent to
AT 1 1fT
Ic(v)1 2 dv::s 1 IC(V)121~
1fT
Iw(v + f )1 1
2
dv
::s BT Jot fT
Ic(v)1 2dv
C42 Orthononnal Systems of Shifted Functions 165
for any c(v) E L~([O, IjT]). It then foIlows that
ATS~lw(v+ ~)12 SBT a.e. (54)
THEOREM C4.6. Let (w( - n T) }neZ be a Riesz basis of some Hilbert subspace
Vo C L~(R). Define the function g E L~(R) by its Fourier transform
(55)
Then (g(. - nT)}nez is a Hilbert basis ofVo.

Proof" In view of (54), the function gis weIl defined and in L~(R). Since (51)
is obviously satisfied, it foIlows that the system {g(. - nT)}neZ is orthonormal.
We must now show that the Hilbert space Vo spanned by (g(. - nT)}neZ is in
fact identical to Vo.1t suffices to show that the generators of Vo belong to \10, and
vice versa. Define
In view of condition (54), a( v) is (1 j T)-periodic and offinite power, and it therefore

admits a Fourier representation
a(v) = L ane-2i7CvnT,
neZ
for some sequence {an} E l~(Z). Since

w(v) = a(v)g(v),
it foIlows that
w(t) = L ang(t - nT).
neZ
Therefore, the generators of Vo are in Vo. The converse is true by the same argument
since
g(v) = a(v)-Iw(v),
where a(vr l is also, in view of condition (54), a (ljT)-periodic of finite
power.
EXERCISEC4.1. Let h(t) be the function of FT (ljJ2B)I[-B.+Bj(v) for some
B > O. Show that there exists no orthonormal basis of L~(R; B) of the form
{k g (t - 2~) }nd where
g(t) = LCkh (t - ~),
keZ 2B
where {cn} nEZ is in CU7-), unless only one of the Cn is nonzero. Show that if the
Fourier sum c(w) ofthe sequence {cn}nEZ is such that A < !c(w)!2 < B for some
o< A :s B < 00 and for all w, then {k g (t - 2~)}
nEZ
is a Riesz basis of
L~(lR; B).
References
[Cl] Daubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf.
[C2] Gasquet, C. and Witomski, P. (1991). Analyse de Fourier et Applications, Masson:
Paris.
[C3] Halmos, P.R. (1951). Introduction to Hilbert space, Che1sea, New York.
[C4] Rudin, W. (1966). Real and Complex Analysis, McGraw-Hill, New York.
[C5] Young, N.Y. (1988). An Introduction to Hilbert Spaces, Cambridge University
Press.
Part D
Wavelet Analysis
Introduction
Although Fourier theory had reached in the L 2 framework a formal mathematical

beauty, it was not entirely satisfactory for important applications in signal process-
ing. Indeed, in many situations, the information contained in a signal is localized
in time and/or in frequency. The typical example is a piece of music, which is
perceived as a succession of notes welliocalized in both time and frequency. The
usual Fourier transform is not adapted to the analysis of music because for a given
frequency (a note) it is related to the total energy of all occurrences of this note in
the entire piece.
This led Dennis Gabor 1 to propose a windowed Fourier transform, whose idea
is very natural. If f(t) is the signal to be analyzed, the local information at time
t = bis contained in the time-Iocalized signal
f(t)w*(t - b),
where w(t) is the time window function, a function negligible outside a relatively
small interval around zero.
Given a window w(t), the local information at time b is obtained by computing
the Fourier transform of the last display:
Wf(v, b):= L f(t)w*(t - b)e-2invt dt = (j, Wv,b),
where
Wv,b(t) = w(t - b)e2invt.
ITheory ofCommunications, J. Inst. Elec. Engrg., Vol. 93, pp. 429-457,1946.

170 Part D Wavelet Analysis
We see that the time information is collected in a time interval around time b of
width of the order of that of the time window. Now, from the Plancherel-Parseval
identity,
Therefore, we see that the frequency information is collected in a frequency interval

around v of the width of the time window's Fr.
These observations point to a fundamental limitation of the windowed Fourier

transform, in relation with the uncertainty principle, which states that time res-
olution is possible only at the expense of frequency resolution, and vice versa.
Indeed, the wider the function, the narrower its Fourier transform, and vice versa.
Of course, this is an imprecise statement, but it is already substantiated by the
Doppler theorem
(Note that as a varies, the energy of lai! !(at) remains the same.) The so-called
Heisenberg uncertainty principle makes the above limitation more explicit and
states that
where a f is the mean-square width of a function ! E L~(lR) (see the definition

in the first lines of Section D 11). The conclusion is that as long as we resort
to windowed Fourier transforms, time resolution and frequency resolution are
antagonistic: If one is increased, the other is necessarily decreased, keeping the area
awafij ofthe time-frequency box above the lower bound of 1/41T. However, the real
inconvenience of the windowed Fr is the fixed shape of the time-frequency box. In
many occasions, it is interesting to have a time-frequency box that adapts itself to
the time-frequency point analyzed. For instance, a discontinuity (abrupt change)
of a signal takes place in a short time and involves high frequencies. Therefore, at
high frequencies, the time dimension of the time-frequency box should be small.
Also, since it takes time to determine the frequency of a low-frequency sinusoid,
at low frequencies the time dimension of the time-frequency box should be large.
Motivated by these imperatives, lean Morlet2 proposed the wavelet transform in
order to take into account the need for an adaptive time-frequency box.
2 Sampling theory and wave propagation, in NATO ASI series, Vol. 1, Survey in Acoustic
Signal! Image Processing and Recognition, C.H. ehen, ed., Springer-Verlag, Berlin, 233-
261,1983.
Introduction 171
In the wavelet transfonn, the role the family of functions Wv,b(t) plays in the
windowed Fourier transfonn is played by a family
(t - b)
1fta,b(t) = lai -1/2 1ft -a- , a, b E IR, a =I- 0,
where 1ft(t) is called the mother wavelet. The wavelet transfonn (WT) of the
function f E L~(IR) is the function
C j(a, b) = (j, 1fta,b) = L f(t)1ft;,b(t) dt, a, b E IR, a =I- 0.

C j(a, b) = (j, :(fra,b) = L !(V):(fr;,b(V) dv,
where
:(fra,b(V) = laI 1/ 2 e- 2i 1l'Vb:(fr(av).
From the above expression, it appears that the wavelet transfonn C j (a, b) analyzes
the function f(t) in the time-frequency box
[b + am - au, b + am + au] x [-m - -,

8- m 8-]
- + -
a a a a
m
(where m is the center of 1ft, u is its width; and 8- are the corresponding objects
relatively to :(fr; see the details in the main text). Assuming > 0, we see that m
the frequency window is centered at v = m/
a and has width 28- / a; therefore, the
ratio center/width
m
~
Q= 28-
is independent of the frequency variable a. The area of the time-frequency box
is constant, but its shape varies with the frequency v = m/
a analyzed. For high
frequencies it has a large time dimension, and for small frequencies it has a small
time dimension, which is the desired effect.
The interest of the wavelet transfonn for signal analysts is that they can "read"
it to extract infonnation about the time-frequency structure that is otherwise
blurred in the brute signal by concurrent phenomena and subsidiary effects. For in-
stance, they can detect the appearance time of a phenomenon linked to a particular
frequency (e.g., the time at which a particular atom starts to be excited).
A fonnula, called the identity resolution, allows us to reconstruct, under mild
conditions that we shall make precise in due time, the function from its wavelet
transfonn:
f(t) = ~ { ( C j(a, b)1fta,b(t) dll ~b ,

K JlRJlR a
where K is some constant depending on 1{1. In this sense, wavelet analysis is
continuous, in that the original function of L 2 is reconstructed as a continuous linear
172 Part D Wavelet Analysis
combination ofthe continuous wavelet basis 1J!a,b(t) = lal-l/21J!e~b) , a, bE lR.,

a =1= O. One would rather store the original function not as a function of two
arguments, but as the doubly indexed sequence of coefficients of a decomposition
along an orthonormal base of L 2 , {1J!j,nb,nEZ, called the wavelet basis,
f = L LU, 1J!j,k)1J!j,k.
JEZ kEZ
where
j .
1J!j,n(t) = 2 2 1J!(2J t - n),
and where 1J!(t) is the mother wavelet. The multiresolution analysis of Stephane
Mallat3 is one particular way of obtaining such orthonormal bases and is the main
topic of Part D. Similarly to the continuous wavelet transform, the coefficients of
the multiresolution decomposition can be used to analyze a signal. But multires-
olution analysis is also a tool for data compression. Indeed, with a good design of
the mother wavelet, many wavelet coefficients are small and can be neglected. The
coefficients that are not neglected are quantized more or less coarsely, depending,
for instance, on the frequency index. This is of course reminiscent of subband
coding, and this resemblance is not at all a coincidence. Mallat's algorithm of
analysis-synthesis is of the same form as the subband analysis-synthesis algor-
ithm. Multiresolution analysis can be considered as a systematic way of doing
subband analysis. One of its advantages is to place the latter in a framework where
the mathematical issues ensuring the efficiency of the algorithms are more easily
dealt with.
Perhaps one of the most striking advantages of multiresolution analysis over
classical Fourier analysis is in the way it handles discontinuities. Consider, for
instance, the signal on top of Figs. DO.la and DO.lb,4 which has a spike. In
both figures the middle signal is a Fourier series approximation of the top sig-
nal, whereas the bottom signal is a wavelet approximation of the top signal. In Fig.
DO.I a, only the first 60 coefficients of the expansions (Fourier or wavelet) are used
to produce the approximated signals, and it appears that there is not a dramatic
difference between the two approximations. This is not the case, however, when,
as in Fig. DO.I b, one uses the 60 largest coefficients. The advantage of the wavelet
approximation is then obvious.
3Multiresolution approximation and wavelets, Trans. Amer. Math. Soc., 315, 1989,69-
88.
4Reproduced with the kind permission of Martin Vetterli.
Introduction 173
0.5
of---------'
o 100 200 300 400 500 600 700 800 900 1000
0.5
o
o 100 200 300 400 500 600 700 800 900 1000
0.5
01------"--"--"'\
o 100 200 300 400 500 600 700 800 900 1000
(a)
0.5
01------'
o 100 200 300 400 500 600 700 800 900 1000
0.5
o
o 300 400 500 600 700 800 900 1000
0.5
01------'
o 100 200 300 400 500 600 700 800 900 1000
(b)
Figure DO.I. Wavelet VS. Fourier

Dl
The Windowed Fourier Transform
D 11 The Uncertainty Princip1e

Fourier analysis, weH as wavelet analysis have an intrinsic limitation, which is
contained in the uncertainty principle. In order to state this result, we need a
definition of the "width" of a function. Here is the one that suits our purpose.
Root Mean-Square Widths
Let w : lR 1-+ C be a nontrivial function in L 2 Define the centers of w and W,
respectively, by
mw = -1
Ew IR
1 t Iw(t)1 2 dt,
mfjj = _1_
Ew
r
JIR
v Iw(v)1 2 dv,
where E w is the energy of w(t):
Ew = L Iw(t)1 2 dt = L Iw(v)1 2 dv.
Then define
and

176 D 1. The Windowed Fourier Transforrn
The numbers o"w and 0"1ii are the root mean-square (RMS) widths of w and w,
respectively. Note that m w and mlii are not always defined. When they are weIl
defined, o"w and 0"1ii are always defined but may be infinite. Therefore, we shall
always assurne that
and
L Ivllw(v)1 2 dv < 00,
to guarantee at least the existence of the centers of w(t) and of its Fr.
EXERCISE D1.I. Check if the centers m w and mlii are well defined, and then
compute o"w and 0"1ii and the product O"wO"Iii in thefollowing cases:
w(t) = l[-T,+Tl
w(t) = e-a1tl , a > 0,
w(t) = e- at2 , a > O.
EXERCISEDI.2. Suppose that the centerm w ofthefunction w E L 2 iswell defined.
Show that the quantity
L It - toI 2 Iw(t)1 2 dt
is minimized by to = m w .
Heisenberg's Inequality
THEOREM DI.I. Under the conditions stated above, we have Heisenberg's
inequality
(1)
Proof: We assurne that the window and its Fr are centered at 0, without loss of
generality (see Exercise D1.3). Denoting the L 2 -norm of a function f by IIfII, we
have to show that
~ 1 2
IItwll x IIvwll :::: 4n IIwll ' (2)
We first assurne that w is a COO-function with bounded support. In particular,

td(v) = (2inv) w(v),
and therefore,
D 11 The Uncertainty Principle 177
Thus, it remains to show that

1
IItwll x IIw'lI ::: "2 IIw1l 2 (3)
By Schwarz's inequality,
IItwll x IIw'lI ::: I(tw, w')1 ::: IRe{(tw, w')}I.
Now,
2Re{(tw, w')} = (tw, w') + (w', tw) = l t(ww'* + w'w*)dt
= tlw(t)121~~ -lIW(t) 12 dt = 0 - IIw1l 2

This gives (3) in the case where w E Coo . We now show that (3) is true in the
general case. To see this, we first observe that it suffices to prove (3) in the case
where w belongs to the Hilbert space
H = {w E L~(lR); tw(t) E L~(lR), vw(v) E L~(lR)},
1
with the norm 11 w 11 H = (11 W11 2 + IItw 11 2 + 11 vwll2) 2: (if w(t) is not in this space,
Heisenberg's inequality is trivially satisfied). Then we use the fact that the subset of
H consisting ofthe Coo -functions with compact support is dense in H (see Exercise
C3.4). The result then follows from the continuity ofthe Hermitian product.
Equality in (1) takes place if and only if w(t) is proportional to a Gaussian signal
e- ct2 , where c > O. We do the proof only in the case where it is further assumed
that WES and is real. Observe that all the steps in the first part of the proof remain
valid since for such a function, tlw(t)121~~ = O. Equality is attained in (1) if and
only if the functions tw(t) and w'(t) are proportional, say,
w'(t) =- ctw(t),
and this gives

w(t) = Ae-ct2 ,
where c > 0 necessarily, because w(t) E L2.

EXERCISE D1.3. Show that, for arbitrary to E lR, Vo E lR,
~ 1 2
lI(t - to)wll x lI(v - vo)wll ::: 4n IIwll . (4)
(Hint: Consider the function g(t) = e-2invot w(t + to).)
The above resulttells us that in Heisenberg's inequality (1), the numbers a w and
aliJcan be taken to be, respectively, the root mean-square width of w around any
time to and the root mean-square width of w around any frequency vo. In particular,
to can be the center of w(t), and Vo can be the center of w(v). This version is the
most stringent, since the RMS widths around the centers are the smallest RMS
widths (see Exercise D1.2).
178 D 1. The Windowed Fourier Transforrn
D 1 2 The WFf and Gabor' s Inversion Formula

Windows
Let f(t) be the signal to be analyzed. The local information at (around) time t = b
is contained in the time-Iocalized signal
f(t)w*(t - b), (5)
where w(t) is the time window function, the support of which is included in a
relatively small interval about zero. Typical examples are the rectangular window
w(t) = l[-a,+aj(t)
and the Gaussian window
wherea > O.
Given a window w(t), the local information at time b is obtained by computing
the Fourier transform of (5):
Wf(v, b):= L f(t)w*(t - b)e-2irrvt dt. (6)
The choice of w* instead of w in (5) is purely for notational cornfort. For

example,
Wf(V, b) = (j, Wv,b), (7)
the Hermitian product in L~(lR) of f(t) with
Wv,b(t) = w(t - b)e2irrvt, (8)
where we assume, of course, that fand w are complex-valued functions that have
finite energy.
DEFINITION DI.I. Let w, f L 2. One caUs the function Wf : lR x lR -+ C
E
defined by (6) the windowed Fourier transform (or WFT) of f associated with the
window w.
When the rectangular window is chosen, W f is called the short-time Fourier

transform of f. If the window is Gaussian, the function Wf is called the Gabor
transform of f.
Inversion Formula
THEOREM DI.2. Under the foUowing assumptions,

(a) w E Lb(lR) n LUlR),
(b) LIw(t)1 2 dt = 1,
D 1 2 The WFf and Gabor' s Inversion Formu1a 179
(c) Iwl is an evenfunction,

we have, lor all I E L~(lR), the energy conservation formula
LLIWf (v,b)1 2 dvdb = LI/(t) 12 dt, (9)
and the reconstruction formula, (inversion formula),
lim ( I/(t)
Atoo JR.
-1 JR.(
Ivl:",A
Wf(V, b)Wv,b(t) dv dbl2 dt = O. (10)
Note that assumption (c) is automatically satisfied if w(t) is areal function.

Assumption (b) is just a convention: The window can always be normalized to
have an energy equal to 1, and the wavelet transform is then just multiplied by a
constant.
Proof The proof is technical and can be skipped in a first reading. We define
IA(t) = 1 JR.(
Ivl:",A
Wf(v, b)Wv,b(t) dv db.
Using the Plancherel-Parseval identity, (7) becomes

Wf(v, b) = (I, Wv,b),
where
Wv,b(f.1,) = e-2irr (/l--v)b W(Jl_ v) (11)
is the Fourier transform of (8). Therefore,
Wf(v, b) = e-2irrvb L I(Jl)w(Jl- v)*e 2irr /l- b dJl. (12)
The function Jl -+ I(Jl)w(Jl - v)* is in LI n L 2. (It is in LI as the product of

!
two L 2-functions; it is in L 2 because E L 2 and Wis bounded, being the Fourier
transform of an L I-function.) By the Plancherel-Parseval identity,
L IWf(v, b)1 2 db = L IL I(Jl)w(Jl- v)*e 2irr /l- b dJl l

2
db
=L 1!(Jl)w(Jl - v)*1 2 dJl,
and, therefore,
L L IWf(v, b)1 2 dbdv = L L II(Jl)1 2Iw(Jl- v)1 2 dJldv
=L {1!(Jl)1 2 L IW(Jl - v)1 2 dV} dJl
=L {11(Jl)1 2 L Iw(v)1 2 dV} dJl.

180 D 1. The Windowed Fourier Transform
Equality (9) foHows because
L Iw(v)1 2 dv = L Iw(t)1 2 dt = 1,
L Ij(JL)1 2 dJL = L IJ(t)1 2 dt (Plancherel-Parsevalidentity).
We show that the function JA is weH defined, that is, (v, b) -+ Wj(v, b)Wv,b(t) is
integrable over [- A, + A] x R In view of (12),
IA(t) := j+A { IWj(v, b)llwv,b(t)1 dv db

-A JIR
By Schwarz's inequality and the Plancherel-Parseval identity, and using assump-

tion (b),
L 1.r-I{j(.)W(. - V)*}(b)IIW(t - b)1 db
:s ( L l.r-I{j(.)w(. - v)*}1 db
2 )1/2 (
L
Iw(t - b)1 2 db
)1/2
= (L l.r-I{j(.)w(. - V)*}1 2 db )1/2
= L Ij(JL)w(JL - v)*1 2 dJL
= (lj12 * Iw 12 ) (v) := h(v).
This function h(v) is in LI, being the convolution product oftwo LI-functions. In
particular,
IA(t):s
j +A
-A Ih(v)1 dv < 00,
and JA is therefore weH defined. Using a previous calculation, we have
JA(t) = j+A
-A g(v) dv,
where
g(v):= L .r-I{j(.)w(. - v)*}(b)w(t - b)e2inv (t-b) db.

Dl2 The WFT and Gabor's Inversion Forrnula 181
g(v) = L !(f.1.)w(f.1. - v)*w(f.1. - v)e 2in /Lt df.1.
= L !(f.1.)lw(f.1. - v)1 2 e2in /Lt df.1..
Therefore,
fACt) = i: (L A
!(f.1.)lw(f.1. - v)1 2 e2in /Lt df.1. ) dv.
In order to change the order of integration in the above integral, we first verify that
(v, f.1.) -+ 1!(f.1.)llw(f.1. - v)1 2 is integrable over [- A, + Al x lR. But I!I E L 2 and
Iwl 2 E L 1 ; therefore, I!I * Iwl 2 E L 2 , and the integral of an L 2-function over a
i: (L i: (L
finite interval is finite. But this integral is just
A A
1!(f.1.)llw(v - f.1.)1 2d f.1.) dv = 1!(f.1.)llw(f.1. - V)1 2d f.1.) dv
since Iwl is even. We can now apply Fubini's theorem to obtain
fACt) =L(i: A
!(f.1.)lw(f.1. - v)1 2e2in /Lt dV) df.1.
=L (i: !(f.1.)e 2in /Lt

A
Iw(f.1. - v)1 2 dV) df.1.
=L !(f.1.)C{JA(f.1.)e 2in /Lt df.1.,
where 0:::: C{JA(f.1.) :::: 1, in view of assumption (b). In particular, !C{JA E L 2, and
1
fA = F- ~
(jC{JA).
We now show that limA too fA = f in L 2. For this, we write, using the Plancherel-
Parseval identity,
IIf - fAlli2 = IIF- 1! - F- 1!C{JAlli2
= IIF- 1{!(1 - C{JAmi2
Wehave
= j /L-A
-00
Iw(Y)1 2 dy +
1+00
/L+A
Iw(Y)1 2 dy,
182 D 1. The Windowed Fourier Transform
and, therefore, if I/LI ::; A12,

o ::; 1 - ((JA(/L)
::; j -A 12
Iw(Y)1 2 dy +
1+00 Iw(Y)1 2 dy
-00 AP
= y(A) -+ 0 (A -+ (0).
Also,
-+ 0 (A -+ (0)
sinee j E L 2 . Therefore, finally,
lim
Atoo
111 - IAlli2 = O.
From (7) and the Planeherel-Parseval identity, we obtain the two expressions
for the windowed Fourier transform:
(13)
where WV.b is defined by (11). We assume (without diminishing the generality of

the diseussion to follow) that w and W are funetions eentered at zero, that is,
L tlw(t)1 2 dt =0 and L vlw(v)1 2 dv = O.

The RMS width a w is an indicator of the loealization of Wv.b about b, whereas
aJij is an indieator ofthe loealization of WV.b about v. The reetangle [b - a w , b +
a w ] x [v - aJij, v+ aJij] is the loeal time-frequeney box about (b, v) analyzed by
the windowed Fourier transform at (b, v).
It is of interest to have a sharp resolution, that is, to make the area 4awaJij of
the time-frequeney box as small as possible. However, windows have the basic
limitation eontained in the uneertainty prineiple, whieh says that
1
awaJij:::: 4n ' (14)
with equality if and only if w(t) == Ae-ct2 , where c = 4n2a~. The last result
shows that.
THEOREM D1.3. The Gabor window is optimal, in the sense that it minimizes the
uncertainty aJijaw.
Dl2 The WFf and Gabor's Inversion Formu1a 183
The windowed Fr is a continuous transform, in that the local time-frequency

content of a signal is contained in a function of two continuous arguments. It would
be interesting to have a discrete version, that is, a decomposition of the signal along
a Hilbert basis. More specifically, one asks the question: Is there a window w(t)
such that the family {lj!m,n}mE71,nE71, where
lj!m,n(t) = e2irrmtw(t - n), (15)
is an orthonormal basis of L 2 (ffi.)?

EXERCISE 01.1. Show that the answer to the previous question is positive for the
rectangular window w(t) = 1[0,1](t).
Although such "atomic" windowed Fr bases do exist, they turn out to be very
bad from the view point of time-frequency resolution, as the following result,
called the Balian-Low theorem,5, shows.
THEOREM 01.4. If{ lj!m,n}mE71,nE71, where lj!m,n is defined by (15)(with gEL 2 (ffi.),
is an orthonormal basis of L 2 (ffi.), then at least one of the following equalities is
true:
L t 2Iw(t)1 2dt = 00 or Lv 2Iw(v)1 2dv = 00.
EXERCISE 01.2. Show that the system {lj!m,n}mE71,nE71, where

lj!m,n(t) = e2irrmt e-a (t-n)2,
()( > 0, is not an orthonormal basis of L 2 (ffi.).
SR. Balian, Dn principe d'incertitude fort en theorie du signal et en mecanique quantique,

CR Acad. Sei. Paris, 292, Sero II, 1981, 1357-1361; F. Low, Comp1ete sets of wave pack-
ets, A Passion for Physics-Essays in Honor of Geoffrey Chew, 17-22, World Scientific:
Singapore, 1985.
D2
The Wavelet Transform
D21 Time-Frequency Resolution of Wavelet

Transforms
Definition of the Wavelet Transform

We mentioned in the introduction to Part D the shortcomings of the windowed
Fourier transform. This chapter gives another approach to the time-frequency
issue of Fourier analysis. The role played in the windowed Fourier transform by
the family of functions
Wv,b(t) = w(t - b)e+ 2i :rrvt, b, v E IR,
is played in the wavelet transform (WT) by a family
1/Ia,b(t) -_ lai -1/2 1/1 (t - b)

-a- , a, b E IR, a =I 0, (16)
where 1/I(t) is called the mother wavelet. The function 1/Ia,b is obtained from the
mother wavelet 1/1 by successively applying a change of time scale (accompanied
by a change of amplitude scale in order to keep the energy constant) and a time
shift (see Fig. D2.1).
DEFINITION D2.1. The wavelet transform ofthe function f E LUIR) is the function
C j : (IR - {On x IR 1-* C defined by
Cj(a, b) = (j, 1/Ia,b) = l f(t)1/I:,b(t)dt. (17)

186 D2. The Wavelet Transform
1lV2
11
,-------, 1
11 1
11 1y'2
1
1 .
11
1 )
-1 0 11 1+1
1
3.51 1 17.5
1
t
1 1 1
1 1
-y'2
-1 L - J 1 1
-V2 L.....
'ljJ(t) 'ljJ! ,3.5 (t) 'ljJ2,7.5(t)
Figure D2.1. Dilations and translations
The Adaptive Time-frequency Box

C j(a, b) = (j, Vla,b) = 1 j(v) VI;,b(V) dv, (18)
where
(19)
Let m", and a", be, respectively, the center and RMS width of the mother wavelet
1/1 , respectively defined by
m", = -1
E",
1lR I 11/I(t)1 2 dl,
aJ = _1_ r(I -
E", llR
m",)211/1(1)1 2 dt,
and similarly define m;;; and a;;;, where VI is the Fourier transform of 1/1. The center
and width of 1/1a,b are, respectively,
b+am"" aa""
whereas the center and width of Vla,b are
1 1
-m;;;, -a;;;.
a a
We shall simplify notations by writing
We see that C j(a, b) is the result of the analysis of the function f in the time-
frequency box (see Fig. D2.2)
[b+am-aa,b+am+aa]x [ma - ~a m
a
+ ~J.
a
D22 The Wavelet Inversion Forrnula 187
i
o b
Figure D2.2. Time-frequency tiling in wavelet analysis
Let us assurne that fii > O. The frequency window is then centered at v = fii / a
and has width 2a / a; therefore,
center frequency fii
Q- --
- bandwidth - 2a
is independent of the frequency variable a. This is called constant-Q jiltering.
Calling v = fii / a the center frequency, we see that the area of the box is constant
a,
and equal to 4a but that its shape changes with the frequency v = 1/a analyzed.
For high frequencies it has a large time dimension, and for small frequencies it has
a small time dimension (see Fig. D2.2). The interest of such features is discussed
in the introduction to this chapter.
We shall see in the next subsection that in order to guarantee perfect reconstruc-
tion of the signal from its wavelet transform, the center of :(f must be zero. Also,
the center of the wavelet itself can be taken equal to zero without loss of generality.
The Fourier transform of a wavelet has bumps at positive and negative frequencies
(see Example D2.3, the Mexican hat). The centers of the bumps then play the role
of the center of the wavelet in the first part of the above discussion (where fii was
assumed to be nonzero ).
D22 Wavelet Inversion Formula

Under mild conditions, there exists a wavelet inversion formula similar to the WFT
inversion formula.
THEOREM D2.1. Let 1jJ : ~ f-+ ~ be a mother wavelet such that 1jJ E LI n L 2,
L 11jJ(t)1 2 dt = 1, (20)
188 D2. The Wavelet Transfonn
and
( 1~(v)12 dv = K < 00.

(21)
JR. lvi
Define 1/1a,b E L 2 n LI by (16). To the function J E L 2 is associated its wavelet
transJonn C j : IR\{O} X IR -+ C, defined by (17). Then
~ { { IC j(a, b)1 2 da ~b = ( IJ(t)1 2 dt (22)

K JR.\/OjJR. a JR.
and
J(t) = K1 { (C j(a, b)1/Ia,b(t) da ~b , (23)

JR.\/OjJR. a
in the sense that Je -+ J in L 2 as e -+ 0+, where
Je(t) = -1 ~~ dadb
C j(a, b)1/Ia,b(t) - 2- .
K R.x/lal:::eJ a
Proof' First observe that 1/Ia,b has the same energy as 1/1, equal to unity.
From (18) and (19),
C j(a, b) = lal 1j2 f !(v)~*(av)e-2i7rvb dv
The function inside the curly brackets is in L I because it is the product of two
L 2-functions, and it is in L 2 as it is the product of an L 2-function with a bounded
function (~ is bounded because 1/1 E LI). By the Plancherel-Parseval identity,
L'C j(a, b)1 2 db = laILIF;I{!(v)~*(av)}12 (b) db
= laILI!(v)121~(av)12 dv,
and therefore,
11R. R.
dbda
IC j(a, b)1 2 - 2- =
a
11
R. R.
A ~
IJ(v)1 2 11/I(av)1 2 da
dv-
la I
= LI!(V)1 2 K dv =K LIJ(t)1 2 dt.

D22 The Wavelet Inversion Formula 189
This proves (22). To prove (23), first compute
l(a) = L C j(a, b)1/!a,b(t) db
= lal l / 2 L F;I{!(v)V;*(av)}(b)1/!a,b(t)db
= lal l / 2 L !(v)V;*(av)F;I{1/!a,b(t)}(v)dv,
where we have used the Planchere1-Parseval identity. Now,

F;I{1/!a,b(t)}(V) = lal l / 2 V;(av)e 2i Jl'vt,
and therefore,
and, for c > 0,
J(t) = ~ ( l(a) d~
K J11al?} a
= ~ { ( { !(v)IV;(av)1 2 e2i Jl'vt dV) da . (25)

K J11al?} JR lai
With a view to applying Fubini's theorem we must check that the function
~ ~ 2 1
(a, v) -+ IJ(v)II1/!(av)1 ~
is integrable in the domain lR. x {la I ~ c}. We have
1 = 11
R lal?
~ ~
IJ(v)II1/!(av)1 -
lai
1
da dv
= { 1!(v)1 ( ( 1V;(av)1 2 da) dv

JR J1al? lai
= { 1!(v)1 (
JR
(
J1xl?lvl
1V;(x)1 2
Ix I
dx) dv
= 1+ + {
-I
1
J1VI?1
= 11 +h
But
11 = 1+ -I
1
1!(v)1 ( (
J1xl?IVI
1V;(x)1 2
lxi
dX) dv
:s K 1 -I
+1
1!(v)1 dv < 00
because! E L2e~) and, in particular, ! E L 2 e[ - 1, + 1]). Also,
lz = 1 Ivl:::l
l!ev)1 (1 ~ 2
11/Iex)1 dX) dv
Ixl:::slvl Ix I
But
l11/let)1 2 dt = 1,
and, therefore, using Schwarz' inequality,
lz < ! 1
- e Ivl:::l
l!ev)1 dv
lvi
< -1 (1 IJev)1 2 dv )1/2

A (1 -dV)I/2
- c Ivl:::l Ivl:::l v 2
We can therefore change the order of integration in (25):
~ 2
KJset) = { !ev)e 2i ;rvt ( ( 11/Ieav)1 da) dv
JlR Jlal:::s lai
= l !ev)e 2i ;rvt gsev)dv = F-1{!gs}(t).

Since we want to prove that Js -+ J in L 2 , we must evaluate the L 2 -norm of
J- Js:
K 2 11J - Jsll 2 = K 2 I1F- 1{j - Js}1I 2 = lI!eK - gs)1I 2
= { IK - gsev)1 2 1!ev)1 2 dv =
JlR Ivl:o:e-
+
Ivl>e-
1 1
= A + B.
1/ 2 1/ 2
On {lvi :s c- I / 2 },
gsev) = { l1freav)1 2 da = { l1frex)1 2 dx
Jlal:::s lai Jlxl:::SIVI lxi
D22 The Wave1et Inversion Formu1a 191
where
Ke :s K and lim K e
e-+O+
= K.
Therefore,
Also,
since j E L 2 . We have therefore proved that
IIf - fell --+ 0 as 8 --+ 0+.
The proof is lengthy because we have only required f to be in L 2

THEOREM D2.2. If f E L 2 n LI and j E LI, the inversionformula (23) is true
for almost all t.
11
Proof" We start from (24):
1 Ja
[Ca) -da
a2
=
Ja Ja
~ ~
f(v)Il{!(av)1 2 2 dvda
e "rvt - -
a2
= [ j(v)e2irrvt ( [ 1~(av)12 da) dv

lJa lJa lai
= K L g(v)e2irrvt dv.
This quantity is almost everywhere equal to K f (t) by the Fourier inversion formula
in LI.
Recall that if f is continuous the equality in the Fourier inversion formula holds
for all t and, therefore, (23) is then true for all t E R
Oscillation Condition
Since ~ is continuous (l{! E LI), the assumption (21) implies that ~(o) = 0, that
is, to say,
L l{!(t) dt = o. (26)
In most situations it suffices to verify (26), and then (21) follows. For example, if
1/I(t) and t1/l(t) are integrable, then Vi is Cl; therefore, if Vi(o) = 0, the quantity
IVi(v)1 2 /lvl is integrable in a neighborhood of zero and therefore on ~, since at
infinity there is no problem, due to the hypothesis 1/1 E L 2 (which implies that
Vi E L2 ).
EXAMPLE 02.1 (Modet's pseudo-wavelet). Morlet used the mother wavelet
1/I(t) = ye- t2 / 2 cos(5t),
where y is a normalization factor that makes the energy equal to unity. The
theoretical problem here is that
Vi(O) = L 1/I(t) dt > O.
However, the numerical results obtained with this wavelet were satisfactory
because the value of Vi(O) is in fact very smalI.
EXAMPLE 02.2 (Haar wavelet). The Haar wavelet
ifO ~ t < ~,
if ~ ~ t < 1,
otherwise,
satisfies the conditionsfor the reconstructionformula (23) to be valid. Here
~ 1 - cos(7l' v)
1/I(v) = le. -;rrv .
7l'V
EXAMPLE 02.3. In practice, a mother wavelet 1/1 should be weil localized in

addition to Vi, and it should also be oscillating (so as to guarantee at least that
IR 1/I(t) dt = 0). Derivatives ofthe Gaussian pulse are goodfor this purpose. For
example, the second-order derivative, called the Mexican hat (see Fig. D2.3),
1/I(t) "" (1 - t 2 )e- t2 / 2
with the Fourier transform
~ 2 2 2 2
1/I(V) "" v e- rr v
is interesting because both 1/1 and Vi are rapidly decreasing C OO -functions.

We shall now give a pictorial example. Fig. D2.4 shows a simple signal and
Fig. D2.5 shows its wavelet transform. The mother wavelet used is not given,
since it is irrelevant to this qualitative illustration. In the latter figure, the time
axis is horizontal, and the time axis vertical, the bottom part corresponding to
high frequencies. We observe the good time localization and the fact that sharp
discontinuities are represented in the bottom part.
D22 The Wavelet Inversion Formula 193
~ilr-----~----II
(a)
-10 -8 -6 -4 -2 0 2 4 6 8 10
JJ(/:::
(b)
1
-10 -8 -6 -4 -2 0 2 4 6 8 10
(c)
j:~
-10 -8 -6 -4 -2 0 2 4 6 8 10
Figure D2.3. The Mexican hat

194 D2. The Wavelet Transform
0.8
0.6
0.4
0,2
01 - - - - - - - ' L _ _ _ _ _- - - '
~.2 ~ __ ~ __ ~ __ ___
~ __ ~ __ __
~ __ ~ ~ ~~
o 500 1000 1500 2000 2500 3000 3500 4000
Figure D2.4. Spike + sinusoid
Figure D2.5. The wavelet transform of the signal in Fig. D2.4

D3
Wavelet Orthonormal Expansions
D31 Mother Wavelet

The wavelet analysis of Chapter D2 is continuous, in that the original function of
L 2 is reconstructed as an integral, not as a sumo One would rather store the original
function not as a function of two arguments, but as the doubly indexed sequence of
coefficients of a decomposition along an orthonormal base of L 2 Multiresolution
analysis is one particular way of obtaining such orthonormal bases.
In the remainder, we adopt a slightly different definition of the Fourier transform.
The FT j(w) ofthe signal f(t) is now defined by
j(w) = L f(t)e- iwt dt.
The inversion formula, when it holds true, takes the form
f(t) = _1 [j(w)e iwt dw, a.e.,

2n JR
and the Plancherel-Parseval identity, when it holds true, reads
[ f(t)g(t)* dt = _1 [j(w)g(w)* dw.

JR 2n JR
Also, the necessary and sufficient condition for {cp( . - n) }nEZ to be an orthonormal
sequence of L~(IR) (Theorem C4.5) now reads
L 1<7J(w + 2kn)1
kEZ
2 = 1, a.e.

196 D3. Wavelet Orthonormal Expansions
One reason for abandoning the definition in terms of the frequency v is that
the topic of MRA involves a mixture of analog signals and of digital filtering,
and digital signal processing is traditionally-as in the present text---dealt with in
terms of the pulsation w.
Scaling Function and MRA

DEFINITION D3.1. A multiresolution analysis (MRA) of L 2 = L~(IR) consists of a
function cp E L 2 together with afamily {Vj}jez of Hilbert subspaces of L 2 such
that
(a) {cp(. - n)}neZ is an orthonormal basis ofVo,
(b)for all j E Z, Vj ~ Vj+1 (the Vj 's are said to be nested; see Fig. 4.7),
(c) fE Vo {::=} f(2 j .) E Vj ,
(d) njez Vj = 0 and c10S(Ujez Vj ) = L 2 .
The function cp is called the scaling function of the MRA. The index j represents
a resolution level: The projection Pj f of a function f E L 2 on Vj is interpreted
as the observation of this function at the resolution level j.
Usually, the projection on Vo is the function itself, in which case the projections
at all levels j 2: 0 are identical. The projection at level 0 is, in applications, the
one offered by the recording device.
Observe that, since the mapping f -+ ,J2 f(2 ) is an isometry from Vo onto VI
and since (cp(. - n) }neZ is an orthonormal basis of Vo, the set (,J2 cp(2 . - n) }neZ
is an orthonormal basis of VI. More generally, {CPj,n}nez is an orthonormal basis
of Vj , where
(27)
With respect to (d), recall that (Exercise C1.6)

nV
jeZ
j = 0 {::=} lim P_j/ = 0
J .... +OO
for all fE L 2 (28)
Figure D3.1. Nested subspaces

D31 Mother Wavelet 197
](1/)
j(.) E Va
-7f o +7f 1/
}(I/)
~.
o +27f 1/
j(.) E Vi
](1/)
j(.) E V- 1
1/
Shannon multiresolution
Figure D3.2. Nesting in the Shannon MRA
and
closU Vj = L 2 {:::::::} .lim Pj / = / for all / E L2 (29)

jE71 J-->+OO
EXERCISE D3.1 (Shannon scaling function). For alt j E Z, define
Vj = {f E L 2 : supp (1) c [- 2 j 1T, + 2 j 1T]}.

Define the function rp by its Fourier trans/orm
$(w) = ll[-rr,+rrj(w)
(see Fig. D3.2). Verify that {Vj }jE71 is a multiresolution analysis 0/ L 2 associated
with the scaling function rp.
EXERCISE D3.2 (Haar scaling function). For alt j E Z, define
Vj = {f E L2 : / is a.e. constant on the intervals (kTj, (k + l)Tj]).

Define
rp(t) = ll(o,l](t)
(see Fig. D3.3). Verify that {Vj }j E71 is a multiresolution analysis 0/ L 2 associated
with the scaling function rp.
We shall see later that some regularity of the scaling function is desirable.
,...........J--I
L...-
I ~
j(.) E Vo
16 j(.) E V-I
2 I
Haar multiresolution
Figure D3.3. Nesting in the Haar MRA
DEFINITION D3.2. The function ({J is said to belong to Sr for some rEN if ({J is r
times continuously differentiable with rapidly decreasing derivatives, in the sense
that
I({J(k)(t)1 :s C kp , for k = 0, 1, ... , r, and all p E N. (30)

(1 + Itl)P
This is a multiresolution analysis for wh ich the scaling function ({J E sr is called
r-smooth.
The Haar and Shannon scaling functions are not in Sr (for any rEN).
Conditions (a), (b), (e), and (d) of Definition 4.3 are not independent. In fact,
the first part of (d) is always true under conditions (a), (b), (e), whereas the latter
conditions are almost sufficient for the second part of (d). The result below makes
this statement precise.
THEOREM D3.1. Suppose that eonditions (a), (b), and (e) of the definition of an
MRA are satisfied. Then njEZ Vj = 0. Moreover, ifep is eontinuous at the origin,
then
lep(O) I =1= 0 {::=> closU Vj = L~(l~). (31)

jEZ
In this ease, neeessarily, I?CO) I = 1.
Proo!" The first statement will be proven in the more general Theorem D4.1.
We now prove the second statement. (The proof is technical and can be skipped
in a first reading.) Denote by Ta the translation operator defined by Taf(x) =
f (x - a). We shall first show that the Hilbert space W = clos ( Uj EZ Vj ) is
invariant under translations.
D31 Mother Wave1et 199
We begin with dyadic translations (a = m2-c, where e, mEZ). Let f E W.

Therefore, for any given c: > 0, there exist jo E Z and h E Vjo such that 11 f - h 112 :s
c:. For all j ::: jo, h is also in Vj . In particular,
h = L c{q;(2 j . -k),
kE'lL
and the function
Tm2 -(h = L c{q;(2 j . -2 j - Cm - k)

kE'lL
is therefore in Vj if j ::: e (because 2 j - Cm is then in Z). Therefore, for all j ::: e,
and Tm 2-( h E Vj This means that Tm 2-( f is c:-c1ose to Vj for all j ::: e. From this
and the arbitrariness of c:, we deduce that Tm 2-( f E W.
Let now a E lR be arbitrary. Given c: > 0, there exists 8 such that, for all
c E (a - 8, a + 8),
(use Theorem C3.1 stating that the map a 1--+ Taf is uniformly continuous). In
particular, we can find a dyadic number c for which the above inequality is satisfied.
Since Tc! E W and c: is arbitrary, we deduce that Ta f E W.
We now proceed to the proof of (31). We assurne that ~is continuous at 0 and
that I~(O)I -=1= O. Therefore, ~(w) -=1= 0 on (-c, +c), for some c > O. Consider any
function g orthogonal to W, that is, orthogonal to all f E W. Since W is invariant
under translations, for all x E lR and for all f E W,
0= L f(x + t)g(t)* dt.

0= L eiwx j(w)g(w)* dw,
for all x ER The function jg* E L~(lR), and therefore, by the Fourier inversion
theorem in L I , j g* = 0 almost everywhere.
In particular, with f(t) = 2 j q;(2 j t) (indeed, such f E Vj C W), we obtain
~(Tjw)g(w)* = 0, a.e.
Since ~(2-jw) -=1= 0 if w E (-2 j c, +2 j c), we have that g(w) = 0 if w E

(-2 j c, +2 j c). This being true for all j E Z, we have that g, and therefore g,
is almost everywhere null. We have thus proven that the only function in L~(lR)
orthogonal to W is the null function. Therefore, W is exactly L~(lR).
Assurne now that W = L~(IR). Let 1 be the function with the Fr j = 1[-1.+1].
In particular,
By (29),
lim 111 - Pj/1I2

jt+ oo
= 0,
and therefore, by continuity of the norm,
. 2 2 1
.hm IIPj/1I 2 = 1I/\1z =-,
Jt+ oo rr
that is,
Itim 11
J +00
LU, qJj.k)qJj,kiI~ = ~.rr
ke'Z
We have by the Plancherel-Paseval identity for orthonormal bases (Theorem C2.2),
11 LU, qJj,k)qJj,kll~ = LI (l(t)qJj,-k(t)* dtl2

ke'Z ke'Z JIR
The last sum equals, by another Plancherel-Paseval identity,
For large enough j, [-2- j, +2- j] c [-rr, +rr], and therefore the last displayed
expression is 2 j times the sum of the squared absolute values of the Fourier
coefficients of 1[-2-i,+2-ijr. Therefore, by the appropriate Plancherel-Parseval
identity,
L 1_1 j+2-i e-ikwq;(w)* dwl2 = _1 j+2-i \q;(w) \2 dw = ~.

ke'Z 2rr -2-j 2rr -2-j rr
Therefore,
By continuity of q; at 0, this limit is also ~ \qJ(0)\2. Therefore, \qJ(0) \ = 1.

D31 Mother Wavelet 201
Wavelet Expansion
We shall suppose in the sequel that the scaling functions cp satisfy 1qJ'(0) I > 0, and
then take (without further loss of generality)
qJ'(0) = 1. (32)
DEF1NITION D3.3. A wavelet orthonormal basis 01 L 2 = L~(lR) is an orthonormal
basis olthelorm {1/Ij,n}j,nez, where
(33)
The function 1/1 is then called the mother wavelet of the wavelet basis. The
expansion
1 = LL(f, 1/Ij,k)1/Ij,k (34)
jeZ keZ
is called the wavelet expansion of I.
A wavelet orthonormal basis can be obtained from an MRA in the following
way. Let Wj be the orthogonal complement of Vj in Vj+l:
Vj+l = Vj EB Wj. (35)
From property (d) of the definition of MRA,
L2 = EBWj . (36)
jeZ
Also, from (e),

1E Wo ~ 1(2 j .) E Wj.
Therefore, ifwecan exhibit an orthonormal basis of Wo ofthe form {1/1( . -n)}nez,
then {1/1 j.n }neZ is an orthonormal basis of Wj. Therefore, by (36) {1/1 j,n b,nez is an
orthonormal basis of L 2
Recall that Pj is the projection on Vj . We have
Pj+I! = Pj/ + L (f, 1/Ij.k)1/Ij,k for all 1E L2. (37)

keZ
Pj 1 is the result of observing 1 at the resolution level j: As j increases, the
resolution increases (note that Vj C Vj+l); the difference
Pj+I! - Pj/ = L(f, 1/Ij,k)1/Ij.k

keZ
is the additional detail required to pass from the resolution level j to the higher
resolution level j + 1.
A first issue is: How to compute the mother wavelet 1/1 from the scaling function
cp? The next question is: How to obtain a scaling function cp? Finally, one would
like to obtain a mother wavelet with "good" numerical properties, that is fast
convergence of the wavelet expansion (34).
D32 Mother Wavelet in the Fourier Domain

We address the first issue, that of explicitly finding a mother wavelet given a scaling
function.
EXAMPLE D3.1. We seek to obtain the mother wavelet corresponding to the Haar
scaling function. Recall that VI is the Hilbert space of L 2 -functions that are con-
stant almost everywhere on the intervals (nI2, (n + 1)/2]. The mother wavelet 1/1
must be ofthis type since Wo C VI. Thefunction ofnorm 1, with support (0,1],
!],
1/I(t) = 1
+1
-1
ift
ift
E
E
(0,
(!' 1],
does it. To see this, it suffices to verify that any f E VI with support (0, 1] is a
linear combination of cp and 1/1 and that cp and 1/1 are orthogonal. Orthogonality
is obvious. Any fE VI such that supp(f) E (0, 1] is oftheform
!],
f(t) = l
a

ift
ift
E
E
(0,
(!' 1].
and we therefore have the decomposition (see Fig. D3.4)
a+ a-
f = - 2 - cp + -2-1/1
The function 1/1 is called the Haar mother wavelet.
'Cl
/
~x
5 1
4" ----, f(t)
l--- L --
1
'ljJ(t)
~
1 +1
4"
I
01 1 1
~
lx
2
21
1
1
1
1 1
-1 --~
Figure D3.4. Haar decomposition

We now give a simple example of wavelet analysis to illustrate the notions

of projection and detail. Fig. D3.5a gives (from top to bottom) a signal and its
successive projections on the nested subspaces at decreasing resolution levels,
whereas Fig. D3.5b gives (from top to bottom) the successive projections on the
detail subspaces at decreasing resolution levels. In particular, the second function
in Fig. D3.5b is the difference between the first and second functions in Fig. D3.5a.
The general case will now be treated. We shall obtain a necessary condition for
the scaling function ({J to be a scaling function. First, since {({J(' - n)}nEZ is an
orthonormal system, we have, by Theorem C4.5,
L I$(w + 2kn)1 2 = 1, a.e. (38)
kEZ
The scaling function ({J E Vo and therefore, ({J E VI. Requirements (a) and (e) in
the definition of an MRA imply that {({JI,n}nEZ is a Hilbert basis of VI. and we
therefore have the expansion ({J = LnEZ hn({JI,n, that is,
({J = v'2 L h n({J(2 . -n), (39)
nEZ
where
(40)
In the Fourier domain (39) reads
~
((J(w) = 1M ~hne-
'""' inW~(W)
'i({J "2 '
-v2 nEZ
that is,
(41)
where mo(w) is the 2n-periodic function defined by
mo(w) = 1M ~hne
'""' -inw . (42)
-v2 nEZ
It is called the low-pass filter MRA, because mo(O) = 1 (recall the running
assumption that $(0) = 1; see (32)). Substituting identity (41) in (38) gives
300~
200
100
O~ __ ~ ____- L____ ~ ____L -_ _ ~ _ _ _ _- L____~____L-~~____~
50 100 150 200 250 300 350 400 450 500
::~' '~' '''A~' AM' '~

100~~-,
__ -L____
O~ - J_ _ _ _~_ _ _ _- L_ _ _ _
,~-, LJJ
- L_ _ _ _~_ _ _ _~_ _ _ _L -_ _ _ _L -_ _~~
50 100 150 200 250 300 350 400 450 500
300~
200
100
O~ __- J_ _ _ _ ~ _ _ _ _- L_ _ _ _~_ _ _ _- L_ _ _ _~_ _ _ _~_ _ _ _~_ _ _ _~_ _~~
50 100 150 200 250 300 350 400 450 500
300~
200
100
O~ __ ~ ____- L____ ~ ____L -__ ~ ____- L____ ~ ____L -_ _ ~ ____ ~
50 100 150 200 250 300 350 400 450 500
300~
200
100
O~ __- J_ _ _ _ ~ _ _ _ _- L_ _ _ _- L_ _ _ _- L_ _ _ _~_ _ _ _~_ _ _ _L -_ _ _ _~_ _~~
50 100 150 200 250 300 350 400 450 500
(a)
50 100 150 200 250 300 350 400 450 500
,oo~
-10:
50 100 150 200 250 300 350 400 450 500
,oo~
-10:
50 100 150 200 250 300 350 400 450 500
100~
-10:
50 100 150 200 250 300 350 400 450 500
(b)
Figure D3.5. Haar wavelet analysis

Therefore,
or, equivalently,
(43)
The filter with frequeney response eiwmo(w + n)* is ealled the high-pass filter
of the MRA. Eqn. (43) shows that the high-pass and the low-pass filters altogether
extraet the whole energy eontained in the band [-n, +n].
We now eharaeterize the spaees V-I and Vo. This will be a preliminary to the
eharaeterization of W_I, the orthogonal eomplement of V-I in Vo. Onee this is
done, we shall obtain the eharaeterization of Wo and then the mother wavelet
itself.
LEMMA D3.1. f E V-I if and only if it has an FT of the form
f(w) = m(2w)mo(w)qJ'(w), (44)
for some 2n -periodic function m E L~ ([ -n, +n]).
Proof" Indeed, any f E V-I ean be deeomposed along the orthonormal basis
{CP-I,n}nEZ, that is,
1 1
f(t) = M I>kCP( -t - k), (45)
",2 kEZ 2
where {Cn}nEZ E l~. Taking the FT, we obtain
f(w) = hI>ke-i2kWqJ'(2w).
kEZ
This is (44) (using (41) and defining m(w) = v'2 LkEZ Cke-ikw).
f
Conversely, eonsider a funetion defined by (44), where m is a 2n-periodie
funetion in L~([-n, +n]). We show that fis in L~(lR.). First, observe that it is
of the form h(w)qJ'(w), where h is a 2n-periodic funetion in L~([ -n, +n]) (sinee
m E L~([ -n, +n]) and sinee mo is bounded in view ofEq. (43. Now
[ Ih(w)qJ'(w)1 2 dw =L j+Jr Ih(w)1 21qJ'(w + 2k1T)1 2 dw

JR kEZ -Jr
= j +Jr 2
-Jr Ih(w)1 dw < +00.
206 D3. Wavelet Orthononnal Expansions
f
This proves that E L~(lR). Since f
E L~(lR), it is the Fr of a function I E
LUlR). Tracing back the computations in the first part of the proof, we obtain that
(45) holds true, with {cn}nEZ E e~, which implies that I E V-j.
LEMMA D3.2. I E Vo if and only if it has an FT 01 the lonn
f(w) = d(w)$(w), (46)
lor some 2rr -periodie function d E L~([ -rr, +rr]).
Proof Indeed, let I E Vo. It can be decomposed along the orthonormal basis
{fPo,n }nEZ, that is,
I(t) = L dkfP(t - k), (47)
kEZ
where {dn}nEZ E e~. Taking the Fr, we obtain
f(w) = L dkeikwqJ(w) = d(w)$(w),

kEZ
where d E L~([-rr, +rr]). Arguing as in the proof ofLemma D3.l, we can show
f
that any function of the form (46) is the Fr of a function I E Vo.
Consider the mapping U : Vo !--+ L~([ -rr, +rr]) defined by UI = d (where d

is defined by (46)). Clearly, this mapping is linear, and
IIUfllh([-rr,+rr]) = IIdlli~([_rr,+rr]) = 2rr L Id l
kEZ
k 2 = 2rrll/ll~. (48)
By the polarization identity, for all I, g E Vo,

1
(j, g) L~(R) = 2rr (UI, U g) L~([-rr,+rr])' (49)
We are now ready to state and prove the Fourier characterization of Wo, the
Hilbert space of details at level O.
THEOREM D3.2. The function I E Wo if and only if
f(w) = ei~mo(~ + rr)* V(w)$(~) , (50)
lor some 2rr -periodie function v in L~( -rr, +rr).

Proof Observe that it is equivalent to prove that the function lEW_I if and
only if
(51)
for some 2rr-periodic function v in LU -rr, +rr).
Let lEW-I, that is, I E Vo and 1..1 V-I. Being in Vo, I has a representation
oftype (46). By (49) and the characterization (44) of V-I, the orthogonality of I
and V-I is equivalent to
(2rr
0= 10 d(w)m(2w)*mo(w)* dw,
for all2Jr-periodie funetion m E L~([ -Jr, +Jr]). This ean also be written
0= Ln: m(2w)* [d(w)mo(w)* + d(w + Jr)mo(w + Jr)*] dw.

The funetion in the square braekets is therefore orthogonal to all g E L~([O, +Jr]),
and therefore,
d(w)mo(w)* + d(w + Jr)mo(w + Jr)* = 0 (52)
almost everywhere in [0, +Jr] (and therefore almost everywhere in [-Jr, +Jr)).
Define
mo(w) = (mo(w), mo(w + Jr.
In view of the identity (43), this is a unitary veetor in C 2 eonsidered as a 2-

dimensional veetor spaee (with sealar field C). The veetor
m'o(w) = (mo(w + Jr)*, -mo(w)*)
is unitary and orthogonal to mo(w). Defining
do(w) = (d(w), d(w + Jr,
we have, by (52), do.lmo(w). Therefore,

do = (w)m'o(w),
where
(w) = (do, m'o(w) = d(w)mo(w + Jr) - d(w + Jr)mo(w).
In partieular,
(w + Jr) = -(w + 2Jr), a.e.
or, equivalently,
(w) = -(w + Jr), a.e.,
whieh implies in partieular that is 2Jr-periodie. It is also in L~([ -Jr, +Jr)).
Indeed,
j(w) = d(w)qJ(w) = (w)mo(w + Jr)*qJ(w),
and therefore,
1n: I(w)1 2 dw = 111: I(w)12(lmo(w)12 + Imo(w + Jr)1 2) dw

208 D3. Wavelet Orthononnal Expansions
where the last equality follows from (48). Defining
gives the representation (51).
Conversely, suppose that

j(w) = eiwmrf,w + n)*v(2w)qy(w),
for some 2n-periodic function v in L~([ -n, +n]). That is,
j(w) = d(w)ifI..w),
where
d(w) = eiwmriw + n)*.
Since Imo(w)1 ::: 1, this implies that d(w) E L~([-n, +n]). Therefore, I E Vo
(Lemma D3.2). Also, from the expression of d(w), do(w) = eiwv(w)m'o(w), and
therefore
do..lmo(w),
that is, d(w)mo(w)* + d(w + n)mo(w + n)* = O. By Lemma D3.1 and Eq. (48),
this implies that 1.1 V-I. But also I E Vo. Therefore, I E Wo.
We are now ready for the main result of this subsection, the Fourier characteri-
zation of the mother wavelet in terms of the scaling function and of the high-pass
filter.
r
THEOREM D3.3. The junction 'tjJ is a mother wavelet if and only if
:V;(w) = eiW/2mo(~ + n V(w)qy(~) , (53)
lor some 2n-periodicjunction v such that Iv(w)1 = 1 almost everywhere.

Proof" Since:V; is of the form (50) with lvi = 1, it is in L~(IR) (by the now
standard argument) and, therefore, it is the Fr of a function 'tjJ E L~(IR), which is
in Wo by Lemma D3.2. By Lemma D3.2 again, any function g E Wo has an Fr of
r
the form
g(w) = eiW/2mo(~ + n s(w)qy(~) ,

for some 2n-periodic function s in L~([ -n, +n]. In particular, since v-I = v*,
g(w) = s(w)v(w)*:V;(w).
Since sv* E L~([ -n, +n],
s(w)v(w)* = LCke-inw,
nEZ
for some sequence (cn}nEZ E l~, and therefore,
g(t) = I:>k1fr(t - n).

nEZ
Therefore, the translates of 1fr generate Wo. The system {1fr(. - n) }nEZ is
orthonormal because
L 11fr(w + 2br)1
kEZ
2 = 1, a.e.,
as can be checked by the usual routine.
Conversely, let 1fr be an orthonormal wavelet. Being in Wo, it is of the form (50).
By the usual calculations, we find that
L 11fr(w + 2krr)1
kEZ
2 = Iv(w)1 2 ,
and therefore, by the orthonormality condition, Iv(w)1 2 = 1.

In summary, the mother wavelet is of the form
(54)
where ml(w) = eiWmo(w+rr)*v(2w) is a high-pass filter (Iml(rr)1 = 1). Werecall

that the scaling function cp and the low-pass filter are related by
q;(w) = mo (~) q;(~) . (55)
We also have the identity

(56)
These three relations fuHy describe the MRA: (55) is called the dilation equation
and teHs us that Vj - 1 is obtained by low-pass filtering Vj ; (54) teHs us that Wj - 1
is obtained by high-pass filtering Vj; and (56) guarantees that there is no loss of
energy.
The choice v == 1 leads to
1fr(w) = eiW/2mo(~ + rr)* q;(~),

or, equivalently, up to the sign,
1fr(t) =.J2 L(-It-1h"'-n_lcp(2t - n), (57)
where h n is defined by (40).

EXAMPLE D3.1 (The Haar Wavelet). We shall obtain the Haar wavelet by the
general method just described. Recall that the scaling function is cp(t) = I[o,1](t)
(a) (b)
- r-
0.5 0.5
0 o
-0.5 -0.5
-1 -1 ~
-3 -2 -1 0 2 3 -3 -2 -1 o 2 3
(c) (d)
1.5 1.5 ,-----------~--~--~--____,
0.5 0.5
OL-------~--~--~--~
o 0.1 0.2 0.3 0.4 0.5 0.1 0.2 0.3 0.4 0.5
Figure D3.6. Haar scaling function and the corresponding wavelet (left: scaling
function; right: wavelet; top: time domain; bottom: frequency domain)
and, therefore,
hn = v'2 L q;(x )q;(2x - n)* dx
forn = 0, I,
otherwise,
and, using (57),
1jI(t) = q;(2t) - q;(2t - I).
Thus, we recover the Haar wavelet (Fig. D3.6).
EXAMPLE D3.2 (Shannon wavelet). Here
fi(w) = I[-n,+nj(w),
and, therefore,
sin(nt)
q;(t) =
nt
D33 Mallat's A1gorithm 211
Wefirst choose mo such that (41) holds, i.e.,

$(2w) = mo(w)$(w).
Therelore, necessarily,
mo(w) = $(2w) on [-n, +n],
that is,
By periodicity,
mo(w) =L $(2w + 2kn).
kEZ
Ourchoice 011/1 is as in (53), with v(w) = _ie- iw :

V!(2w) = - e-iwmo(w + n)*$(w)
= - e- iw (L kEZ
$(2w + 2kn + 1) $(w)
=- e- iW ($(2w + n) + $(2w - n
= _e- iw (1 [_lI"._lj](w) + 1[+lj,+lI"](w).

This gives the Shannon wavelet (Fig. D3.7)
1/I(t) = cos(ln(t _ ! sin( !n(t -

2
!
2
2 2 ~n(t _ ~)
D33 Mallat's Algorithm

Mallat's algorithm6 is a fast algorithm for obtaining from the projection at a given
level the wavelet's coefficients at coarser levels of resolution.
Let 1 be a function in L~(IR).lts projection on Vj , the resolution space at level
j, is
Pjl = LCj,n'Pj,n,
nEZ
where
(58)
6Mal1at, S. A theory of multireso1ution signal decomposition: The wave1et repre-

sentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 1989,
674-693.
(a) (b)
0.5 0.5
0 0
-0.5 -0.5
-1 -1
-15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15
(c) (d)
1.5 1.5
0.5 0.5
0.1 0.2 0.3 0.4 0.5 0.1 0.2 0.3 0.4 0.5
Figure D3. 7 Shannon sealing funetion and the eorresponding wavelet (left: sealing
funetion; right: wavelet; top: time domain; bottom: frequeney domain)
Its projeetion on Wj , the spaee of details at level j, is
Dj/ = Ldj,nVrj,n,
nEZ
where
(59)
and wehave
(60)
Denote by Cj and d j the sequenees {Cj,n}nEZ and {dj,n}nEZ, respeetively. The pur-
pose of Mallat's algorithms is to decompose the funetion f, that is, to pass from
CM to dM-I, dM-I, ... , do, Co, and to reconstruct that is to pass, from co, do,d l ,
" " d M to CM.
The sequenee d M - lo d M - lo ... , do, Co is the wavelet encoding ofthe wavelet data
CM' We shall explain the interest of this eneoding onee we have derived Mallat's
algorithm.
D33 Mallat's Algorithm 213
Since the function cp(t/2) is in V-I, and V-I C Vo, and since {cp(. - n)}nEZ is a
Hilbert basis of Vo, we have the decomposition
1 1
2CP(2 t ) = L ancp(t + n),
nEZ
where
a n =- 11m cp(-t)cp(t+n)dt.
1
2 lR 2
Therefore,
L::!. 1 1 .
=2 2 -cp( -(2' t - 2n))
2 2
= 2-9- LakCP(2jt - 2n + k),

kEZ
that is,
CPj-l.n = V2L a kCPj,2n-k. (61)

kEZ
Sirnilarly, since the function 1/1'(t/2) is in W_I, and W_I C Vo, and since {cp(. -
n)}nEZ is a Hilbert basis of Vo, we have the decomposition
where
n = ~ [ 1/1'(~t)cp(t+n)dt.
21lR 2
Therefore, it follows by computations sirnilar to those above that
1/1'j-l,n = V2LkCPj,2n-k. (62)

kEZ
Denoting the low-pass filter by
mo(w) =L ane inw

nEZ
and the high-pass filter by

ml(w) = Lneinw,
nEZ
we have, from (55) that

qy(2w) = mo(w)cp(w),
and, from (54) that
In Theorem D3.3, we now make the particular choice of the mother wavelet
corresponding to v(w) = 1:
that is,
L n einw = L ( _l)n+la;_neinw.
nEZ nEZ
Therefore,
(63)
Substituting (61) in (58), we obtain
Cj-l.n = hLaZcj,Zn-k. (64)

kEZ
Similarly, substituting (62) in (59), we obtain
d j - 1n = hLZCj,Zn-k. (65)
kEZ
These are the basic recursions of the decomposition algorithm (see Fig. D3.8).
The recursions for the reconstruction algorithm (see Fig. D3.9) are obtained from
(60), (61), and (62). This gives
Cj,n =h L [azk-nCj-l,k + Zk-ndj-l,k]' (66)

kEZ
EXERCISE D3.1. Show that for the Haar wavelet,
Cj,Zk-l + Cj,Zk
Cj-l,k = h
(64) (64) (64) (64) (64)
@ ~CM-I ~CM-2 ~ ~ Cl ~Co
(~ (6~ (6~
dM-I dM-2
Figure D3.8. Mallat's decomposition algorithm
/ /
~CM-I~CM
Figure D3.9. Mallat's reconstruction algorithm

s
D33 Mallat's Algorithm 215
and
Cj,2k-1 - Cj.2k
dj - I k = "fi
We shall now evaluate the algorithmic complexity of the decomposition
algorithm. (Similar results hold for the reconstruction algorithm.)
For this we suppose that the low-pass and high-pass filters, mo and ml, respect-
ively, have finite impulse responses, that is, the sequences {an }nEZ and {n }nEZ have
a finite-number (say, K) of nonzero terms. Suppose that the infinite-dimensional
vector CM has in practice a finite number N of nonzero terms (say, after truncation).
Then there are approximately N /2 terms in CM-I, and therefore, in view of (64),
the passage from CM to CM-I costs approximately K N /2 multiplications; so does
the passage from CM to d M -I. For the decomposition algorithm, we therefore have
approximately
N
(K2 N N) N
+ K 4' + ... + K 2 M + K 2M = K N
multiplications. The complexity of Mallat's algorithm is therefore linear in data
size.
Note that Mallat's algorithm encodes N numbers into N numbers. Thus the
compression gain seems to be null. However, only a few terms in the sequence of
details dj,e, e E Z, j = M - 1, ... ,0, are nonnegligible, provided the MRA is
sufficiently smooth. The smoothness issue is discussed in Chapter D5.
D4
Construction of an MRA
D41 MRA from an Orthonormal System

The Fourier structure of an MRA is now elucidated, and we know how to obtain a
wavelet basis when an MRA is given. This chapter gives two methods for obtaining
anMRA.
In the previous chapter, we started from a nested family of resolution spaces

{Vj bEZ and we discovered a scaling function q; in rather simple examples. Now,
obtaining the scaling function from a given nested family of resolutions spaces can
be a difficult task in general. However, if we are interested in a wavelet basis rather
than in a given family of resolution spaces, we might as weH start from a given
function q; E L 2 with the property that {q;(. - n)}nEZ is an orthonormal system,
and define the resolution spaces in an ad hoc manner guaranteeing that q; is indeed
the corresponding scaling function.
If q; is to be the scaling function, there is but one choice for the resolution spaces,
namely,
Vj = span {q;j,n :n E Z}.
An inspired choice of q; will make the Vj 's nested as required, and this has to
be verified because there is no reason why it should be so when one starts from an
arbitrary orthonormal system {q;(. - n)}nEZ, A necessary and sufficient condition
for this is that
q;(t) = L cnq;(2t - n), (67)

nEZ

218 D4. Construction of an MRA
for some sequence {cn}nEZ E e~(Z) or, equivalently, that the dilation equation
fi(w) = mo (~) fi(~) (68)
holds for some 2JT -periodic function mo in L~( -JT, +JT). We must also verify that
conditions (d) in the definition of, an MRA are satisfied. By Theorem D3.1, it
suffices that fibe continuous at the origin and that Ifi(O) I = 1.
Meyer's Wavelet
Define cp by
. 2JT
lflwl S 3'
fi(w) = . 2JT 4JT (69)
l f - < Iwl <
3 - - 3 '

where v is a smooth function (C k or C OO ) such that
otherwise,
ifx SO,
v(x) = {~ ifx:::l
(70)
and
v(x) + v(l - x) = 1. (71)
U sing (71) it is easy to verify that
L lfi(w + 2kJT)1 2 = 1,
kEZ
and, therefore, {cp(. - n)}nEZ is an orthonormal system. We must now verify that
the Vj are nested, and for this it suffices to verify that Vo c Vj or, equivalently,
that cp E Vj. But this is true if and only if there exists a 2JT -periodic function mo
of finite power such that
fi(w) = mo (~) fi(~) .

It turns out that
mo(w) = Lfi(2w+4kJT)
kEZ
accomplishes what is required. In fact,
=~ ~(w)
cp(w)cp "2 '
D41 MRA from an Orthononnal System 219
since the supports of $(w + 2klr) and of $(w /2) do not overlap if k =1= O. But since
$(~) = 1 if w E supp(~,
we have
$(w)$(~) = $(w),
as desired. We obtain a mother wavelet by formula (53) of with v(w) = 1. This
gives
which gives
e iw / 2 sin( ~ v (2~ Iwl - 1)) .f 2JT

1 -
3
::slwl::s
4JT
3'
=
:;j;(w)
e iw / 2 cos( ~ v (2~ Iwl - 1)) if-
3
4JT
::s Iwl ::s
3'
8JT (72)
0 otherwise.
EXERCISE D4.1. Let P be a probability measure on lR with support in [-8, +8] C
[-1' +1]' and define q;(t) by its Fourier trans/orm
Check that q;(t) is indeed in LUlR) and that the system {q;(. -n ]}nEZ is orthonormal.
I
Check that the dilation equation (55) holds with
~ 4JT
mo(~) = q;(w) iflwl::S 3'
o otherwise.
Show that q;(t) so defined is the scaling function 0/ some multiresolution analysis
and that a mother wavelet is given by its Fourier trans/orm
Examine the case where P is the Dirac measure at O.

D42 MRA from a Riesz Basis

Now we do not impose orthonormality. To be specific, we have an L 2 -function W
such that
W(t) = I>nw(2t - n), (73)
nEZ
where {cn}nEZ E eUZ), and we define the resolution spaces by

Vj = span {Wj,n :n E Z}. (74)
Of course, condition (73) guarantees that these spaces are nested. In order to obtain
a Hilbert basis of Vo, we use Theorem C4.6 which says that under the "frame"
condition
o< a ~ L Iw(w + 2krr)1 2 ~< 00, (75)
kEZ
the system {cp(. - n)}nEZ is a Hilbert basis of Vo, where
~ w(w)
cp(w) = "~ Iw(w + 2k:rr)1 2 . (76)
kEZ
Here we shall also have to verify that
For this we can use the following result.

THEOREM D4.1. Let W E L~(lR.) satisfy
o< a ~ L Iw(w + 2krr)1 2 ~ < 00, (77)

kEZ
and define
Vj = span {Wj,k : k E Z}. (78)
Suppose that the Vj are nested. Then
nVj
JEZ
= 0. (79)
Proof The inequalities (77) are equivalent to the existence of A > 0, B < 00
such that
0< AllfII 2 ~ L l(f, wo,k)1 2 ~ BllfII 2 , (80)
kEZ
for all f E Vo, and therefore equivalent to
0< AllfII 2 ~ L l(f, wj,k)1 2 ~ BllfII 2 < 00,
kEZ
D42 MRA from a Riesz Basis 221
for all 1 E Vj , 1 # 0, and allj E Z.

With any 1 E njEZ Vj and s > 0, one can associate a compactly supported and
continuous function 1 E L~(~) such that 11 1 - 111 :::: s, and therefore on denoting
the orthogonal projection on Vj by Pj , we have
111 - Pjlll = IIPj(f - j)11 :::: 111 - 111 :::: s.

Therefore, for all j E Z,
By (80),
Now, with c > 0 such that

suppIC[-c,+c] and M=supll(x)l,
XEIR
we have
1(1, wjk)1 2 = ITj 1 Ixl<c

II(x)w(Tjx - k)1 dxl2
:::: TjM22cl Iw(Tjx -k)1 2 dx,

Ixl<c
where the last inequality is Schwarz's inequality. Therefore,
1(1, wjk)1 2 :::: 2cM 2 l

k +2- jC
. Iw(x)1 2 dx.
k-2- 1 c
Assuming j to be large enough for 2- j :::: 1/2 to hold, and then summing with
respect to k E Z,
where
A(c,j) = I)k-Tjc, k+Tjc].
kEZ
By the dominated convergence theorem this term tends to 0 as j -+ 00. In part-

icular, there exists a j such that 11 Pj 111 ::: 8, and therefore 11 f II ::: 28. Since 8 is
arbitrary, this implies 11 111 = 0, which proves (79).
Let us now see how this program works in a classic example:

Franklin's Wavelet
Take 7 for w the piecewise linear spline
I - Ix I if 0 ::: Ix I ::: 1,
w(x) ={ (81)
o otherwise,
and observe that (73) is verified. More explicitly,
w(x) = !w(2x + 1) + w(2x) + !w(2x - 1). (82)
The Fourier transform of w is
w(w) = ( sinz'-~(' I ) )2
Wehave
L Iw(w + 2krr)1 2 = ~ + ~ cos(w)
kE'L
= ~ ( 1 + 2 cos2 (~) )
(One way to prove this is to compute the Fourier coefficients of the left-hand side
= 1 w(t)w(t + n)* dt,

and this immediately gives the result. Note the generality of the method and its
interest when w(t) is compactly supported.) The mother wavelet is then obtained
from (76). This gives
~ ~ .f3
cp(w) = w(w) ( (W))1/2 . (83)
1 + 2cos 2 "2
If we can compute, at least numerically, the Fourier coefficient Cn in
7pranklin, P. A set of continuous orthogonal functions, Math. Ann., 100, 1928,522-529.

D43 Spline Wavelets 223
then we obtain q;(t) as

q;(t) = L::>nW(t - n).
nEZ
The corresponding low-pass filter mo(w) is
1 + 2cos (
W))1/2
(
(jJ(2w) w 2 -
= -:::::-- =
mo(w)
q;(w)
cos 2
(2) 1+2cos (w)2 ,
_
2
and this leads to an expression for the mother wavelet's Fourier transform. Again
the (numerical) evaluation of the Fourier coefficients of the function factoring (jJ( w )
yields an evaluation of 'ifr(t) in terms of the translates of q;(2x).
D43 Spline Wavelets

Franklin's wavelet is a particular case ofthe Battle-Lemarie spline wavelets, which
are now described. We first introduce a family of functions, the basis splines, or
B-splines, that play an important role in numerical analysis, in the theory of spline
approximation. The B-spline functions B n (t), n :=:: 0, are defined recursively by
Bo(t) = 1[O,1](t)
and, for n :=:: 1,
B n+1(t) = (Bo * Bn)(t) = [I

I-I
Bn(x) dx.
For n = 3, we have
ifO~t~1,
if 1 ~ t ~ 2,
if t < 0,
the rest of the function being obtained by symmetry around 2.
In the general case, Bn(t) is (for n :=:: 1) in cn- I , its support is the interval
[0, n + 1], and
L Bn(x)dx = 1.
Wehave
Bo(w) = e-i~sinc (2:)'
and, therefore, in the Fourier domain, the recurrence defining the B-splines
becomes by the convolution-multiplication rule
Bn+I(W) = e-i~sinc (2:) Bn(w).

This gives, for n 2: 0,
Bn(w)
A
= (.W
e-'Tsinc (w))n+!
2Jr (84)
We observe that Bn(O) = 1 and that in the neighborhood ofO
Bn(w) =0 (IWI~+! ) . (85)
We shall now seek a scaling function for the B-spline of order n. From the
observation
e-iIsinc (2:) = e-i~ cos (4:) x e-i~sinc (4:)'
it follows that
where
mo(w) = (e- iI cos (~)) n+! = (1 + -iw )n+!

;)
The impulse response of the low-pass filter of the MRA is therefore
.fi n+!
hk = 2n+! (k)' 0~k~ n + 1,
and the scaling equation is
1 n+!
Bn(t) = --;; L(~+!)Bn(2t - k).
2 k=O
In view of the estimate (85), the series

L IBn(w + 2kJr)1 2 (86)
kEZ
is absolutely convergent. Using (84) and the estimate

sin(w) 2 Jr
-->-, O<w< -
w - - 2'
we have, for Iwl ~ Jr,
IBn (~) 2 = /Sin~I) /2n+2 2: (~yn+2

1
Therefore, there exist positive finite constants A and B such that

A ~ L IBn(w + 2kJr)1 2 ~ B,
kEZ
and {Bn (. - k)} kEZ constitutes a Riesz basis of the Hilbert subspace that it generates.
In order to compute the scaling function of the MRA, we need the following lemma.
LEMMA D4.1. There exists a polynomial Pn of degree n such that
L IBn(w + 2br)1 2 = Pn(cos(w. (87)

kEZ
Moreover, the coefficients of this polynomial are rational and can be computed
recursively.
Proof Denote the left-hand side of (87) by Fn(w). Inserting (86) in(84) gives
Fn(w) = (sin (~) )2n+2 Gn(w),

where
1
Gn(w) =L
kEZ h- + 1Tk) 2n+2'
(J)
One verifies easily that, for n 2: 1,
Gn(w) = n(2n2+ 1) Gn(w),

"
and, therefore,
2 (. (W))2n+2 ( Fn-l(W) )"

Fn(w) = n(2n + 1) sm "2 (sin (~))2n
We introduce the new variable y = cos(w), and define the function Pn by Fn(w) =
Pn(y). Since Fo(w) = 1, we have Po(y) = 1. The recursion in the last display
becomes
2 n+ld (Pn-1(y)
Pn(y) = n(2n + 1) (1 - y) dw (1 _ y)n .
Using the differentiation rules

d d
- = (- sin(w-
dw dy
and
d2 d d2
-dw 2 = (-y)-
dy
+ (1 - l ) - ,
dy 2
we obtain, after simplification and rearrangement,
2
Pn(y) =
n(2n + 1)
(n(n + 1 + nY)Pn-l(Y) + (1 - y)(2n + (2n - I
l)y)Pn- 1
+ (1 - y)2(1 + y)P~'--l)'
Therefore, if Pn - 1 is a polynornial of degree n - 1, then Pn is a polynornial of
degree n. The conclusion follows since Po is indeed a constant.
The general method of the previous section gives for scaling function (fi = (fin
Therefore,
cp(t) = L CkBn(t - k),

kEZ
where the Ck'S are given by the power-series expansion
(Pn(z+r' )
1
r = LCkl.
!
kEZ
Observe that Ck = Ck. Also, since the function in the left-hand side is analytic,
ICkl ~ plk l ,
for some Ipl < 1. In particular, the scaling function cp(t) has exponential decay.
We now proceed to compute the mother wavelet. We have to compute the impulse
response of the low-pass filter mo(w). We have
A 1
_ cp(2w) _ Bn(2w) ( Pn(cos(w ) 2:
mo(w) - - - - - A - - ,
cp(w) Bn(w) Pn(cos(2w
that is,
. n+! !
l+e- WW ) (Pn(COS(W)2
mo(w) = (
2 Pn(cos(2w
We compute the Fourier expansion
1
( Pn(cos(w ) 2:
= '~qke
" -ikw
,
Pn(cos(2w kEZ
where
i
1
rr ( Pn(cos(w )
qk = q-k = - 2:
cos(kw)dw.
n 0 Pn(cos(2w
Therefore,
The mother wavelet is then
1/r(t) = v'2L(-ll-IL k - 1CP(2t - k).

kEZ
Putting all this together, we finally obtain

1/!(t) = L bk B n (2t - k),
kEZ
where
br = hL(-ll-ILk-1qr-k.
kEZ
D5
Smooth Multiresolution Analysis
D51 Autoreproducing Property of the Resolution

Spaces
The axiomatic framework of multiresolution analysis is Fourier analysis in L 2 ,
and the convergence of the wavelet expansion is therefore in the L 2- norm. The
smoothness properties of the scaling function and of the mother wavelet are, how-
ever, of great interest to obtain fast L 2 -convergence of the wavelet expansion, or
to obtain pointwise convergence of this expansion.
We first recall a definition.
DEFINITION D5.1. Let r ::: 0 be an integer. The junction q; : lR -+ C is said to be

in Sr iffor all n E N, all 0 .::: k .::: r, there exist finite nonnegative constants Ck,n
such that, for all x E :IR,
I (k)( )1 < Ck,n (88)

q; X - (1 + Ix Dn
DEFINITIOND5.2. Consider an MRA with scaling junction q;. Let r be an integer.
The MRA is called r-smooth if q; E Sr.
We now consider an r-smooth multiresolution analysis. We associate with it the

function q : lR X lR -+ C, called the kernel of the MRA and defined by
q(x, t) = L q;*(x - n)q;(t - n). (89)

neZ

230 D5. Smooth Multiresolution Analysis
U sing the inequality
(1 + laD(l + IbD ::: 1 + Ib - al,

wehave
Iq(x, t)1 :s L IqJ(x - n)llqJ(t - n)1
nEZ
"" CO,k+2 CO,k+2

:s nEZ
~ (l + Ix - nDk+2 (1 + It - nDk+2
<C 2 L 1 1 1
- O,k+2 nEIU'" (1 + Ix - nD 2 (l + It - nl)2 (1 + Ix - tDk '
It foUows that, for aU k E N, there exists a finite nonnegative constant Ck such

that, for all x, t E IR,
(90)
In particular, for each t E IR, the function qt : IR --+ C defined by qt(x) = q(x, t)
is in L 2 , and the development of any function f E Vo along the orthonormal basis
{qJn}nEZ = {qJ(' - n)}nEZ
takes the form
f(t) = 1 q(x, t)f(x)dx. (91)
DEFINITION D5.3. Let E be some set, and let H be a Hilbert space offunctions f :
E --+ CwiththeHermitianproduct(, }.IfthereexistsafunctionK: ExE --+ C
such thatfor each x E E, thefunction K(x, .) E H, and fex) = (K(x, .), j}, H
is called an autoreproducing Hilbert space with reproducing kernel K.
EXERCISE D5.1. Let E be some set, and let H be a Hilbert space of functions
f : E --+ C with the Hermitian product (., .). Suppose that for each x E E,
the mapping f --+ fex) from H to C is continuous. Show that H is then an
autoreproducing Hilbert space.
Equation (91) therefore teUs us that Vo is an autoreproducing Hilbert space with
reproducing kernel q(x, t). Similarly, for all mEZ, Vm is an autoreproducing
Hilbert space with reproducing kernel qm (x, t), where
(92)
We know (Theorem D3.1) that 1$(0)1 = 1, and we can assume without loss of
generality that $(0) = 1. Therefore, in view of property (38),
$(2klr) = l[k=o}. (93)
D52 Pointwise Convergence Theorem 231
It follows from this and the weak Poisson formula (Theorem A2.3) that
L qJ(x - n) = 1. (94)
Finally, from (89) and (94),
L qm(x, t)dx = 1, (95)
for m = 1, and therefore for all mEZ.

EXERCISED5.2. Give the kernel q(x, t) ofthe Haar MRA (qJ(t) = I[o,l](t) and of
the Shannon MRA (qJ(t) = sin(rrt)jrrt).
In general, the kernel of an MRA does not have a c10sed form, and the examples
in the previous exercise are exceptions.
D52 Pointwise Convergence Theorem

Let f E L~(lR), and denote by fm its projection on Vm. We have
fm(t) = L qm(x, t)f(x)dx,
where qm is the autoreproducing kernel of Vm, defined by (92). This kernel re-
presentation allows us to obtain pointwise convergence results, in the manner
of Dirichlet's pointwise convergence analysis of Fourier series. We need some
preliminary results on the kernel.
DEFINITION D5.4. Let {m}mEZ be a sequence of functions m : lR x lR ~ C.
It is called a quasi-positive delta sequence if it satisfies the three following three
conditions:
(a) There exists afinite nonnegative constant K such that
L Im(x, t)ldx < K, for all t E lR, all mEZ. (96)
(b) There exists afinite nonnegative constant c such that
lim
mtoo
ft-c
t+c
m(x, t)dx = 1, (97)
uniformly with respect to t in compact sets.

(c) For all y > 0,
tim sup 18m (x, t)1 = 0. (98)
mtoo Ix-tlo::y
EXERCISE DS.3. Show that Fejer's kernel sequence

t)
= 2rr (m + I tsm2 e(
sin 2 (m+l(x -
8m (x, t)
'2 x - t
) I[-n,+nj(x - t)
is a quasi-positive delta sequence. Show that Dirichlet's kerne! sequence

sin (m + !)(x - t))
8m(x, t) = . (' ) 1[-n,+nj(x - t)
2:n: sm 2:(x - t)
is not a quasi-positive delta sequence.
THEOREM D5.1. lfthe MRA is r-smooth, the sequence {qm}m E Z defined by (92)
is a quasi-positive delta sequence.
Proof We first prove property (a) of Definition D504:
L Iqm(x, t)ldx =L 2mlq(2mx, 2m t)1 dx
=L Iq(x, 2m t)1 dx
<C2
-
r 1
J[{ (1 + lxi?
dx=K<oo
'
where we have used inequality (90).
We now prove property (b) ofDefinition D504: Let c > 0 be finite. We have
f t+c
qm(X, t)dx =
12m (t+c)
q(x,2mt)dx
t-c ~Q-~
where we have used (92) and (95). But

2mt-2mc 1 _2mc 1
1/,1< C2j dx = C2j
- -00 (1 + Ix - 2m tl)2 -00 (1 + Ixl)2 dx '
and this quantity tends to zero as m tends to infinity. A similar conclusion holds
for h Therefore, property (b) ofDefinition D504 is satisfied.
For property (c) ofDefinition D504, it suffices to observe that
1
2mlq(2m x 2mt)1 < C 2m_ _ _ _ __=_
, - 2 (1+2 mlx-tI)2
in view of (90).

We can now state the main result.
THEOREM D5.2. lf I E L~(JR;)nL~(JR;) is continuous on (a, b), then the projection
Im = PVm I converges pointwise to I, uniformlyon compact subintervals [a, ] E
(a, b), as m -+ 00.
D52 Pointwise Convergence Theorem 233
Proof" This is an immediate consequence of Theorem D5.1 and of the

regularization lemma below.
LEMMA D5.1. Let {8m }mEZ be a quasi-positive delta sequence. Let f E Lb(lR) be
continuous on (a, b), and define for all mEZ thefunction fm by
fm{t) = L 8m(x, t)f(x)dx.
Then
tim fm{t)
m--+oo
= f(t)
uniformlyon any compact subinterval [a, ] E (a, b).
Proof" For y > 0, write
fm{t) = ( 8m(x, t)f(x)dx = ft+ y + f+oo + jt- y

J'R. t-y Hy -00
= f(t) f HY
t-y
8m(x, t) dx
+f t+ y 8 (x, t)(f(t) - f(x dx + (f+oo + jt-

y)
m
t-y t+y -00
= A+B +(C).
Let [a, ] E (a, b), t E [a, ]. Let c be as in (b) of D5.4. Choose y such that
o< y < c, + y < b, a - y > a. For any 0 < 8 < 1, further restrict y so
that If(x) - f(t)1 < 8 whenever t E [a, ] and Ix - tl < y (in which case both t
and x are in a compact subinterval contained in (a, b), and we can then invoke the
uniform continuity of f in this closed interval). We then have
IBI ~ 8 ft+ y 18m(x, t)1 dx ~ 8 { 18m(x, t)1 dx ~ 8K

t-y J'R.
and
ICI ~ sup 18m(x, t)1 { If(x)1 dx ~8 { If(x)1 dx

Ix-tl2:Y J'R. J'R.
for large enough m. Also, for large enough m,
1 1- f~:y 8m(x, t)dxl ~ 8

(use property (b) of the definition of quasi -delta sequences, and the fact that
lim ft+c 8m(x, t)dx =0

mtoo t+y
uniformly with t E ~, and the same for the limit of J/~:). Therefore, If(t) - AI ~
8.
Putting all this together, we have for large enough m
I/(t) - Im(t)1 ::s I/(t) - AI + IBI + ICI ::s Me + e III + Ke,f

where M = SUPtE[a.1I/(t)l.
D53 Regularity Properties ofWave1et Bases
In the wavelet expansion
1= L(f, 1{Ij.n}1{Ij,n = L d j,n1{lj,n,

j,n j,n
where
it is highly desirable from a numerical point ofview that the coefficients dj,n decay
rapidlyas IJ I, Iml -+ 00, thus ensuring fast convergence ofthe wavelet expansion.
This is not the case, however, even for smooth functions (say, I E C oo n L 2 ) if
no further conditions are imposed on the mother wavelet 1{1. To understand this
and see what type of conditions 1{1 should satisfy, let us examine the asymptotic
behavior of
dj,o = 2 j /2 l l(x)1{I(2 j x)* dx
as J -+ 00. Let 2 j = N and set
d(N) = JN l l(x)1{I(Nx)* dx
= JNa(N).
A Taylor expansion of I (assumed to be C OO ) with Lagrange residue gives
where
RK(X) = l
o
x (x
-
K!
t)K
lK+l)(t) dt.
We assurne that the scaling function has a Fourier transform at 0 equal to 1, which
implies that the mother wavelet has a null Fourier trans form at 0 or, equivalently,
that it integrates to O. Therefore,
a(N) = 1'(0) JLl + 1"(0) JL2 + ... + I(K)(O) JLK +r (K)

N2 1! N3 2! N K +1 K! N ,
D53 Regularity Properties of Wavelet Bases 235
where the J,Lk'S are the wavelet moments:
J,Lk = L x k1/l(x)*dx,
and the rest is readily bounded above by

c
rN(K) .:::
NK+2 '
for some finite nonnegative c. In particular, a wavelet with moments that are zero
up to order K implies
deN) .::: -JN N~+2'

We shall see how the smoothness of 1/1 relates to moment conditions.
THEOREM D5.3. Let 1/1 E Sr and assume that {1/Ijkli,kEZ is a Hilbert basis of
LUIR), where
Then
(99)
Let N be a dyadic integer (that is, N = 2- j oko) such that 1/I(N) #- 0 (the
existence of N follows by the density of dyadic integers in IR and by the fact that
1/1 is continuous and not identically zero).
Let j > 1 be sufficiently large for 2j N to be an integer. By orthogonality
o= 2j L 1/1 (x )1/t(2 j x - 2 j N) dx
= L 1/t(Tjy + N)1/I(y)dy. (*)
Passing to the limit j ~ 00 gives, we have by dominated convergence
1/I(N) L 1/I(y)dy = O.
Therefore, (99) is proved for k = O.
Suppose that (99) is true for k = 1, ... , n - 1, where n .::: r. We have the Taylor
expansion
=L
n (x N)k (x N)n
1/t(x) 1/I(k)(N) - + rn(x) - ,
k=O k! n!
where rn(x) is uniformly bounded. Choose N such that 1/I(n)(N) #- O. Substituting
LI~
in (*), we obtain
0= 1/I(k)(N) (2-:~l + rn(Tj y + N) (2-~~)n } 1/I(y) dy

= 1jI(n\N) { Tjn yn 1jI(y)dy +( r n (2- j y + N) 2- jn y n 1jI(y)dy.

J1I? n! J1I? n!
By dominated convergence, the last integral goes to zero as j -+ 00, and therefore,
1jI(n)(N) l yn1jl(y)dy = 0 .
Here is an apparent paradox relative to the moment conditions, and especially

to the condition
l1jl(X)dx = 0,
which is always satisfied and implies that the projection Im = Pvml satisfies
l Im(x)dx = 0, (100)
a surprising fact at first glance, since the function I that is analyzed is in general
not such that
l Im(x)dx = O. (101)
There is actually no contradiction since one cannot pass in the limit m -+ 00 in

(100) to obtain (101): Convergence of Im to I is in L 2 , and this does not imply
that
lim { Im(x)dx = ( Im(x)dx.

mt oo J1I? J1I?
In Mallat's algorithm one first computes the projection Po I that is the approxim-
ation of I at the resolution level 0, and then the coarser resolution approximations
Pj I, j :s -1. As we have just seen, the moment conditions on 1jI are useful for
the first part of the algorithm. For the second part fast decay of the coefficients
hn = h l <p(x)<p(2x - n)* dx
is needed for rapid numerical convergence. An ideal situation is when only a finite
number of h n are nonzero, which is guaranteed ifthe scaling function has compact
support. Note that if this is the case, then the compactness of the scaling function
carries over to the mother wavelet, and this is why one usually talks of compact
wavelets rather than compact scaling functions.
Let us mention at this point that if we start from a Riesz basis of Vo, as in the
method explained in Section D42, the compactness of w (there defined) does not
imply compactness of the scaling function. In the face of this negative statement
one needs to be reassured about the transmission of exponential decay from w to
<p. As a matter of fact the situation is not too bad, and a result in this direction is,
for example, Proposition 5.4.1 in [D3]. We end this section by showing how the
decay ofthe scaling function is transmitted to the coefficients h n Localization of
References 237
the scaling function can be taken in many related senses. We mentioned previously
one of them, namely ({J E Sr. Another definition of localization could be
L+ (1 Ixl)m l({J(x)1 2 dx < 00, forallm E N. (102)
It follows from (102) that for finite constants Cm ,
1 Ixl~A
1({J(x)12 dx::: : : ' (103)
for all A > 0, mE N. By Schwarz's inequality,
Ihni::: ../211
Ixl~A
({J(x)({J(2x -n)dxl
: : . /2 (1
Ixl~A
1({J(2x - n)1 2dx)1/2 +../2 (1 Ixl::::A
1({J(x)1 2 dx)1/2 ,
and therefore with a proper choice of A, saya = n, in view of the tail majorization
(103), we obtain
Dm
Ihn I ::: for all m E N, (104)
(1 + n)m
where the Dm are finite. Thus, the Fourier coefficients of mo are rapidly decaying
and this implies that mo E C oo .
The topic of compact wavelets is an important one, but it is rather technical. The
interested reader is refered to [D3] for the detailed theory.
References
[01] Blatter, C. (1998). Wavelets, a Primer, A. K. Peters: Natick, MA.
[02] Chui, C.K. (1992). An lntroduction to Wavelets, Academic Press: New York.
[03] Oaubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf
[04] Hemandez, E. and Weiss, G. (1996). A First Course on Wavelets, CRC Press: Boca
Raton,FL.
[05] Kahane, J.-P. and Lemarie-Rieusset, P.G. (1998). Stfries de Fourier et Ondelettes,
Cassini: Paris.
[06] Mallat, S. (1998). A Wavelet Tour 0/ Signal Processing, Wiley: New York.
[07] Meyer, Y. (1993). Wavelets Algorithms and Applications, SIAM: Philadelphia, PA.
[08] Vetterli, M. and Kovacevic, J. (1995). Wavelets and Sub-Band Coding, Prentice-
Hall: Englewood Cliffs, NJ.
[09] Walter, G. (1994). Wavelets and Other Orthogonal Systems with Applications,
CRC Press: Boca Raton, Fl.
Appendix
The Lebesgue Integral
Introduction
Integration is almost as old as mathematics. It is at least as old as Greek
mathematics,8 since Eudoxus and Archimedes used the exhaustion method to
compute the volume ofvarious solids, in particular, the pyrarnid and the cone. 9
The modem theory of integration is intimately linked to Fourier series. Indeed,

Bernhard Riemann (1826-1866) developed his theory of integration as a tool for
studying Fourier series, the theme of his memoir of habilitation to professorship at
the University of Gottingen. Also, Renri Lebesgue (1875-1941), who conceived
his theory of integration in the period from 1902 to 1906, stated in a 1903 artic1e: "[
am going to apply the notion of integral to the study of the trigonometrie expansion
offunetions that are not integrable in the sense of Riemann."
The Riemann integral has a few weak points, the two main ones being that
8Sir Thomas Heath, A History of Greek Mathematics; Vol. I: From Thales to Euclid,
Clarendon Press, Oxford, 1921; Dover edition, 1981.
9Exhaustion is the procedure by which we compute, for instance, the volume of the cone
of height h and circular base of radius R, as the limit of a heap of circular tiIes:
lim 2:>' (k)2

n
ntoo k=! n
h
-R -.
n
242 Appendix
(1) The dass of nonnegative functions which are Riemann-integrable is not large
enough. Indeed, some functions have an "obvious" integral, and Riemann's integ-
ration theory denies it, while Lebesgue's theory recognizes it (see Example 9), and
its stability properties under the limit operation are too weak.
(2) The Riemann integral is defined with respect to the Lebesgue measure (the
"volume" in ffi.n), whereas the Lebesgue integral can be defined with respect to a
general abstract measure, a probability for instance.
The last advantage is an excellent argument to convince a student to invest a
little time in the study of the Lebesgue integral, because the return is considerable.
Indeed, the Lebesgue integral ofthe function f with respect to the measure p, (see
the meaning in the first chapter), modestly denoted by
Ix fex) p,(dx),
contains a variety of mathematical objects, for instance, the usual Lebesgue integral
on the line,
L f(x)dx,
and also the Lebesgue volume integral. An infinite sum
can also be viewed (with profit) as a Lebesgue integral with respect to the counting
measure on Z. The Stieltjes-Lebesgue integral
L f(x)dF(x)
with respect to a function F of bounded variation, the expectation of a random

variable Z:
E[Z]
are also in the scope ofLebesgue's integral. For the student who is reluctant to give
up the expertise dearly acquired in the Riemann integral, it suffices to say that any
Riemann-integrable function is also Lebesgue-integrable and that both integrals
then coincide.
Is Lebesgue's theory hard to grasp? Not at all, because most of the results are
very natural, and in that respect, the Lebesgue integral is much easier to manipulate
correctly than the Riemann integral. A tedious (but not difficult) part is the step-by-
step construction of the Lebesgue integral. However, if one just gives a summary
of the main steps without going into the details, this is usually not a cause of
frustration for the student interested in applications. The really difficult part is the
proof of existence of certain measures, but students usually do not mind admitting
such results. For instance, there is an existence theorem for the Lebesgue measure
e (the "length") on JEt It says: There exists a unique measure e on JR: that gives to
the intervals [a, b] the measure b - a. Of course, in order to understand what all the
fuss is about, and what kind of mathematical subtleties hide behind such a harmless
statement, we shall have to be more precise about the meaning of "measure". But
when this is done, one is very much ready to approve the statement although the
proof is not immediate. Of course, in this appendix, the proofs of such "obvious"
results are not given. In fact, the goals of this appendix are to provide a tool and to
give a few tips as to how to use it safely. The reader who has no previous knowledge
ofintegration theory will therefore be very much in the situation of the new recipient
of a driving license who takes the road in spite of her inexperience. Experience is
best acquired on the road, and the main text contains many opportunities for the
student to check her reflexes and to apply the roles that are briefly explained in the
appendix. The student wishing to purehase good insurance is directed to the main
companies, a few of which are listed in the bibliography of this appendix.
Farewell and bon voyage!
Measurable Functions and Measures

In this section, the basic steps in the construction of Lebesgue's integral are
described, and the elementary properties of the integral are stated.
We first recall the notation: N, Z, Q, JR:, C are the sets of, respectively, integers,
relative integers, rationals, real numbers, complex numbers; iR = JR: U {+oo, -oo};
N+ and JR:+ are the sets of positive integers and nonnegative real numbers; iR+ =
JR:+ U {+oo}.
Sigma-Fields
P(X) is the collection of all subsets of an arbitrary set X; card(X) is its cardinal,
that is, the "number" of elements in it.
DEFINITION 1. Afamily X c P(X) of subsets of X is ca lied a sigma-fie1d on X if
(a) X E X,
() (A E X) ===} (:4 EX),
(y) (An E X for all n E N) ===} (U~oAn EX).
One then says that (X, X) is a measurable space.
Two extremal examples of sigma-fields on X are the gross sigma-field X =
{0, X} and the trivial sigma-field X = P(X).
The following situation is often encountered in measure theory: One has a col-
lection of elementary sets, easy to describe mathematically, and one needs to define
a sigma-field that contains these elementary sets and that is not too big.
DEFINITION 2. The sigma-field generated by a collection of subsets C c P(X)
is, by definition, the smallest sigma-field on X containing all the sets in C. It is
denoted by a(C).
244 Appendix
Let {Xi}' i E I, be the collection of all sigma-fields on X containing C. This

collection is not empty, because P(X) belongs to it. Furthermore, one readily
checks that ni EI X; (by definition, the collection of subsets of X that belong to
all the Xi, i E I) is a sigma-field. It contains C and, obviously, it is the smallest
sigma-field containing C. This proves the existence of 0" (C).
For the next definition, the reader not familiar with abstract topology may take
X = ]Rn with the Euclidean topology.
DEFINITION 3. Let X be a topologieal spaee and let 0 be the eolleetion of open
sets defining the topology. The sigma-field B(X) = 0"(0) is ealled the Borel sigma-
field on X associated with the given topology. A set B E B(X) is ealled a Borel
set ofX.
If X = ]Rn is endowed with the Euclidean topology, the Borel sigma-field B(]Rn)
is denoted Bn. For n = 1, we write B(]R) = B. For I = Hi=-l
I j , where I j is
a general interval of]R (I is then called a general reetangle of ]Rn), the Borel
sigma-field B(I) on I consists of all the Borel sets contained in I.
THEOREM 1. B(]Rn) is generated by the eolleetion C of all reetangles ofthe type
TI7=1 (-00, ail, where ai E Qfor all i E {I, ... ,n}.
Measurable Functions
One ofthe central notions ofLebesgue's integration theory is that of a measurable
function.
DEFINITION 4. Let (X, X) and (E, e) be two measurable spaees. Afunetion f :
X ~ E is said to be measurable with respeet to X and e if
f-1(C) E X forall CE e.
This situation is denoted in various ways:
f: (X,X) ~ (E,e), or fEeiX, or fEX,
where the third notation will be used only when (E, e) = (I, B(I, I being a
general rectangle of]Rn, provided the context is clear enough as to the choice of I.
If f : (X, X) ~ (]Rk, Bk) one says that f is a Borelfunetion from X to ]Rk.
(However, this is not quite standard terminology; in the standard terminology,
(X, X) must be some (]Rn, Bn).)
Let B be the sigma-field on i: generated by the intervals of type ( - 00, a], a E :IR.
A function f : (X, X) ~ (i:, B), where (X, X) is an arbitrary measurable space,
is called an extended Borel funetion, or simply a Borel funetion. As for functions
f : (X, X) ~ (]R, B), they are called real Borelfunetions. In general, in a sentence
such as "f is a Borel function defined on X," the sigma-field X is assumed to be
the obvious one in the given context.
It seems difficult to prove measurability since most sigma-fields are not defined
explicitly (see the definition of Bn, for instance). However, the following result
often simplifies the task.
THEOREM 2. Let (X, X) and (E, E) be two measurable spaces, where E = a(C)
for some collection C of subsets of E. Then f : (X, X) f-+ (E, E) if and only if
f-I(C) E X for all C E C.
One immediate application of this result is
EXAMPLE 1. Let (X, X) be a measurable space and let n ~ I be an integer. Then
f = (fl, ... , fn) : (X, X) f-+ (IRn, Bn) if and only iffor all a = (al, ... , an) E
IQt, {f :s a} E X. (Here
{f:S a}:= {x E IR n : !;(x):s a;joralll:s i:s n}.)
The proof follows immediately from 2 and the definition of Bn.
EXAMPLE 2. Let X and E be two topological spaces with respective Borel sigma-
fieids B(X) and B(E). Any continuous function f : X f-+ E is measurable with
respect to B(X) and B(E).
The above result is a direct consequence of Theorem 2 and of the abstract
definition of continuity: f : X f-+ Eis said to be continuous if f-I( 0) is an open
set of X whenever 0 is an open set of E.
Measurability is stable by composition:
THEOREM 3. Let (X, X), (Y, y), and (E, E) be three measurable spaces, and let
q; : (X, X) f-+ (Y, y), g : (Y, y) f-+ (E, E). Then f := g 0 q; : (X, X) f-+ (E, E).
(This follows immediately from the definition ofmeasurability.)

The next result shows that the set of Borel functions is stable by the "usual"
operations.
THEOREM 4. (i) Let f, g : (X, X) f-+ (IR, B). Then f g, f + g, (f/ g ) I golD are real
Borel functions.
(ii) Let fn : (X, X) f-+ (IR, 8), n E N. Then lim infntoo fn and lim SUPntoo fn
are (possibly extended) Borel functions, and the set
{lim sup fn = lim inf fn} = {3 lim fn}
ntoo ntoo ntoo
belongs to X. In particular, if {3limntoo fn} = X, the function limntoo fn is a
(possibly extended) Borelfunction.
Measures
DEFINITION 5. Let (X, X) be a measurable space and let /-L : X f-+ [0,00] be
a setfunction such thatforany denumerablefamily {An}n:::1 ofmutually disjoint
sets in X,
(105)
246 Appendix
The setfunction JL is called a measure on (X, X), and (X, X, JL) is ca lied a measure
space.
Property (105) is the sigma-additivity property.
The following three properties are easy to check:
JL(0) = 0;
(A ~ B and A, B E X) ===} (JL(A):::: JL(B));
(An E X for all nE N) ===} (JL(U~oAn):::: L~o JL(An)).
EXAMPLE 3. Let a E X. The measure Ba defined by Ba(C) = lc(a) is the Dirac
measure ata EX. The setfunction JL : X 1-+ [0,00] defined by
L aj lai(C),
00
JL(C) =
j=O
where aj E i:+ for all i E N, is a measure denoted JL = L~o aj Bai.

EXAMPLE 4. Let {a n}n:::l be a sequence ofnonnegative numbers. The setfunction
JL : P(Z) 1-+ [0,00] defined by JL(C) = LnEC an is a measure on ('I." P(Z)). If
an == 1 we have the counting measure V on '1." where v( C) = card (C).
Next theorem introduces Lebesgue's measure.
THEOREM 5. There exists one and only one measure l on (~, S) such that
la, b]) = b - a. (106)
This measure is called the Lebesgue measure on R

DEf1NITION 6. Let JL be a measure on (X, X).
If JL(X) < 00 the measure JL is called a finite measure.
If JL(X) = 1 the measure JL is called a probability measure.
Ifthere exists a sequence {Kn }n:::l of X such that JL(Kn) < oofor all n ::: 1,
and U~l K n = X, the measure JL is called a sigma-finite measure.
A measure JL on (~n , sn) such that JL( C) < 00 for all bounded Borel sets C is
called aRadon measure.
EXAMPLE 5. The Dirac measure Ba is a probability measure. The counting measure
Von 'I., is a sigma-finite measure. Any Radon measure on (~n , sn) is sigma-jinite.
The Lebesgue measure is aRadon measure.
Cumulative Distribution Function

DEf1NITION 7. Afunction F : ~ 1-+ ~ is called a cumulative distribution function
(c.dj.) ifthefollowing properties are satisfied:
1. Fis nondecreasing;
2. F is right-continuous;
3. F admits a left-hand limit, denoted F(x-), at all x E R
6. Let f-L be aRadon measure on (lR, B), and define
I
EXAMPLE
f-LO, tD ift 2: 0,
FJL(t) = (107)
-f-Lt,OD ift < 0.
This is a c.d.f (use next lemma), and, moreover,
FJL(b) - FJL(a) = f-La, bD,

FJL(a) - FJL(a-) = f-L({a}).
FJL is called the c.d.f of f-L.
From the last formula, we deduce that any point set {a}, a E lR has null Lebesgue
measure, and therefore, any countable subset of lR (Ql, for instance), has null
Lebesgue measure.
The following lemma features the sequential continuity properties of measures.
LEMMA 1. Let (X, X, f-L) be a measure space. Let {An}n~1 be a non-decreasing
(that is, An ~ A n+ 1for all n 2: 1) sequence of X. Then
f-L (0 n=1
An) = lim t
ntoo
f-L(A n). (l08)
Let {Rn }n~ 1 be a nonincreasing (that is, Rn+1 ~ Rn for all n 2: 1) sequence of X
such that f-L(R no ) < 00 for some no E N+. Then
f-L (n n=1
Rn) = lim +f-L(Rn).
n,!,oo
(l09)
Proof: We shall prove (108). This equality follows direct1y from sigma-additivity
since
n-I
f-L(A n) = f-L(Ad +L f-L(Ai+1 - Ai)
i=1
and
f-L(QAn) = f-L(A 1) + ~f-L(Ai+I-Ai)'

The proof of (l09) is left to the reader.

The necessity of the condition f-L(Rno ) < 00 for some no is illustrated by the
following counterexample. Let v be the counting measure on Z, and for all n 2: 1
define Rn = {i E Z : lil 2: n}. Then v(R n) = + 00 for all n 2: 1, and
v (0 Rn) = v(0) = 0.
We now state a fundamental existence result, which generalizes Theorem 5.

248 Appendix
THEOREM 6. Let F : lR f--+ lR be a c.d.j. There exists a unique measure fL on

(lR, ) such that Ffl- = F.
This result is easily stated, but it is not trivial.lt is typical of the existence results,
which ans wer the following type of question: Let C be a collection of subsets of X
with C c X, where X is the sigma-field on X generated by C. Given a set function
u : C f--+ [0,00], does there exist a measure fL on (X, X) such that fL(C) = u(C)
for all C E C, and is it unique?
The reason why such results are nontrlvial is that the sigma-field generated by
C is not explicitly constructed. It is therefore not easy to say what fL( C) should be
when one does not really know what a typical C E X should look like!
Negligible Sets
A very important concept in measure and integration theory is that of negligible
sets, with the correlated notion of "almost everywhere."
DEFINITION 8. A fL-negligible set is a set contained in a set N E X such that
fL(N) = O.
Let P be some property relative to the elements x E X, where (X, X, fL) is
a measure space. One says that P holds fL-almost everywhere (fL-a.e.) if the set
{x EX: x does not satisfy P} is a fL-negligible set.
For instance, if I and g are two Borel functions defined on X, the expression
I Sg fL-a.e.
means that
fL({X : I(x) > g(x)}) = o.
The following result is easy to prove.
THEOREM 7. A countable union 01 fL-negligible sets is a fL-negligible set.
The following result is used several times in the main text.
THEOREM 8. Iltwo continuousfunctions I, g : lR f--+ lR are l-a.e. equal, they are
everywhere equal.
Praol: Let t E lR be such that I(t) =j:. g(t). For any c > 0, there exists s E
[t - c, t + c] such that I(s) = g(s). (Otherwise, the set {t; I(t) =j:. g(t)} would
contain the whole interval [t - c, t + c] and therefore could not be of null Lebesgue
measure.) Therefore, one can construct a sequence {tnk:::l converging to t and
such that l(tn) = g(tn) for all n ~ 1. Letting n tend to 00 yields l(t) = g(t), a
contradiction.

The Integral
Having defined measures and measurable functions, we are ready to construct the
abstract Lebesgue integral.
The Simple Case

A Borel function I: (X, X) 1-+ (R B) ofthe type
k
I(x) = Lai 1A ,(x),
i=l
where k E N+, al,"" ak E lR., Al,"" A k E X, is called an elementary Borel

function (defined on X).
The following result is the key to the construction of the Lebesgue integral:
THEOREM 9. Let I : (X, X) 1-+ (iR, B) be a nonnegative Borelfunction. There ex-
ists a nondecreasing sequence {/n}n2:l olnonnegative elementary Borelfunctions
that converges pointwise to I.
Proof" Take
n2- n -l
In(x) = L kr n 1Ak ,n(x),
k=O
where
Ak,n = {x EX: kr n < I(x) ~ (k + l)r n }.
For any nonnegative elementary Borel function I : (X, X) 1-+ (lR., B) of the
form
k
I(x) = Lai lA/X),
i=l
where ai E lR.+, Ai E X for all i E {l, ... , k}, one defines the integral of I with
respect to J1" denoted
Ix I dJ1" or Ix I(x) J1,(dx), or J1,(f),
by
(110)
The Case of Nonnegative Measurable Functions

If I : (X, X) 1-+ (iR, B) is nonnegative, the integral is defined by
dJ1, = lim t {
Jx{ I ntoo Jx In dJ1" (111)
where {fn}n2:l is a nondecreasing sequence ofnonnegative elementary Borel func-

tions In : (X, X) 1-+ (lR., B) such that limntoo t In = I. This definition can be
shown to be consistent, in that the integral so defined is independent of the choice
250 Appendix
of the approximating sequence. Note that the quantity (111) is nonnegative and
can be infinite. It can be shown that if I ::: g, where I, g : (X, X) 1--+ (~, 13) are
nonnegative, then
In particular, if
1+ = max(f, 0) and 1- = max( - I, 0),
wehave
and therefore,
(112)
Integrable Functions
DEFINITION 9. A measurable function I : (X, X) 1--+ (~, 13) is called a

JL-integrable function if
Ix III dJL < 00. (113)
In this case (see (112)) the right-hand side of
(114)
is meaningful and defines the left-hand side. Moreover, the integral of I with
respect to JL defined in this way is finite.
The integral can be defined for nonintegrable functions in certain circumstances;

for example, it is defined in the nonnegative case even when the function is not
integrable. More generally, if I : (X, X) 1--+ (~, 13) is such that at least one ofthe
integralsIx 1+ dJL or Ix 1- dJL is finite, one defines
(115)
This leads to one of the forms "finite minus finite," "finite minus infinite," and
"infinite minus finite." The case JL(f+) = JL(f-) = + 00 is rigorously excluded
from the definition, because it leads to the indeterminate form "infinite minus
infinite."
Counting Measure and Dirac Measure

The results in the following two examples are easy to prove.
EXAMPLE 7. Any funetion I : Z t-+ lR. is measurable with respeet to P(Z) and .
With the measure IJ. defined in Example 4, and with I ~ Olor instanee, we have
=L
00
1J.(f) anl(n).
n=!
EXAMPLE 8. Let Ca be the Dirae measure at point a E X. Then any I : (X, X) t-+
(lR., ) is ca-integrable, and
ca(f) = I(a).
Elementary Properties of the Integral
First, recall that for all A EX,
Ix lA dlJ. = IJ.(A). (116)
Also, recall the notation JA I dlJ. for Jx lAI dlJ..

THEOREM 10. Let I, g : (X, X) t-+ (i:,13) be IJ.-integrable funetions, and let
a, bE R Then
(a) al + bg is IJ.-integrable and lJ.(al + bg) = alJ.(f) + blJ.(g),
(b) if I = 0 IJ.-a.e., then 1J.(f) = 0; If 1= g IJ.-a.e., then 1J.(f) = lJ.(g),
(e) if I :s g IJ.-a.e., then 1J.(f) :s lJ.(g),
(d) 11J.(f)1 :s IJ.(I/I),
(e) if I ~ 0 IJ.-a.e. and 1J.(f) = 0, then I = 0 IJ.-a.e.,
(f) if 1J.(1 Af) = Olor all A E X, then I = 0 IJ.-a.e..
(g) if I is IJ.-integrable, then III < 00 IJ.-a.e.
For a complex Borel function I : X t-+ C (Le., I = !I + ih, where fI, h :

(X, X) t-+ (lR., )) such that IJ.(I/I) < 00, one defines
(117)
The extension to complex Borel functions of the properties (a), (b), (d), and (f) in
Theorem lO is immediate.
Riemann and Lebesgue
The following result tells us that all the time spent learning about the Riemann
integral has not been in vain.
THEOREM 11. Let I : (lR., ) t-+ (R ) be Riemann-integrable. Then it is
Lebesgue-integrable with respeet to l, and the Lebesgue integral is equal to the
Riemann integral.
EXAMPLE 9. The eonverse is not true: The funetion I defined by I (x) = 1 if x E Ql
and I (x) = 0 if x f/. Ql is a Borel funetion, and it is Lebesgue-integrable with its
integral equal to zero beeause {I =I- O} = Ql, has l-measure zero. However, I is
not Riemann-integrable.
252 Appendix
EXAMPLE 10. The function f : (JR;, B) ~ (R 8) defined by

x
fex) = 1 +x2
does not have a Lebesgue integral, because
+ x x
f (x) = - - 2 1[0 oo)(x) and f-(x) =- - - 2 1(-00 Oj(X)
l+x ' l+x '
have infinite Lebesgue integrals. However, it has a generalized Riemann integral
lim
Atoo
j-A
+A
--2
l+x
x
dx = O.
Limits Under the Integral

The three main results that we need to know in this book are the Lebesgue theorems
(when can we interchange the order of limit and integration ?), the Tonnelli-Fubini
theorems (when can we interchange the order of integration in a multiple integral ?),
and the theorem of completeness of L 2
Lebesgue, Fatou, and Beppo Levi
The following result of Beppo Levi is often called the monotone convergence
theorem.
THEOREM 12. Let fn : (X,.1:') ~ (JR;, 8), n ~ 1, be such that
(i) fn ~ O/L-a.e.,
(ii) fn+l ~ fn /L-a.e.
Then there exists a nonnegativefunction f : (X,.1:') ~ (i, B) such that
lim t fn = f /L-a.e.
ntoo
and
d/L = lim t [
Jx[ f ntoo Jx fn d/L.
The next result is a useful technical tool called Fatou 's lemma.
THEOREM 13. Let fn : (X, X) ~ (i, B), n ~ 1, be such that fn ~ 0 /L-a.e. for
all n ~ 1. Then
J[x (limntooinf fn) d/L :'S limntooinf (1x

fn d/L) . (118)
The domina ted convergence theorem is also called the Lebesgue theorem:
THEOREM 14. Let fn : (X, .1:') ~ (i, B), n ~ 1, be such that, for some function
f: (X,.1:') ~ (i, B) and some /L-integrablefunction g : (X,.1:') ~ (i, B),
(i) lim fn
ntoo
= f, /L-a.e.,
(ii) Ilnl :s Igl JL-a.e.lorall n ~ 1.

Then
Ix I dJL = !~~ (Ix In dJL ) .
The results in Theorems 12 and 14 ensure that under certain circumstances limit
and integration may be interchanged (that is, JL(lim In) = lim JL(fn. The classical
counterexample that shows this is not always true is the following:
EXAMPLE 11. For (X, X, JL) = (IR., {3, l), define
1
In(x) =0 if Ixl>-
n
1
n
:s x :s 0,
1
n
One has
lim In(x) =0 if x =j:. 0,
ntoo
that is, limntoo In = 0 JL-a.e. Therelore, JL(limntoo In) = O. However, JL(fn) =1
lor alt n ~ 1.
Differentiation Under the Integral

A very useful application of the dominated convergence theorem is the theorem
of differentiation underthe integral sign. Let (X, X, JL) be a measure space and let
(a, b) C IR.. Let I : (a, b) x X f-+ IR. and, for all t E (a, b), define It : X f-+ IR.
by It(x) = I(t, x). Assume that for all t E (a, b), It is measurable with respect
to X, and define, if possible, the function I : (a, b) f-+ IR. by the formula
let) = Ix I(t, x) JL(dx). (119)
THEOREM 15. Assume thatlor JL-almost alt x thefunction t "rl I(t, x) is continu-
ousatto E (a, b)andthatthereexistsaJL-integrablefunctiong : (X, X) f-+ (i:, )
such that I/(t, x)1 :s Ig(x)1 JL-a.e. lor alt t in a neighborhood V 01 to. Then
I : V f-+ IR. is welt defined and is continuous at to. Furthermore, assume that
(a) t -+ I(t, x) is continuously differentiable on V lor JL-almost alt x,
() For some JL-integrable function h : (X, X) f-+ (i:, )
/ al (t, X)/ < Ih(x)1 JL-a.e ..

at -
Then I is differentiable at to and
I I (to) = 1- al (to, x) JL(dx).

x at
(120)
254 Appendix
Proof" Let (tn}n~l be a sequence in V \ {to} such that limntoo tn = to, and define
fn(x) = f(t n, x), f(x) = f(to, x). Then, by dominated convergence,
lim I(tn) = I(to).
ntoo
1
Also,
l(tn) - I(to) f(tn, x) - f(to, x) dx
---'--'-~ = JL( ),
tn - to x tn - to
and for some 9 E (0, 1), possibly depending upon n,
If(tn,:~ =~(to, x) I ~ Iir (to + 9(tn - to), X)I
The latter quantity is bounded by Ih(x)l. Therefore, by dominated convergence,
lim l(tn) - I(to) = { (lim f(tn, x) - f(to ) JL(dx)

ntoo tn - to Jx ntoo tn - to
{ af
= Jx at
(to, x) JL(dx).
The Fubini Theorem
Product Measures
Let (Xl, Xl, JLl) and (X2, X2, JL2) be two measure spaces where JLl and JL2 are
sigma-finite measures.
Define the product set X = Xl X X2 and the product sigma-field X = Xl X X 2,
where by definition the latter is the smallest sigma-field on X containing all sets
ofthe form Al x A 2, where Al E Xl. A 2 E X2.
THEoREM 16. There exists unique measure JL on (Xl X X2, Xl x X2) such that
JL(A l x A2) = JLl(A l )JL2(A2) (121)
for all Al E Xl> A 2 E X 2.

The measure JL is the product measure of JLl and JL2, and is denoted JLl JL2.
The above result extends in an obvious manner to a finite number of sigma-finite
measures.
EXAMPLE 12. The typical example of a product measure is the Lebesgue measure
on the space (Rn, Bn): It is the unique measure in on that space that is such that
in( Ai) = i(Ai) forall Al,.., An E B.

Tonnelli and Fubini

Going back to the situation with two measure spaces (the case of a finite number
of measure spaces is similar) we have the following result:
THEOREM 17. Let (X I, XI, Ji,1) and (X I, X 2, Ji,2) be two measure spaces in which
Ji,1 and Ji,2 are sigma-finite. Let (X, X, Ji,) = (XI x X2, XI X X 2, Ji,1 Ji,2).
(A) ToneIli. lf I is nonnegative, then, lor Ji,1-almost all XI, the function X2 --+
I(XI, X2) is measurable with respect to X 2, and
XI --+ ( I(XI, X2) Ji,2(dx2)

JX2
is a measurablefunction with respect to XI. Furthermore,
Ix I dJi, = Ixl [Ix/(XI' X2) Ji,2(dx2)] Ji,1 (dxd. (123)
(B) Fubini. If I is Ji,-integrable, then, lor Ji,1-almost all XI, the function X2 --+
I(XI, X2) is Ji,2-integrable and XI --+ JX2 I(XI, X2) Ji,2(dx2) is Ji,2-integrable, and
(123) is true.
In this text we shall refer to the global result as the Fubini-Tonelli theorem.
Part (A) says that one can integrate a nonnegative Borel function in any order
of its variables. Part (B) says that the same is true of an arbitrary Borel function if
that function is Ji,-integrable. In general, in order to apply Part (B), one must use
Part (A) with I = 1I1 to ascertain whether or not J 1I1 dJi, < 00.
EXAMPLE 13. Consider thefunction I defined on XI x X2 = (1, 00) x (0, 1) by

thelormula
Wehave
= h(X2) ~ 0,
However;
1 1
h(X2) dx2 1= 1
00
(- h(xI)) dxl,

since h ~ f-a.e. on (0, 00). We therelore see that successive integrations yield
different results according to the order in which they are perjormed. As a matter
ollact, I(XI, X2) is not integrable on (0,1) x (1,00).
256 Appendix
Integration by Parts
THEOREM 18. Let 111 and 112 be two sigma-finite measures on (R 8). For any
interval (a, b) c lR,
111 a, b ])112a, b]) = { 111 a, t]) 112(dt) + ( 112a, t)) 111 (dt). (124)
~a.bl ~a.bl
Observe that in the first integral we have (a, t] (c1osed on the right), whereas in
the second integral we have (a, t) (open on the right).
Proof' The proof consists of computing the l1-measure of the square (a, b] x
(a, b] in two ways. The first one is obvious and gives the left-hand side of (124).
The second one consists of observing that 11a, b] x (a, b]) = I1(Dd + I1(D2),
where D I = {(x, y);a < y ::::: b, a < X::::: y} and D 2 = (a, b] x (a, b] \ D I . Then
I1(D 1) and I1(D2) are computed using Tonelli's theorem. For instance,
I1(Dd = L(L I D ,(x, y)111(dX)) 112(dy)
and
L I D ,(x, y)111(dx) = L l\a<x:o;yjI11(dx) = 111a, y]).

Let 11 be aRadon measure on (lR, 8) and let F/i- be its c.dJ. The notation
L g(x) F/i-(dx)
stands for IIR g(x) l1(dx). When this integral is used, it is usually called the
Lebesgue-Stieltjes integral of g with respect to F w With this notation, (124)
becomes
F 1(b)F2(b) - F1(a)F1(b) = ( F 1(x)dF2(x) +( F2(X-) dFl(X),(125)

~.bl ~a.~
where F i := F/i- i (i = 1,2). This is the Lebesgue-Stieltjes version of the
integration by parts formula of ca1culus.
The Spaces LP
For a given P :::: 1, L~(I1) is, roughly speaking (see the details below), the collection
of complex-valued Borel functions J defined on X such that Ix
IfIP dl1 < 00.
We shall see that it is a complete normed vector space over C, that is, a Banach
space. Of special interest to Fourier analysis is the case P = 2, since L~(I1) has
additional structure that makes of it a Hilbert space.
Let (X, X, 11) be a measure space and let J, g be two complex-valued Borel
functions defined on X. The relation R defined by
(fRg) ~ (f = g l1-a.e.)
is an equivalence relation, and we shall denote the equivalence dass of I by {f}.

Note that for any p > 0 (using property b of 10),
(fRg) ===} (Ix III P dJL = Ix Igl P dJL) .
The operations x, +, *, and multiplication by a scalar a E C are defined on the

equivalence dass by
{f} + {g} = {f + g}, {f}* = {f*}, a{f} = {af}, {f}{g} = {fg}.
The first equality means that {f} + {g} is, by definition, the equivalence dass
consisting of the functions 1+ g, where land g are abritrary members of {f}
and {g}, respectively. A similar interpretation holds for the other equalities.
By definition, for a given p ::: 1, L~(JL) is the collection of equivalence dasses
{f} such that IxIIIP dJL < 00. Clearly, it is a vector space over C (for the proof
recall that
CII; Igl) s ~ III P + ~ Igl P

since t -+ t P is a convex function when p ::: 1).
In order to avoid cumbersome notation, in this seetion and in general whenever
we consider LP -spaces, we shall write I for {f}. This abuse of notation is harmless
since two members of the same equivalence dass have the same integral if that
integral is defined. Therefore, using loose notation,
(126)
The following is a simple and often used observation.

THEOREM 19. Let pandq be positive realnumberssuchthat p > q.lfthemeasure
JL on (X, X, JL) is finite, then L~(JL) ~ Lt(JL). In particular, L~(JL) ~ L~(JL).
Proof' From the inequality la Iq S 1 + la IP, true for all a E C, it follows that
JLOIlq) S JL(I) + JLOIIP). Since JL(I) = JL(lR.) < 00, JL(IIlq) < 00 whenever
JL(IIIP) < 00.
Hlder's and Minkowski's Inequalities
THEOREM 20. Let p and q be positive real numbers different Iram 1 such that
1 1
-p + -q = 1
(p and q are then said to be conjugate), and let I, g : (X, X) t-+ (i,8) be
nonnegative. Then, we have Hlder's inequality
(127)
In particular, if I, g E L~(lR.), then Ig E L~(lR.).

258 Appendix
Proof' Let
A = (Ix (fP) dJl)l/P ,

B = (Ix (gq) dJl )l/q
We may assurne that 0 < A < 00, 0 < B < 00, because otherwise Hlder's
inequality is trivially satisfied.
Define F = J/ A, G = g/ A, so that
Ix FP dJl Ix = Gq dJl = 1.
The inequality
1
F(x)G(x)::::: - F(x)P + -1 G(x)q (*)
p q
is trivially satisfied if x is such that F(x) = 0 or G(x) = O. If F(x) > 0 and
G(x) > 0, define
sex) = p In(F(x)), tex) = q In(G(x)).

From the convexity of the exponential function and the assumption that 1/ p +
l/q = 1,
es(x)/p+t(x)/q ::::: .!.. es(x) + .!.. et(x),

p q
and this is precisely the inequality (*). Integrating this inequality yields
1x
(FG)dJl::::: -
1
p
1
+ - = 1,
q
and this is just (127).
THEOREM 21. Let P ~ 1 and let J, g : (X, X) 1-+ (i:, B) be nonnegative and such
that
Ix jPdJl < 00, Ix gP dJl < 00.
Then, we have Minkowski's inequality
Proof" For p = 1 the inequality is obvious. Therefore, assurne p > 1. From

Hlder's inequality,
Ix J(f + g)p-l dJl ::::: [Ix jP dJl r/ [Ix

p
(f + g)(p-l)q r/ q
and
Ix g(f + g)p-l dlL S [Ix gP dlL flP [Ix (f + g)(p-l)q flq

Adding together the above two inequalities and observing that (p - l)q = p, we
obtain
One may assume that the right-hand side of (128) is finite and that the left-
hand side is positive (otherwise the inequality is trivial). Therefore, !x(f +
g)P dlL E (0,00). We may therefore divide both sides of the last display by
t
[Jx (f + g)P dlL q. Observing that 1 - 1/q = 1/P yields the desired inequality
(128).
For the last assertion of the theorem, take p = q = 2.
THEOREM 22. Let p ::: 1. The mapping vp : L~(IL) 1-+ [0,00) defined by
vp(f) = (Ix'/'p dlL )I IP (129)
defines a norm on L~(IL).

ProoJ- Clearly, vp(cxf) = Icxlvp(f) for all cx E C, I E L~(IL).
Also, (vp(f) = 0) <===> UX
I/IP dlLjlP = 0 ===> (f = 0).
Finally, vp(f + g) S vp(f) + vp(g) for all I, g E L~(IL), by Minkowski's
inequality.
Therefore, vp is a norm.
Riesz-Fischer Theorem
We shall denote vp(f) by IIfll p. Thus L~(IL) is a normed vector space over C,
with the norm 11 . 11 p and the induced distance
dp(f, g) = 111 - gllp.
THEOREM 23. Let p ::: 1. The distance d p makes 01 L~ a complete normed space.
In other words, L~(IL) is a Banach space for the norm 11 . I p
ProoJ- To show completeness one must prove that for any Cauchy sequence
(fn}n~1 of L~(IL) there exists I E L~(IL) such that limntoo dp(fn, f) = O.
Since {fn}n~1 is a Cauchy sequence (that is, limm,ntoo dp(fn, Im) = 0), one can
select a subsequence (fn,}i~1 such that
dp(fni+l - In) S Ti. (*)
Let
k
gk =L I/ni+l - Inil,
i=1
260 Appendix
00
g= L
;=1
I/ni+' - Ini I
By (*) and Minkowski's inequality we have IIgk I p :s 1. Fatou's lemma ap-

plied to the sequences {gfk~:1 gives IIgli p :s 1. In particular, any member of
the equivalence c1ass of g is finite /L-almost everywhere, and therefore
L (jni+' (X) -
00
In, (X) + ln/x))

;=1
converges absolutely for /L-almost all x. Call this limit I(x) (set I(x) = 0 when
this limit does not exist). Since
k-I
In, +L (jni+' - Ini) = Ink'

;=1
we see that
I = lim Ink /L-a.e.

ktoo
One must show that I is the limit in Lt(/L) of Unkk~:I' Let e > O. There exists an
integer n = N(e) such that II/n - Im I p :s e whenever m, n 2: N. For all m > N,
by Fatou's lemma we have
ImIPd/L:S liminfll/ni - ImIPd/L:S e P

Jx[ 1I - I~OO x
Therefore, I - Im E Lt(/L), and consequently, I E Lt(/L). It also follows from

the last inequality that
lim
m--+oo
111 - Imllp = O.
Terminology. For p 2: 1, Lt(/L) is a Banach space (a complete normed vector
space) over Co This phrase will implicitly assume that the norm is defined as in
(129). When /L is the Lebesgue measure on jRn, we write Lt(jRn) instead of Lt(/L)
(with a slight symbolic inconsistency).
In the proof of Theorem 23 we obtained the following result.
THEOREM 24. Let Unk:1 be a convergent sequence in Lt(/L), where p 2: 1, and
let I be the limit.
A subsequence {/ni k::1 can then be chosen such that
lim f,n
;too '
=I /L- a.e. (130)
Note that the statement in (130) is about functions and not about equivalence
c1asses. The functions thereof are any members of the corresponding equivalence
c1ass. In particular, since when a given sequence of functions converges /L-a.e. to
two functions, these two functions are necessarily equal/L-a.e.
References 261
THEOREM 25. If{fnln:o:! converges both to f in L~{J.1) and to g f-L-a.e., then f =g

f-L-a.e.
A most interesting special case of LP -space, particularly in view of its relevance
to signal processing, is when p = 2. In this case
THEOREM 26. L~(f-L) is a complete normed space with respect to the norm
IIfII = [Ix Ifl 2 df-L f/2

This norm is derived from a Hermitian product, namely,
(j, g) = Ix fg* df-L
in the sense that
IIfII 2 = (j, f).

L~(f-L) is a Hilbert space over C (see Section CII).
Approximation Theorems
We now quote the approximation results used in the main text.
THEOREM 27. Let f E L~(lR.), P ~ 1. There exists a sequence {fnln:o:! of con-
tinuous functions fn : IR t-+ C with compact support that converges to f in
L~(IR).
(To have compact support means, for a continuous function, to be null outside
some c10sed bounded interval.)
THEOREM 28. Let f E L~(IR), P ~ 1. There exists a sequence {fnln:o:! offunctions
fn : IR t-+ C which are finite linear combinations ofindicatorfunctions ofintervals,
that converges to f in L~(IR).
THEOREM 29. Let fE Lt([-n, +nD be a 2n-periodicfunction (that is, f(t) =
f(t + 2n)forall t E IR, and J~: If(t)1 d t < (0). There exists a sequence {fnln:o:!
of functions fn : IR t-+ C with continuous derivatives that converges to f in
Lt([ -n, +n D.
References
[Dl] de Barra, G. (1981). Measure Theory and Integration, EIlis Horwood: Chichester.
[D2] Halmos, P.R. (1950). Measure Theory, Van Nostrand: New York.
[D3] Royden, H.L. (1988). Real Analysis, 3rd ed., MacMillan: London.
[D4] Rudin, W. (1966). Real and ComplexAnalysis, McGraw-Hill: New York.
[D5] Taylor, A.E. (1965). General Theory of Functions and Integration, Blaisdell,
Waltham, MA, Dover edition, 1985.
Glossary of Symbols
P(X), the collection of all subsets of set X.

card (X), or lXI, the cardinal of set x; the number of elements in X.
N, the integers.
N+, the positive integers.
Z, the relative integers.
IR, the reals.
IR+, the positive reals.
C, the complex numbers.
z*, the complex conjugate of z E C.
(a, b], interval of IR open to the left, c10sed to the right; and similar notation for
the other types of intervals.
Re(z), the real part of z E C.
Im(z), the imaginary part of z E C.

J : IR 1-+ C, a function from IR to C; equivalent notation: J, JO, J(t).
J(n), the nth derivative of J; J(O) = J.
J * g, the convolution product of J and g:
Cf * g)(t) = L J(t - s)g(s)ds = L g(t - s) J(s)ds.
264 Glossary of Symbols
f*n, the nth convolution product of 1 by itself:

1* = I; 1*(n+1) = 1 * f*n.
lA, the indicator function of a set A; lA (t) = 1 if t E A, = otherwise.
h(t) = l(t)l[o,Tj(t).
sinc (f) = sin(:rrt)/m, the cardinal sine function.
rectT (t) = l[_~,+~j(t), the rectangle function.
l, the Lebesgue measure on lR; l([a, b]) =b- a.
a.e., almost everywhere with respect to the Lebesgue measure.
f-L-a.e., almost everywhere with respect to the measure f-L.
L~(lR), the set (equivalence c1asses) of measurable functions 1 : lR ~ C such
that flR I/(t)IP dt < 00.
1 : Ca, b]
f:
L~([a, b]), the set (equivalence c1asses) ofmeasurable functions ~ C
such that I/(t)IP dt < 00.
L~,loc(lR), the set (equivalence c1asses) of measurable functions 1 : lR ~ C such

that I(t)l[a,bj(t) E L~(lR) for all Ca, b] c lR.
Lfoc' short for L~,loc(lR).
l~(Z), the set of complex sequences {Xn}nEZ such that LnEZ IXn 12 < 00.
Cn , the set of ntimes continuously differentiable functions 1 : lR ~ C.

COO, the set of infinitely differentiable functions 1 : lR ~ C.
Co, the set of continuous functions 1 : lR ~ C.
C~, the set of continuous functions 1 : lR ~ C with bounded support.
C([O, T]), the set of continuous functions 1 : [0, T] ~ C.
V, the set oftest functions q; : lR ~ C; C oo and compact support.

V', the set of distributions on lR; the set of linear forms on V.
S, the set of functions 1 : [0, T] ~ C in C oo and with all its derivatives rapidly
decreasing.
Sr. the set of functions 1 : [0, T] ~ C in er and with all its derivatives up to
order r rapidly decreasing.
S', the set of tempered distributions on lR; the set oflinear forms on S.
(x, Y) H, the Hermitian product of x, y EH, H Hilbert space.
IIx 11 H = (x, x) H )1/2, the norm ofx EH, H Hilbert space.

265
x..ly; x is orthogonal to y; (x, y) H = O.
G.L, the orthogonal complement of G.
PG, the orthogonal projection on G.
cn(f) = (I/2n) J~rr f(t)e- int dt, the nth Fourier coefficient of f.
Sn(f) = L~: ck(f)e+ ikt , the Fourier series.
S(f) = LnEZ cn(f)e+ int , the formal Fourier series development.

j(v) = JR. f(t)e-2irrvt dt, the Fourier transform of f.
H(z) = LnEZ hnz n, the z-transform of {hn}nEZ,
CPj,n(t) = 2 j / 2cp(2 j t - n).
Index
Aliasing, 79, 100 convolution, 14

all-pass, 110 convolution-multiplication rule, 14,25
almost everywhere, 248 convolutional filter, 55, 101
amplitude gain, 57 counting measure, 246
analytic signal, 69 cut-off frequency, 58
autocorrelation function, 14, 156
autoreproducing Hilbert space, 230 Decomposable signal, 61
dense, 138
B-splines, 223 differentiating filter, 59
band-pass, 58 dilation equation, 209
base-band, 68 Dini's theorem, 34
Bessel's inequality, 146 Dirac comb, 91
biorthonormal system, 149 Dirac measure, 246
Borel function, 244 Dirichlet integral, 32
Borel set, 244 Dirichlet kerneI, 32, 36
Borel sigma-field, 244 dispersive channel, 72
bounded support, 8 distance, 135
bounded variation, 36 distribution function, 246
Butterworth filter, 67 dominated convergence, 252
down-sampling, 118
C.dJ., 246
cardinal sine, 10 Elementary Borel function, 249
Cauchy sequence, 135
causal filter, 55, 101, 109 Fejer's idenlity, 110
complete, 135 Fejer's kerneI, 40
complex envelope, 69 Fejer's lemma, 112
complex signal, 7 Fatou's lemma, 252
268 Index
feedback filter, 60, 105 localization principle, 33

finite-energy signal, 8 locally integrable, 8, 23
finite measure, 246 locally square-integrable, 8
finite-power signal, 24 locally stable, 23
FIR,115 low-pass, 58
formal Fourier series, 31
Fourier coefficient, 24 Mallat's algorithm, 211
Fourier inversion formula, 17 measurable function, 244
Fourier sum, 100 measure, 246
Fourier transform, 9 measure space, 246
Franklin's wavelet, 222 metric space, 135
frequency response, 56, 101 Mexican hat, 192
frequency transposition, 69 Meyer's wavelet, 218
Ff,9 Minkowski's inequality, 258
Fubini's theorem, 255 monotone convergence, 252
Morlet's pseudo-wavelet, 192
Gabor transform, 178 mother wavelet, 185,201
Gaussian pulse, 10 MRA,196
Gibbs's phenomenon, 38 multiresolution analysis, 196
Gram-Schmidt,145
group delay, 73 Norm, 134
norm of a linear form, 143
Hlder's inequality, 257 Nyquist condition, 85
Haar filter, 121
Haar mother wavelet, 202 Octave band filter, 122
Haar wavelet, 192 orthogonal, 134
Heisenberg's inequality, 176 orthogonal complement, 139
Hermitian product, 133 orthogonalprojection,140
Hilbert basis theorem, 148 orthogonalsum, 143
Hilbert space, 135 orthonormal system, 145
Hilbert span, 136
Hilbert subspace, 136 Parallelogram identity, 134
Hilbert's filter, 58 partial response signaling, 87
periodic signal, 23
Impulse response, 55, 101 phase, 57
indicator function, 8 phase delay, 73
integrating filter, 59 Plancherel-Parseval identity, 157, 162
integration by parts, 256 Poisson sum formula, 91
isometry extension theorem, 138 polarization identity, 134
isomorphic, 137 pre-Hilbert space, 133
probability measure, 246
Jordan's theorem, 36 product measure, 254
product sigma-field, 254
Kernel of an MRA, 229 projection principle, 141
projection theorem, 140
Lebesgue measure, 246 pulse amplitude modulation, 84
Lebesgue-Stieltjes integral, 256 Pythagoras' theorem, 134
linear form, 143
linear isometry, 137 QMF,120
Index 269
quadrature components, 68 sigma-fie1d, 243

quadrature mirror filter, 120 sigma-finite measure, 246
quadrature multiplexing, 70 spectral decomposition, 62
quasi-positive delta sequence, 231 spectral factorization, 112
spectrum folding, 79
Radon measure, 246 stable, 7, 100
realizable filter, 55, 101 synchronous detection, 70
rectangular pulse, 9
regularization lemma, 19 Tonelli's theorem, 255
regularizing function, 19 total, 148
reproducing kernel, 230 transfer function, 104
residue theorem, 105 triangular pulse, 14
resolution level, 196
Riesz basis, 164 Uncertainty princip1e, 175
Riesz's representation theorem, 144 up-sampling, 118
Riesz-Fischer theorem, 259
root mean-square width, 176 Wavelet orthonormal basis, 201
Weierstrass theorem, 41
Sampie and hold, 78 WFT,178
scalar product, 133 WFT inversion formula, 179
scaling function, 196 window function, 169, 178
Schwarz inequality, 134 windowed Fourier transform, 178
Shannon wavelet, 210 WT, 185
short-time Fourier transform, 178
sigma-additivity, 246 Z-transform, 104

Pierre Bremaud - Mathematical Principles of Signal Processing PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Pierre Bremaud - Mathematical Principles of Signal Processing PDF

Uploaded by

Copyright:

Available Formats

Mathematical Principles of

Library of Congress Cataloging in Publication Data

Printed on acid-free paper.

2002 Springer Science+Business Media New York

Production managed by Allan Abrams; manufacturing supervised by Joe Quatela.

A2 Fourier Series of Locally Stable Periodic Signals 23

A3 Pointwise Convergence of Fourier Series 31

C Fourier Analysis in L 2 127

D Wavelet Analysis 167

03 Wavelet Orthonormal Expansions 195

Gif sur Yvette, France Pierre Bremaud

In 1807 Joseph Fourier (1768-1830) presented a solution ofthe heat equation l

(The solution of the heat equation is derived in Seetion A 11.)

f(t) = L cne int .

Special cases of trigonometric developments were known, for instance, by

Outside continuity, the hope for a reasonable theory seemed to be completely

where J is an integrable function (satisfying some additional conditions to be

6Une serie de Fourier-Lebesgue divergente partout, CRAS Paris, 183, 1327-1328.

L If(t)1 dt < 00.

P. Brmaud, Mathematical Principles of Signal Processing

or, for short, I E Lloe'

is denoted by L~(IR). It is the set of square-integrable functions. A signal I(t) in

or, for short, I E L toe'

A function I : IR 1-+ C is said to have bounded support if there exists a bounded

s(v) = L s(t) e-2i1rvI dt. (1)

s(t) ~ s(v) or :F: s(t) --* s(v).

Delay s(t - to) ~ e- 2ill'vtos(v)

Modulation e2i1rvot s(t) ~ s(v - vo)

AISI (t) + A2S2(t) ~ AISI(V) + A2 S2(V)

EXERCISE A1.2. Prove the assertions in Table Al.I.

(See Fig. AI.I.)

s(v) Hs(v + vo) + s(v - vo)}

Figure Al.l. Modulation theorem

and the cardinal sine

recT(t) ~ Tsinc (vT). (3)

reCT(t) Tsinc(vT) = recT(v)

Figure A 1.2. Fourier transfonn of the rectangle function

Yl : (-a, 0) -+ (+a, 0),

For /Z, for instance, if we parameterize Y2 as folIows,

Therefore, since v .:::: a,

Using for YI the obvious parameterization

!im { = lim j+a e- rrt2 dt = { e- rrt2 dt = 1.

Going back to (5), we obtain

which gives the announced result.

Figure A1.4. The integration path in the proof of (6)

The limit as A t 00 of I Yl e- Z dz is I y e- Z dz, and that of I Y3 e- Z dz = IoA e- t dt

I{ e-zdzl ::: e- A IY21,

where IY21 = K x A is the length of Y2.

y(t) =1 h(t - s)x(s) ds (7)

f1xIR Ih(t - s)llx(s)1 dt ds = (l'h(t)' dt) (l'X(t)' dt) < 00.

This implies that, for almost aIl t,

l'h(t - s)x(s)1 ds < 00.

1,y(t)' dt = 111 h(t - s)x(s) dsl dt

Therefore, y(t) is stable. By Fubini's theorem,

L(Lh(t - S)X(S)dS) e-2irrvt dt

By the convolution-multiplication rule,

TriT(t) ~ (Tsine (vT)f (9)

(see Fig. Al.5).

c(t) =L x(s + t)x*(s) ds

Figure A 1.5. FT of the triangle funetion

EXERCISEAl.9. Showthatthenthconvolutionpoweroff(t) = e-atlt~o(t), where

lim (Isn(t) - s(t)1 dt =0

for finite numbers Kn From the inequality