ECE 8443 Pattern Recognition ECE 8423 Adaptive Signal Processing

Objectives:
Periodograms
Bartlett Windows
Data Windowing
Blackman-Tukey
Resources:
Wiki: Periodograms
LS: Bartlett Windows
LS: Blackman-Tukey
SPW: Blackman-Tukey
URL: .../publications/courses/ece_8423/lectures/current/lecture_11.ppt
MP3: .../publications/courses/ece_8423/lectures/current/lecture_11.mp3
LECTURE 11: PERIODOGRAMS AND
BLACKMAN-TUKEY SPECTRAL ESTIMATION
ECE 8423: Lecture 11, Slide 1
Introduction
Recall the power spectrum of a zero-mean stationary signal, x(n), with
autocorrelation r(n), is defined by:
Our concern in this chapter is how to estimate the spectrum of x(n) from more
than one finite set of data. For example, can we average successive estimates
to obtain a better estimate than simply using all the data? (We considered a
similar problem in the Pattern Recognition course.)
Methods of spectral analysis are divided into two groups:
Classical: operate as nonparameteric estimators and do not impose any
model or structure on the data.
Modern: assume a model structure for the observed data and estimate the
parameters of that model.
There are many possible ways to estimate the power spectrum derived from
the autocorrelation function, such as:
Spectral estimates derived directly from the data are known as periodograms;
those derived from the autocorrelation function are known as correlograms.

=
n
n j j
e n r R(e
e e
) ( )
( ) { } ( )

=

=
)
`

+
=
M
M n
n j j
M
j
M
M
j
e n x e X e X E
M
R(e
e e e e
) ( where
1 2
1
lim )
2
ECE 8423: Lecture 11, Slide 2
The Periodogram
The periodogram estimate of the power spectrum is defined as:
Of course, there are many ways to estimate the autocorrelation function. This
particular estimate can be rewritten as:
This shows us the periodogram is the Fourier transform of the biased
correlation estimate, r(m).
Bias: r(m) is itself biased with , but is also
asymptotically unbiased because this estimate converges as M .
Variance: the periodogram is not a consistent estimator, which means
increasing the number of data samples does not decrease the variance of the
estimator at any one frequency.
Analytic evaluation of the variance is difficult, but we can demonstrate the
problem both experimentally and analytically for a simple case.

( ) ( ) ( ) ( )


=

= = =
1
0
2
2
1
0
) ( where
1
or ) (
1
M
n
n j j j j
p
M
n
n j j
p
e n x e X e X
M
e R e n x
M
e R
e e e e e e
( ) ) ( ) (
1
) ( where ) (
1
0
1
) 1 (
m n x n x
M
m r e m r e R
m M
n
M
M m
n j j
p

=

+ = ' ' =
e e
{ } ) ( ) ( m r
M
m M
m r E
|
|
.
|

\
|
= '
ECE 8423: Lecture 11, Slide 3
Variance of the Periodogram Experimental
Consider a simple AR process:
Generate 1024 samples:
Compute the periodogram for
M=32, 128, 1024.
Observe that at any frequency
the variance does not decrease,
though the shape of the overall
spectrum does change because
the discrete frequencies used in
the DFT are more closely
spaced.
) ( ) 1 ( 9 . 0 ) ( n w n x n x + =
ECE 8423: Lecture 11, Slide 4
Variance of the Periodogram Analytic
Consider the simple case when x(n) is a zero-mean IID Gaussian sequence.
The variance is given by:
The second term is easy to evaluate:
The first term is a bit more tedious, but, for a zero-mean IID process can be
shown to reduce to:
The overall variance becomes:
However, observe that:
which means that our estimate of the autocorrelation is not consistent.
( ) { } ( ) { } ( ) { }
2
2 e e e j
p
j
p
j
p
e R E e R E e R Var =
( ) { }
2
2
1
0
) (
1
w
M
n
n j j
p
e n x
M
E e R E o
e e
=

( ) { }

)

|
.
|

\
|
+ =
2
2 2
sin
) sin( 1
2
e
e
o
e
M
M
e R E
w
j
p
( ) { }

)

|
.
|

\
|
+ =
2
4
sin
) sin( 1
1
e
e
o
e
M
M
e R Var
w
j
p
( ) { }
4
2
4
sin
) sin( 1
1 lim lim
w w
M
j
p
M
M
M
e R Var o
e
e
o
e
=

|
.
|

\
|
+ =

ECE 8423: Lecture 11, Slide 5
Generalizations
We can generalize our result by calculating the covariance of the periodogram
for two frequencies:
For white input signals, the covariance between adjacent frequencies is zero
(which is bad), but as M increases, the covariance remains zero, which means
the variance does not decrease (refer to the plots on slide 3).
Similarly, for a signal generated by passing a zero-mean IID sequence through
a linear time-invariant system, we can show:
The variance of the periodogram estimate is proportional to the square of the
true spectrum.
Once again we observe the variance does not decay with increasing M,
confirming that the periodogram is not a consistent estimator.
These observations led researchers to develop parametric techniques that
were asymptotically consistent. These observations also led to ways to
precondition the data so that the periodogram would be consistent or at the
very least, more accurate.
( ) ( ) { } ( ) ( ) { } ( ) { } ( ) { }
2 1 2 1 2 1
cov
e e e e e e j
p
j
p
j
p
j
p
j
p
j
p
e R E e R E e R e R E e R e R =
( ) { } ( )
e e j
xx
j
px
M
e R e R Var
2
{ lim =

ECE 8423: Lecture 11, Slide 6
The Bartlett Window
Recall our expression for the bias in the autocorrelation estimator:
We can view this as a windowing process:
This window is known as the triangular or Bartlett window. We recall that the
impact on the spectrum is a convolution of the signal and the frequency
response of the window:
( )
{ }
( ) { }

|
|
.
|

\
|
=
|
|
.
|

\
|
= '
' =
1
) 1 (
1
) 1 (
) (
) ( ) (
) (
M
M m
n j j
p
M
M m
n j j
p
e m r
M
m M
e R E
m r
M
m M
m r E
e m r e R
e e
e e
( ) { }

<

= =

elsewhere 0
) ( where , ) ( ) (
M m
M
m M
m w e m r m w e R E
b
m
n j
b
j
p
e e
( )
2
)
2
sin(
)
2
sin(
1
|
|
|
|
.
|

\
|
=
e
e
e
M
M
e W
j
b
ECE 8423: Lecture 11, Slide 7
The Modified Periodogram
For these reasons it is common to use a time-domain window with the
periodogram:
We can show this is equivalent to:
Bias:
Variance and Consistency:
( )
2
1
0
) ( ) (
1

=
M
n
n j
d
j
w
e n x n w
MC
e R
e e
( ) ( )
) ( ) ( ) ( ) (
1
) (
where
) (
1
0
1
) 1 (
m n x m n w n x n w
MC
m r
e m r e R e R
m M
n
d d
M
M m
n j j
p
j
w

+ + = ' '
' ' = =
e e e
( ) { }


=

+ = =
1
0
1
) 1 (
) ( ) (
1
) ( where ) ( ) (
m M
n
d d
M
M m
n j j
w
m n w n w
MC
m g e m g m r e R E
e e
( ) { } ( )
e e j j
w
e R e R Var
2
~
ECE 8423: Lecture 11, Slide 8
Averaging Periodograms
We can produce smooth estimates of the periodogram by averaging:
This is known as Bartletts estimate.
Bias:
Variance and Consistency:
This demonstrates that the variance decreases monotonically as the number
of averages increases.

=
= =
=
= + =
K
i
j i
p
j
A
L
n
n j i j i
p
i
e R
K
e R
K i e n x
L
e R
L n
K i iL n x n x
1
) (
2
1
0
) ( ) (
) (
) (
1
) (
1 ..., , 1 , 0 ) (
1
) (
1 ..., , 1 , 0
1 ..., , 1 , 0 ); ( ) (
e e
e e
( ) { }

=
1
) 1 (
) ( ] 1 [
L
L m
m j j
A
e m r
L
m
e R E
e e
( ) { } ( ) { }
e e j
p
j
A
e R Var
K
e R Var
1
~
ECE 8423: Lecture 11, Slide 9
Blackman-Tukey Spectral Estimation
We can generalize the notion of windowing:
The window, w
l
(n), is a symmetric data window of length 2M
1
-1.
This estimate is known as the Blackman-Tukey or correlogram. When the
window is rectangular, equivalent to the raw periodogram.
The window is symmetric so that the BT estimate is a real and even function.
As before, this can be viewed in the frequency domain as the convolution of
the raw periodogram estimate and the frequency response of the window.
Bias:
Variance and Consistency:
Note that this is a consistent estimator because the variance tends to zero as
M .
( ) ) ( ) (
1
) ( where ) ( ) (
1
0
1
) 1 (
1
1
m n x n x
M
m r e n r n w e R
m M
n
M
M n
n j
l
j
BT

=

+ = ' ' =
e e
( ) { } ( ) u
t
u u e e
d e R e W e R E
j
W
W
j
l
j
BT
}

~ ) (
2
1
) (
( ) { }
( )
|
|
.
|

\
|
~

=
1
) 1 (
2
2
1
1
) (
M
M m
l
j
j
BT
m w
M
e R
e R Var
e
e
ECE 8423: Lecture 11, Slide 10
Introduced the periodogram.
Discussed the bias and variance of the raw periodogram.
We generalized this to a windowed periodogram and discussed the Bartlett
window.
We also introduced the concept of a smoothed periodogram computed by
averaging adjacent windows.
Finally, we introduced the Blackman-Tukey approach and discussed its bias
and variance.
Summary