Signal Processing FIR & IIR

Digital Signal Processing II
Lecture 2: FIR & IIR Filter Design
Marc Moonen
Dept. E.E./ESAT, K.U.Leuven
marc.moonen@esat.kuleuven.be
www.esat.kuleuven.be/scd/
DSP-II p. 1
PART-I : Filter Design/Realization
• Step-1 : define filter specs

(pass-band, stop-band, optimization criterion,…)
• Step-2 : derive optimal transfer function
FIR or IIR design Lecture-2
• Step-3 : filter realization (block scheme/flow graph)
direct form realizations, lattice realizations,… Lecture-3
• Step-4 : filter implementation (software/hardware)
finite word-length issues, …
question: implemented filter = designed filter ?
Lecture-4
DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 2
Lecture-2 : FIR & IIR Filter Design
• FIR filters
– Linear-phase FIR filters
– FIR design by optimization
Weighted least-squares design, Minimax design
– FIR design in practice
`Windows’, Equiripple design, Software (Matlab,…)
• IIR filters
– Poles and Zeros
– IIR design by optimization
Weighted least-squares design, Minimax design
– IIR design in practice
Analog IIR design : Butterworth/Chebyshev/elliptic
Analog->digital : impulse invariant, bilinear transform,…
Software (Matlab)

FIR Filters
FIR filter = finite impulse response filter

B( z ) 1 N
H ( z)   b0  b1 z  ...  bN z
zN
• Also known as `moving average filters’ (MA)
• N poles at the origin z=0 (hence guaranteed stability)
• N zeros (zeros of B(z)), àll zero’ filters
• corresponds to difference equation
y[k ]  b0 .u[k ]  b1.u[k  1]  ...  bN .u[k  N ]
• impulse response
h[0]  b0 , h[1]  b1 ,..., h[ N ]  bN , h[ N  1]  0,...

Linear Phase FIR Filters
• Non-causal zero-phase filters :

example: symmetric impulse response
h[-L],….h[-1],h[0],h[1],...,h[L]
h[k]=h[-k], k=1..L
k
L
frequency response is e  jx  e  jx  2. cos x
L L
j
H (e )   h[k ].e
k  L
 j . k
 ...   d k . cos( .k )
k 0
i.e. real-valued (=zero-phase) transfer function

• Causal linear-phase filters = non-causal zero-phase + delay

example: symmetric impulse response & N even
h[0],h[1],….,h[N]
N=2L (even)
h[k]=h[N-k], k=0..L
k
0 N
frequency response is
N L
H ( e )   h[k ].e
j  j . k
 ...  e  jL
. d k . cos( .k )
k 0 k 0
= i.e. causal implementation of zero-phase filter, by

introducing (group) delay z
L
 e jL
z e j

Type-1 Type-2 Type-3 Type-4
N=2L=even N=2L+1=odd N=2L=even N=2L+1=odd
symmetric symmetric anti-symmetric anti-symmetric
h[k]=h[N-k] h[k]=h[N-k] h[k]=-h[N-k] h[k]=-h[N-k]

L L 1
. d k . cos( .k ) e  jN / 2 cos( ). d k . cos( .k ) je jN / 2 sin(  ). d k . cos( .k ) j.e  jN / 2 sin(  ). d k . cos( .k )
L L
 jN / 2
e
k 0
2 k 0 k 0 2 k 0
zero at    zero at   0,  zero at   0

LP/HP/BP LP/BP BP HP
PS: `modulating’ Type-2 with 1,-1,1,-1,.. gives Type-4 (LP->HP)

PS: `modulating’ Type-4 with 1,-1,1,-1,.. gives Type-2 (HP->LP)
PS: `modulating’ Type-1 with 1,-1,1,-1,.. gives Type-1 (LP<->HP)
PS: `modulating’ Type-3 with 1,-1,1,-1,.. gives Type-3 (BP<->BP)
PS: IIR filters can NEVER have linear-phase property ! (proof see literature)

Filter Specification
Ex: Low-pass
1.2
Passband Ripple
1
1   P 0.8
 P S
0.6 Passband Cutoff -> <- Stopband Cutoff
0.4 Stopband Ripple S

0.2
0
0 0.5 1 1.5 2 2.5 3

FIR Filter Design by Optimization
(I) Weighted Least Squares Design :

• select one of the basic forms that yield linear phase
L
e.g. Type-1 j
H (e )  e  jN / 2
. d k . cos( .k )  e  jN / 2 . A( )
k 0
• specify desired frequency response (LP,HP,BP,…)

H d ( )  e  jN / 2 . Ad ( )
• optimization criterion is
 
min d0 ,..., d L  W ( ) H (e j )  H d ( ) d  min d0 ,..., d L  W ( ) A( )  Ad ( ) d
2 2
 
 
F ( d 0 ,..., d L )
where W ( )  0 is a weighting function

• …this is equivalent to F ( d 0 ,..., d L )

min x {xT .Q.x  2 xT . p  }
xT  d 0 d1 ... d L 

Q   W ( ).c( ).cT ( ) d
0

p   W ( ). Ad ( ).c( )d
0
cT ( )  1 cos( ) ... cos( L )

  ...
= `Quadratic Optimization’ problem
xOPT  Q 1. p
1.2
Passband Ripple
• Example: Low-pass design 1
0.8
Ad ( )  1,    P (pass - band) 0.6 Passband Cutoff -> <- Stopband Cutoff
Ad ( )  0,  S     (stop - band)
0.4 Stopband Ripple
0.2
optimization function is
P  0
0 0.5 1 1.5 2 2.5 3
F ( d 0 ,..., d L )   A( )  1 d   .  A ( )d  ...
2 2

0
  S
 
pass - band stop - band
i.e.
W ()  1,   P W ()   , S    

• a simpler problem is obtained by replacing the F(..) by…
F (d0 ,..., d L )  W (i ). A(i )  Ad (i )
2
where the wi’s are a set of (n) selected sample frequencies

This leads to an equivalent (`discretized’) quadratic optimization function:
2
 d0  
 
F ( d 0 ,..., d L )  W (i ) cT (i ).  :   Ad (i )   x T .Q. x  2 x T . p  
 
i  d L  
 
Q  W (i ).c(i ).cT (i ), p  W (i ). Ad (i ).c(i ),   ...

i i
1
xOPT  Q . p Compare to p.10
+++ : simple
- - - : unpredictable behavior in between sample freqs.
• This is often supplemented with additional constraints, e.g.
for pass-band and stop-band ripple control :
A( P ,i )  1   P , for pass - band freqs.  P,1 ,  P ,2 ,... ( P is pass - band ripple)
A(S ,i )   S , for stop - band freqs. S ,1 ,  S ,2 ,... ( S is stop - band ripple)
• The resulting optimization problem is :

minimize : F (d ,..., d )  ... (=quadratic function)
0 L
xT  d 0 d1 ... d L 
subject to AP .x  bP (=pass-band constraints)
AS .x  bS (=stop-band constraints)
= `Quadratic Programming’ problem
(II) `Minimax’ Design :

• select one of the basic forms that yield linear phase
L
e.g. Type-1 H (e j )  e  jN / 2 . d . cos( .k )  e  jN / 2 . A( )
k
k 0

H d ( )  e  jN / 2 . Ad ( )
min d0 ,..., d L max 0  W ( ). H (e j )  H d ( )  min d0 ,..., d L max 0  W ( ). A( )  Ad ( )

• this is equivalent to
min x 
subject to
2
W ( ). c ( ). x  Ad ( )  
2 T
for 0    
with
x T  d 0 d1 ... d L 
cT ( )  1 cos( ) ... cos( L )
• the constraint is equivalent to a so-called `semi-definiteness’ constraint
  W ( ).(cT ( ).x  Ad ( )) 
D( )   0
W ( ).(c ( ).x  Ad ( ))
T
1 
where D>=0 denotes that the matrix is positive semi-definite

Filter Design by Optimization
• a realistic way to implement these constraints, is to impose

the constraints (only) on a set of sample frequencies :
min x 
subject to
 D(1 ) 0 ... 0 
 0 D ( ) ... 0 
 2 0
 : : : 
 
 0 0 ... D ( m 
)
i.e. a `Semi-Definite Programming’ (SDP) problem, for

which efficient interior-point algorithms and software are
available.
• Conclusion:
(I) weighted least squares design
(II) minimax design
provide general `framework’, procedures to translate filter
design problems into standard optimization problems
• In practice (and in textbooks):
emphasis on specific (ad-hoc) procedures :
- filter design based on `windows’
- equiripple design

FIR Filter Design using `Windows’
Example : Low-pass filter design

• ideal low-pass filter is
1   C
H d ( )  
0  C     
• hence ideal time-domain impulse response is

1 j sin( c k )
 H
j
hd [k ]  (e ).e .k
d  ...   .  k  
2 c k
d

• truncate hd[k] to N+1 samples :

hd [k ] N /2 k  N /2
h[k ]  
 0 otherwise
• add (group) delay to turn into causal filter

Example : Low-pass filter design (continued)
• PS : it can be shown that the filter obtained by such time-domain
truncation is also obtained by using a weighted least-squares design
procedure with the given Hd, and weighting function
W ( )  1
• truncation corresponds to applying a `rectangular window’ :
h[k ]  hd [k ].w[k ]
1 N /2 k  N /2
w[k ]  
0 otherwise
• +++: simple procedure (also for HP,BP,…)
• - - - : truncation in time-domain results in `Gibbs effect’ in frequency
domain, i.e. large ripple in pass-band and stop-band (at band edge
discontinuity), which cannot be reduced by increasing the filter order N.
Remedy : apply windows other than rectangular window:

• time-domain multiplication with a window function w[k] corresponds to
frequency domain convolution with W(z) :
h[k ]  hd [k ].w[k ]
H ( z)  H d ( z ) *W ( z )
• candidate windows : Han, Hamming, Blackman, Kaiser,…. (see
textbooks, see DSP-I)
• window choice/design = trade-off between side-lobe levels (define peak
pass-/stop-band ripple) and width main-lobe (defines transition
bandwidth)

FIR Equiripple Design
• Starting point is minimax criterion, e.g.
min d0 ,..., d L max 0  W ( ). A( )  Ad ( )  min d0 ,..., d L max 0  E ( )
• Based on theory of Chebyshev approximation and the àlternation

theorem’, which (roughly) states that the optimal d’s are such that the
`max’ (maximum weighted approximation error) is obtained at L+2
extremal frequencies…
max 0  E( )  E (i ) for i  1,.., L  2
…that hence will exhibit the same maximum ripple (èquiripple’)

• Iterative procedure for computing extremal frequencies, etc. (Remez
exchange algorithm, Parks-McClellan algorithm)
• Very flexible, etc., available in many software packages
• Details omitted here (see textbooks)

FIR Filter Design Software
• FIR Filter design abundantly available in

commercial software
• Matlab:
b=fir1(n,Wn,type,window), windowed linear-phase FIR design, n is filter
order, Wn defines band-edges, type is `high’,`stop’,…
b=fir2(n,f,m,window), windowed FIR design based on inverse Fourier
transform with frequency points f and corresponding magnitude
response m
b=remez(n,f,m), equiripple linear-phase FIR design with Parks-
McClellan (Remez exchange) algorithm
See exercise sessions

IIR filters
Rational transfer function :

• B( z ) b0 z N  b1 z N 1  ...  bN b0  b1 z 1  ...  bN z  N
H ( z)   N N 1

A( z ) z  a1 z  ...  aN 1  a1 z 1  ...  aN z  N
N poles (zeros of A(z)) , N zeros (zeros of B(z))
• infinitely long impulse response
• stable iff poles lie inside the unit circle
• corresponds to difference equation
y[k ]  a1. y[k  1]  ...  aN . y[k  N ]  b0 .u[k ]  b1.u[k  1]  ...  bN .u[k  N ]
y[k ]  b0 .u[k ]  b1.u[k  1]  ...  bN .u[k  N ] a1. y[k  1]  ...  aN . y[k  N ]
 
`MA' ` AR '
= also known as ÀRMA’ (autoregressive-moving average)

IIR Filter Design
+++
• low-order filters can produce sharp frequency response
• low computational cost
---
• design more difficult
• stability should be checked/guaranteed
• phase response not easily controlled
(e.g. no linear-phase IIR filters)
• coefficient sensitivity, quantization noise, etc. can be a
problem (see Lecture-4)
IIR filters
Frequency response versus pole-zero location :
(cfr. frequency response is z-transform evaluated on the unit circle)
Example-1 :
pole pole
Low-pass filter
poles at Nyquist freq (z=-1)

0 .8  0 .2 j
Re
Im DC (z=1)
pole near unit-circle introduces `peak’ in frequency response
DSP-II hence pass-band

Version 2008-2009can be set by poleFIR/IIR
Lecture-2 placement
Filter Design p. 25
IIR filters
Frequency response versus pole-zero location :
pole pole
Example-2 :
Low-pass filter
poles at Nyquist freq
0 .8  0 .2 j
zeros at
0.75  0.66 j
zero DC
zero near (or on) unit-circle introduces `dip’ (or transmision zero) in freq. response
DSP-II hence stop-band

Version can be emphasized
2008-2009 by FIR/IIR
Lecture-2 zeroFilterplacement
Design p. 26
IIR Filter Design by Optimization
(I) Weighted Least Squares Design :
• IIR filter transfer function is
B( z ) b0  b1 z 1  ...  bN z  N
H ( z)  
A( z ) 1  a1 z 1  ...  a N z  N
H d ( )

min b0 ,..., bN ,a1 ,..., a N  W ( ) H (e )  H d ( ) d
j 2

  
F ( b0 ,..., bN , a1 ,..., a N )

• stability constraint : A( z)  0, z  1

(II) `Minimax’ Design :

• IIR filter transfer function is
B( z ) b0  b1 z 1  ...  bN z  N
H ( z)  
A( z ) 1  a1 z 1  ...  a N z  N
H d ( )
min b0 ,..., bN ,a1 ,..., aN max 0  W ( ). H ( e j )  H d ( )

• stability constraint :
A( z)  0, z  1

These optimization problems are significantly more
complex than those for the FIR design case… :
• Problem-1: presence of denominator polynomial leads to

non-linear (non-quadratic) optimization
– A possible procedure (`Steiglitz-McBride’) consists in iteratively
(k=1,2,…) minimizing

F (bk , 0 ,..., bk , N , ak ,1 ,..., ak , N )   Wk ( ) B(e j )  H d ( ). A(e j ) d
2

W ( )
Wk ( ) 
Ak 1 ( )
2
…which leads to iterative Quadratic Programming, etc..

(similar for minimax)
These optimization problems are significantly more

complex than those for the FIR design case… :
• Problem-2: stability constraint

(zeros of a high-order polynomial are related to the polynomial’s
coefficients in a highly non-linear manner)
– Solutions based on alternative stability constraints, that

e.g. are affine functions of the filter coefficients, etc…
– Topic of ongoing research, details omitted here

• Conclusion:
(I) weighted least squares design
(II) minimax design
provide general `framework’, procedures to translate filter
design problems into ``standard’’ optimization problems
• In practice (and in textbooks):
emphasis on specific (ad-hoc) procedures :
- IIR filter design based analog filter design (s-domain
design) and analog->digital conversion
- IIR filter design by modeling = direct z-domain design
(Pade approximation, Prony, etc., not addressed here)
Analog IIR Filter Design
Commonly used analog filters : G ( s )  H ( s ).H ( s ) 

1
 s2
• Lowpass Butterworth filters 1 (
 2
c
)N
1
G ( j )  H ( j ) 
2
- all-pole filters (H) characterized by a specific  2N
1 ( )
magnitude response (G): (N=filter order) c
- poles of G(s)=H(s)H(-s) are equally spaced on circle of radius
c
N=4
N
poles of H(s) poles of H(-s)
c
- Then H(jw) is found to be monotonic in pass-band & stop-band, with
`maximum flat response’, i.e. (2N-1) derivatives are zero at   0, 

Commonly used analog filters :
• Lowpass Chebyshev filters (type-I)
all-pole filters characterized by magnitude response
G ( s)  H ( s ).H ( s)  ... (N=filter order)
1
G ( j )  H ( j ) 
2

1   2TN2 ( )
c
T0 ( x )  1
 is related to passband ripple
T1 ( x )  x
TN (x) are Chebyshev polynomials:
T2 ( x )  2 x 2  1
...
TN ( x )  2 x.TN 1 ( x )  TN 2 ( x )

Commonly used analog filters :
• Lowpass Chebyshev filters (type-I)
All-pole filters, poles of H(s)H(-s) are on ellipse in s-plane
Equiripple in the pass-band
Monotone in the stop-band
• Lowpass Chebyshev filters (type-II)
Pole-zero filters based on Chebyshev polynomials
Monotone in the pass-band
Equiripple in the stop-band
• Lowpass Elliptic (Cauer) filters
Pole-zero filters based on Jacobian elliptic functions 1
H ( j ) 
2
Equiripple in the pass-band and stop-band 

1   2U N ( )
c
(hence) yield smallest-order for given set of specs
Frequency Transformations :
• Principle : prototype low-pass filter (e.g. cut-off frequency = 1
rad/sec) is transformed to properly scaled low-pass, high-pass, band-
pass, band-stop,… filter
s
• example: replacing s by  moves cut-off frequency to  C
C
C
• example: replacing s by turns LP into HP, with cut-off frequency  C
s
s 2  1. 2
• example: replacing s by turns LP into BP
s.( 2  1 )
• etc...
Analog -> Digital
• Principle :
design analog filter (LP/HP/BP/…), and then convert it to a digital filter.
• Conversion methods:
- convert differential equation into difference equation
- convert continuous-time impulse response into discrete-time
impulse response
- convert transfer function H(s) into transfer function H(z)
• Requirement: the left-half plane of the s-plane should map into the
inside of the unit circle in the z-plane, so that a stable analog filter is
converted into a stable digital filter.

Analog -> Digital
(I) convert differential equation into difference equation :
-in a difference equation, a derivative dy/dt is replaced by a `backward
difference’ (y(kT)-y(kT-T))/T=(y[k]-y[k-1])/T, where T=sampling interval.
-similarly, a second derivative, etc…
-eventually (details omitted), this corresponds to replacing s by (1-1/z)/T
in Ha(s) (=analog transfer function) : H ( z )  H (s) 1 z 1
a s
T
jw j
s-plane z-plane
s  0  z 1
1 s z 0
-stable analog filters are mapped into stable digital filters, but pole
location for digital filter confined to only a small region (o.k. only for LP
or BP)

Analog -> Digital
(II) convert continuous-time impulse response into
discrete-time impulse response :
-given continuous-time impulse response ha(t), discrete-time impulse
response is h[k ]  ha (k.T ) where T=sampling interval.
-eventually (details omitted) this corresponds to a (many-to-one) mapping
j
ze sT
s-plane
jw
z-plane
s  0  z 1 1
s   j / T  z  1
-aliasing (!) if continuous-time response has significant frequency

content above the Nyquist frequency
Analog -> Digital
(III) convert continuous-time system transfer function into
discrete-time system transfer function : Bilinear Transform
-mapping that transforms (whole!) jw-axis of the s-plane into unit circle
in the z-plane only once, i.e. that avoids aliasing of the frequency
components.
H ( z )  H a (s) s  2 .( 1 z 1 ) s  0  z 1
T 1 z 1 s  j.  z  1
jw j
s-plane z-plane
1
-for low-frequencies, this is an approximation of z  e
sT
-for high frequencies : significant frequency compression (`warping’)

(sometimes pre-compensated by `pre-warping’)
IIR Filter Design Software
• IIR filter design considerably more complicated
than FIR design (stability, phase response, etc..)
• (Fortunately) IIR Filter design abundantly available
in commercial software
• Matlab:
[b,a]=butter/cheby1/cheby2/ellip(n,…,Wn),
IIR LP/HP/BP/BS design based on analog prototypes, pre-warping,
bilinear transform, …
immediately gives H(z) (oef!)
analog prototypes, transforms, … can also be called individually
filter order estimation tool
etc...
DSP-II
See exercise sessions
Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 40

Signal Processing FIR &amp; IIR

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Signal Processing FIR &amp; IIR

Uploaded by

Copyright:

Available Formats

Digital Signal Processing II

Lecture 2: FIR & IIR Filter Design

• Step-1 : define filter specs

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 3

FIR filter = finite impulse response filter

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 4

• Non-causal zero-phase filters :

i.e. real-valued (=zero-phase) transfer function

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 5

• Causal linear-phase filters = non-causal zero-phase + delay

= i.e. causal implementation of zero-phase filter, by

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 6

zero at    zero at   0,  zero at   0

PS: `modulating’ Type-2 with 1,-1,1,-1,.. gives Type-4 (LP->HP)

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 7

0.4 Stopband Ripple S

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 8

(I) Weighted Least Squares Design :

• specify desired frequency response (LP,HP,BP,…)

where W ( )  0 is a weighting function

cT ( )  1 cos( ) ... cos( L )

Ad ( )  1,    P (pass - band) 0.6 Passband Cutoff -> <- Stopband Cutoff

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 11

where the wi’s are a set of (n) selected sample frequencies

Q  W (i ).c(i ).cT (i ), p  W (i ). Ad (i ).c(i ),   ...

• The resulting optimization problem is :

(II) `Minimax’ Design :

• specify desired frequency response (LP,HP,BP,…)

where W ( )  0 is a weighting function

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 14

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 15

• a realistic way to implement these constraints, is to impose

i.e. a `Semi-Definite Programming’ (SDP) problem, for

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 17

Example : Low-pass filter design

• truncate hd[k] to N+1 samples :

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 18

• truncation corresponds to applying a `rectangular window’ :

Remedy : apply windows other than rectangular window:

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 20

• Starting point is minimax criterion, e.g.

min d0 ,..., d L max 0  W ( ). A( )  Ad ( )  min d0 ,..., d L max 0  E ( )

• Based on theory of Chebyshev approximation and the `alternation

…that hence will exhibit the same maximum ripple (`equiripple’)

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 21

• FIR Filter design abundantly available in

See exercise sessions

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 22

Rational transfer function :

= also known as `ARMA’ (autoregressive-moving average)

poles at Nyquist freq (z=-1)

DSP-II hence pass-band

poles at Nyquist freq

DSP-II hence stop-band

where W ( )  0 is a weighting function

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 27

(II) `Minimax’ Design :

where W ( )  0 is a weighting function

DSP-II Version 2008-2009 Lecture-2 FIR/IIR Filter Design p. 28

• Problem-1: presence of denominator polynomial leads to

…which leads to iterative Quadratic Programming, etc..

These optimization problems are significantly more

• Problem-2: stability constraint

– Solutions based on alternative stability constraints, that

Signal Processing FIR & IIR

Signal Processing FIR & IIR