MODULE I

DETECTION AND
ESTIMATION
DIGITAL COMMUNICATION
„ Module I
Introduction to Digital communication: Random
variables and random processes – Detection and
estimation – G G-SS procedure – Geometric
interpretation of signals – Response of a bank of
correlators to noisy input – Detection of known
signals
i l in i noise
i – probability
b bili off error – correlation
l i
and matched filter receiver – detection of signals
with unknown phase in noise.
noise
Estimation concepts criteria: MLE – estimator
quality measures – Cramer Rao bound – Wiener
filter for wave form estimation – Linear
pprediction.
Compiled by MKP for CEC S6-EC, DC, Dec 2008
 g
Orthogonal functions
„ Consider a set of functions φ1 (t ), φ2 (t ), φ3 (t ),........, φn (t ),........ defined
over
o e thee interval
e a
„ Let these functions satisfy the condition
t2
∫ t1
φi (t )φ j (t )dt = 0 i ≠ j
„ A set of functions which has this property is said to be orthogonal
over the interval from t1 to t2 .
„ Suppose we have an arbitrary function s(t) and we are interested in s(t)
only in the interval t1 to t2 .where the set of functions Ф(t) are
orthogonal.
th l
„ Now we can express s(t) as a linear sum of the functions Фn(t).

s(t ) = s1φ1 ( t ) + s2φ2 ( t ) + ...... + snφn ( t ) + ...... − − − (1)

where s1,s2,s3 ,…,sn are constants.

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 g
Orthogonal functions
„ If such an expansion is possible, the orthogonality of the Ф’s makes it
easy to
o compute
co pu e thee coe
coefficients
c e s s n.
„ To evaluate sn we multiply both sides of equation (1) by Фn(t) and
integrate over the interval of orthogonality.
t2 t2 t2 t2
∫ s(t )φn (t ) = s1 ∫ φ1 ( t )φn (t )dt + s2 ∫ φ2 ( t )φn (t )dt + ...... + sn ∫ φn 2 (t )dt + ......
t1 t1 t1 t1

„ Because of the orthogonality all of the terms on the RHS of the above
equation become zero except a single term.
t2 t2
∫ t1
s(t )φn (t ) = sn ∫ φn 2 (t )dt
t1
t2

Then, sn =
∫ t1
s(t )φn (t )dt
t2
∫ t1
φn 2 (t )dt

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 g
Orthogonal functions
t2
„ If we select the functions Фn(t) such that
t2
∫t1
φn 2 (t )dt = 1
Then, sn = ∫ s(t )φn (t )dt
t1
t2
„ When orthogonal functions are selected with the condition, ∫ φn 2 (t )dt = 1
t1
they are said to be normalized to have unit energy.
„ The use of normalized functions has the advantage that sn’s can be
easily evaluated.
evaluated
„ A set of functions which are both orthogonal and normalized is
called an orthonormal set.
set
„ We can represent any real valued signals as linear combinations of
orthonormal basis functions.

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 g
Gram-Schmidt Orthogonalization Procedure
„ We may represent a given set of real-valued energy signals
s1(t), s2(t), s3(t),
(t),……….. sM(t) each
eac of
o duration
du a o T seconds,
seco ds, us
using
g
orthonormal basis functions as given below.

⎧0 ≤ t ≤ T
si (t ) = ∑ j =1 sijφ j (t ) ⎨
N
− − − (1)
⎩i = 1,2,....., M
„ The coefficients of the expansion sij are defined by

T ⎧i = 1,2,....., M
sij = ∫ si (t )φ j (t )dt ⎨
0
⎩ j = 1,2,...., N
„ The functions φ1 ( t ), φ2 ( t ), φ3 ( t ),........, φ N ( t ) are orthonormal which
means
means,
T ⎧0 for i ≠ j
∫0
φi (t )φ j (t )dt = ⎨
⎩1 ffor i = j
Compiled by MKP for CEC S6-EC, DC, Dec 2008
 g
Gram-Schmidt Orthogonalization Procedure
„ The first condition states that the basis functions φ1(t ),φ2 (t ),.....,φN (t )
are orthogonal with respect to each other over the interval 0 to T.
„ The second condition states that the basis functions are normalized to
have unit energy.

„ Given the set of coefficients {sij}, j=1,2,…,N .we can generate the
signal si(t), i=1,2,….M using a scheme as shown in figure (1).
„ It consists of a bank of N multipliers,
multipliers with each multiplier supplied with
its own basis function, followed by a summer.

„ Conversely, given the set of signals si(t), i=1,2,….M , operating as

input, we can generate the coefficients {sij}, j=1,2,…,N using the
scheme given in figure (2)
„ It consists of a set of N product-integrators or correlators with a
common input, and with each supplied with its own basis function.

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure

si1

φ1 ( t )
si 2 si (t )
φ2 ( t )

siN
φN ( t )
N
⎧0 ≤ t ≤ T
si (t ) = ∑ sijϕ j (t ), ⎨ Figure (1)
j =1 ⎩i = 1, 2, …, M

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 g
Gram-Schmidt Orthogonalization Procedure
T
∫ 0
dt si1

φ1 ( t )
T
si 2
∫ 0
dt
si (t ) φ2 ( t )

T
∫ 0
dt siN
φN ( t )
T ⎧i = 1, 2, …, M
sij = ∫ si (t )ϕ j (t )dt , ⎨ Figure (2)
0
⎩ j = 1, 2, …, N
Compiled by MKP for CEC S6-EC, DC, Dec 2008
Gram-Schmidt Orthogonalization Procedure
„ If the given set of signals s1(t), s2(t), s3(t),……….. sM(t) are linearly
dependent,
p then there exists a set of coefficients a1,a2,…..,aM not all
zero, such that we may write
a1s1 (t ) + a2 s2 (t ) + ⋅⋅⋅⋅⋅ + aM sM (t ) = 0
„ If aM ≠ 0 then we can express the corresponding signals sM(t) as
⎡ a1 a2 aM −1 ⎤
sM ( t ) = − ⎢ s1 (t ) + s2 (t ) + ⋅⋅⋅⋅⋅ + sM −1 (t ) ⎥
⎣ aM aM aM ⎦
„ It implies that sM(t) may be expressed in terms of the remaining (M-1)
signals.
„ Next consider the set of signals s1(t), s2(t), s3(t),……….. sM-1(t) . If this set
is linearly dependent there exists a set of numbers b1,b2,…..,bM-1 not all
equal to zero, such that
b1s1 (t ) + b2 s2 (t ) + ⋅⋅⋅⋅⋅ + bM −1sM −1 (t ) = 0

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure
„ Suppose that bM-1 ≠ 0 . Then we may express sM-1(t) as a linear
combination of the remaining M-1 signals.
⎡ b1 b2 bM −2 ⎤
sM −1 (t ) = − ⎢ s1 (t ) + s2 (t ) + ⋅⋅⋅⋅⋅ + sM − 2 ( t ) ⎥
⎣ bM −1 bM −1 bM −1 ⎦
„ Testing
T ti the
th sett off signals
i l s1(t),
( ) s2(t),
( ) s3(t),………..
() ( ) for
sM-2(t) f linear
li
dependencies, and continuing in this fashion, we will eventually end
up with a linearly independent subset s1(t), s2(t), s3(t),……….. sN(t) , N≤M
of the original signal.
„ It is important to note that each member of the original set of signals
s1(t), s2(t), s3(t),……….. sM(t) may be expressed as a linear combination of
this subset of N signals s1(t), s2(t), s3(t),……….. sN(t).

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure
„ Now we may express the linearly independent functions
s1(t), s2(t), s3(t),……….. sN(t) in terms of orthonormal functions
φ1 ( t ), φ2 ( t ), φ3 ( t ),........, φ N ( t ) using

⎧0 ≤ t ≤ T
si (t ) = ∑ j =1 sijφ j (t ) ⎨
N

⎩i = 1,2, ....., N

s1 (t ) = s11φ1 (t)+s12φ2 (t)+ ⋅⋅⋅⋅⋅ + s1 N φN (t) − − − (1 a.)

s2 (t ) = s21φ1 (t)+s22φ2 (t)+ ⋅⋅⋅⋅⋅ + s2 N φN (t) − − − (1 b.))

sN (t ) = sN 1φ1 (t)+sN 2φ2 (t)+ ⋅⋅⋅⋅⋅ + sNNφN (t) − − − (1 n.)

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure
„ In equation (1 a.), set to zero all coefficients except s11. We then have

s1 (t ) = s11φ1 (t)
„ Since φ1 (t) is to be a normalized function,
T
s11 = ∫0
s12 (t )dt = E1
s1 (t ) s1 (t )
φ1 (t) = =
s11 E1
„ In the next step we set to zero all the coefficients except the first two
s21 and s22 in equation (1b)

s2 (t ) = s21φ1 (t)+s22φ2 (t) − − − (3)

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure
„ Multiply both sides of equation (3) by φ1 (t) and integrating over the
interval 0-T
T T T
∫ s2 (t )φ1 (t) = ∫ s21φ1 (t)+ ∫ s22φ1 (t)φ2 (t)
2
0 0 0
T T
∫ 0
s2 (t )φ1 (t) = s21 s21 = ∫ s2 (t )φ1 (t)
0

„ To evaluate s22, we rewrite equation (3) as

s2 (t ) − s21φ1 (t) = s22φ2 (t) − − − (4)

„ Squaring and integrating we have
T T
∫ [ s (t ) − s φ (t)] dt = s22 ∫ φ2 (t)dt =s22 − − − (5)
2 2 2 2
2 21 1
0 0

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure
T
s22 = ∫ [ s (t ) − s φ ((t))] dt
2
2 21 1
0

„ Again re-writing equation (3)

1
φ2 (t) = [ s2 (t ) − s21φ1 (t)]
s22
⎡ s21s1 (t) ⎤ 1 ⎡ s21s1 (t) ⎤
1 φ2 (t)= ⎢ s2 (t ) − s
=
s22 ⎢ s2 (t ) − s ⎥ s22 ⎣
⎥
⎦
⎣ 11 ⎦ 11

„ Continuing in this manner we re-write equation(1c) as

s3 (t ) = s31φ1 (t)+s32φ2 (t)+s32φ2 (t)

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure
„ Manipulating the equation as above we get
T
s31 = ∫ s3 (t )φ1 (t)
0

T
s32 = ∫ s3 (t )φ2 (t)
0

T
s33 = ∫ [ s (t ) − s
0
3 31 1φ (t) − s32φ2 (t)]dt

s3 (t ) − s31φ1 (t) − s32φ2 (t)

φ3 (t ) =
s33

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure
„ We continue in this manner until we have used all the N equations
and obtained all the orthonormal functions φ1 ( t ), φ2 ( t ), φ3 ( t ),........, φ N ( t )
and d allll ththe coefficients
ffi i t sij neededd d tto express th the ffunctions ti
s1(t), s2(t), s3(t),……….. sN(t) in terms of φ1 ( t ), φ2 ( t ), φ3 ( t ),........, φ N ( t )

„ Since all of the derived subset of linearly independent signals

s1(t), s2(t), s3(t),……….. sN(t) may be expressed as a linear combination of
the orthonormal basis functions φ1 ( t )), φ2 ( t )), φ3 ( t )),........, φ N ( t ) , it follows
that each one of the original set of signals s1(t), s2(t), s3(t),……….. sM(t)
may be expressed as a linear combination of the same set of basis
functions
functions.

Compiled by MKP for CEC S6-EC, DC, Dec 2008

Gram-Schmidt Orthogonalization Procedure

s1 (t )
φ1 (t ) =
E1 s1 (t ) = E1φ1 (t ) = s11φ1 (t )
T
s21 = ∫ s2 (t )φ1 (t )dt
d
0

g 2 (t ) = s2 (t ) − s21φ1 (t )
g 2 (t )
φ2 (t ) = T T
∫ g (t )dt s2 (t ) = ∫ g22 (t )dtφ2 (t ) + s21φ1 (t )
2
2
0
0
T
sij = ∫ si (t )φ j (t )dt = s22φ2 (t ) + s21φ1 (t )
0
i −1
g i (t ) = si (t ) − ∑ sijφ j (t )
g i (t ) j =1
φi (t ) = T
∫0
g i2 (t )dt

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
⎧0 ≤ t ≤ T
si (t ) = ∑ j =1 sijφ j (t ) ⎨
N
− − − (1)
⎩i = 1,2,....., M
T ⎧i = 1,2,....
1 2 ., M
sij = ∫ si (t )φ j (t )dt ⎨
0
⎩ j = 1,2,...., N
„ Each signal in the set { Si(t) } is completely determined by the vector of
its coefficients as given by
⎡ S i1 ⎤
⎢S ⎥
⎢ i2 ⎥
s i = ⎢ i ⎥ i = 1, 2, 3, ⋅ ⋅ ⋅⋅, M
⎢ ⎥
⎢ i ⎥
⎢⎣ S iN ⎥⎦

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
„ The vector Si is called signal vector.
„ We mayy now expand p the conventional idea of two and
three dimensional Euclidean spaces to an N-dimensional
Euclidean space.
„ We can visualize the set of signal vectors { Si },
i=1,2,3,……,M as defining a corresponding set of M points
i an N-dimensional
in N di i l Euclidian
E lidi space.
„ The N mutually perpendicular axes are labeled Ф1, Ф2,
Ф3,….. ФN
„ This N-dimensional Euclidian space is called signal space.
„ Figure (3) shows a two-dimensional
two dimensional signal space with
three signals, that is, N=2 and M=3.

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
N = 2, M = 3
φ2 3

s2

s1
0 1
2
3
φ1

s3
2

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
„ The length of a vector si is defined by the symbol Si
„ The dot p
product of any
y vector with itself g
gives the squared
q length.
g
N
= ( si ⋅ si ) = ∑ sij − − − (4)
2 2
si
j=1
„ The cosine of the angle between the vectors Si and Sj is given by

cos θ =
( s i ⋅ sj )
si sj
„ The two vectors are orthogonal if their dot product is zero.

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
„ The energy of a signal si (t) of duration T is defined as
T
Ei = ∫ si (t )dt 2
si (t ) = ∑ j =1 sijφ j (t )
N
0
T
= ∫ si (t )dt2
0
T
= ∫ si (t ) si (t )dt
d
0
T
=∫ ⎡ ∑ N sijφ j (t ) ⎤ ⎡ ∑ N sikφk (t ) ⎤ dt
0 ⎣ j =1 ⎦ ⎣ k =1 ⎦
„ Interchanging the order of summation and integration,
T
Ei = ∑ j =1 ∑ k =1 sij sik ∫ φ j (t )φk (t )dt
N N

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
T
But ∫ 0
φ j (t )φk (t )dt =11 for j = k
Ei = ∑ j =1 ∑ k =1 sij sik = ∑ j =1 sij
N
N N 2
− − − (5)

„ From equations (4) and (5) we can see that the energy of a signal si(t)
is equal to the squared length of the signal vector si representing it.

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
„

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
„

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
„

Compiled by MKP for CEC S6-EC, DC, Dec 2008

 p
Geometric interpretation g
of signals
„

Compiled by MKP for CEC S6-EC, DC, Dec 2008