Channel Coding

Overview
Channel coding

Linear block codes
The error detection and correction capability
Encoding and decoding
Hamming codes
Cyclic codes

Block diagram of a DCS
Channel coding:
Transforming signals to improve
communications performance by increasing
the robustness against channel impairments
(noise, interference, fading, ..)
Waveform coding: Transforming waveforms
to better waveforms
Structured sequences: Transforming data
sequences into better sequences, having
structured redundancy.
Better in the sense of making the decision
process less subject to errors.

What is channel coding?
Waveform Coding
Antipodal Signals
Waveform Coding
Orthorgonal Signals
Transorthogonal
M-Ary Modulation.
Error control techniques
Automatic Repeat reQuest (ARQ)
Full-duplex connection, error detection codes
The receiver sends a feedback to the transmitter,
saying that if any error is detected in the received
packet or not (Not-Acknowledgement (NACK) and
Acknowledgement (ACK), respectively).
The transmitter retransmits the previously sent
packet if it receives NACK.
Forward Error Correction (FEC)
Simplex connection, error correction codes
The receiver tries to correct some errors
Hybrid ARQ (ARQ+FEC)
Full-duplex, error detection and correction codes
Why using error correction coding?
Error performance vs. bandwidth
Power vs. bandwidth
Data rate vs. bandwidth
Capacity vs. bandwidth
(dB) /
0
N E
b
B
P
A
F
B
D
C
E
Uncoded
Coded
Coding gain:
For a given bit-error probability,
the reduction in the Eb/N0 that can be
realized through the use of code:
[dB] [dB] [dB]
c
0
u
0
|
|
.
|

\
|

|
|
.
|

\
|
=
N
E
N
E
G
b b
Channel models
Discrete memory-less channels
Discrete input, discrete output
Binary Symmetric channels
Binary input, binary output
Gaussian channels
Discrete input, continuous output

Binary symmetric channels (BSC)




Modulator
input
1-p
p
p
1
0 0
1
Demodulator
output
) 0 | 0 ( ) 1 | 1 ( 1
) 1 | 0 ( ) 0 | 1 (
p p p
p p p
= =
= =
0
2
c
E
P Q
N
| |
=
|
|
\ .
AWGN channels



2
2
( ) 1
( | ) exp
2
2
k
k
z u
p z u
o
o t
| |

=
|
\ .
Linear block codes

Let us review some basic definitions first
which are useful in understanding Linear
block codes.
Some definitions
Binary field :
The set {0,1}, under modulo 2 binary
addition and multiplication forms a field.




Binary field is also called Galois field, GF(2).

0 1 1
1 0 1
1 1 0
0 0 0
=
=
=
=
1 1 1
0 0 1
0 1 0
0 0 0
=
=
=
=
Addition
Multiplication
Some definitions
Fields :
Let F be a set of objects on which two
operations + and . are defined.
F is said to be a field if and only if
1. F forms a commutative group under + operation.
The additive identity element is labeled 0.

2. F-{0} forms a commutative group under .
Operation. The multiplicative identity element is
labeled 1.

3. The operations + and . distribute:
F a b b a F b a e + = + e ,
F a b b a F b a e = e ,
) ( ) ( ) ( c a b a c b a + = +
Some definitions
Vector space:
Let V be a set of vectors and F a fields of
elements called scalars. V forms a vector space
over F if:
1. Commutative:
2.
3. Distributive:

4. Associative:
5.

V u v V v e = e e a F a ,
v u v u v v v + = + + = + a a a b a b a ) ( and ) (
F V e + = + e u v v u v u,
) ( ) ( , , v v v = e e b a b a V F b a
v v V v = e 1 ,
Some definitions
Examples of vector spaces
The set of binary n-tuples, denoted by



Vector subspace:
A subset S of the vector space is called a
subspace if:
The all-zero vector is in S.
The sum of any two vectors in S is also in S.
Example:
. of subspace a is )} 1111 ( ), 1010 ( ), 0101 ( ), 0000 {(
4
V
n
V
n
V
)} 1111 ( ), 1101 ( ), 1100 ( ), 1011 ( ), 1010 ( ), 1001 ( ), 1000 (
), 0111 ( ), 0101 ( ), 0100 ( ), 0011 ( ), 0010 ( ), 0001 ( ), 0000 {(
4
= V
Some definitions
Spanning set:
A collection of vectors ,
the linear combinations of which include all vectors in
a vector space V, is said to be a spanning set for V or
to span V.
Example:

Bases:
A spanning set for V that has minimal cardinality is
called a basis for V.
Cardinality of a set is the number of objects in the set.
Example:
{ } . for basis a is ) 0001 ( ), 0010 ( ), 0100 ( ), 1000 (
4
V
{ } . spans ) 1001 ( ), 0011 ( ), 1100 ( ), 0110 ( ), 1000 (
4
V
{ }
n
G v v v , , ,
2 1
=
Linear block codes
Linear block code (n,k)
A set with cardinality is called a
linear block code if, and only if, it is a
subspace of the vector space .


Members of C are called code-words.
The all-zero codeword is a codeword.
Any linear combination of code-words is a
codeword.
n
V
n
V C c
k
2

n k
V C V c
Linear block codes contd

n
V
k
V
C
Bases of C
mapping
Linear block codes contd
The information bit stream is chopped into blocks of k bits.
Each block is encoded to a larger block of n bits.
The coded bits are modulated and sent over channel.
The reverse procedure is done at the receiver.



Data block
Channel
encoder
Codeword
k bits n bits
rate Code
bits Redundant
n
k
R
n-k
c
=
Linear block codes contd




A matrix G is constructed by taking as its
rows the vectors on the basis, .
n
V
k
V
C
Bases of C
mapping
} , , , {
2 1 k
V V V
(
(
(
(

=
(
(
(

=
kn k k
n
n
k
v v v
v v v
v v v

2 1
2 22 21
1 12 11
1
V
V
G
Linear block codes contd
Encoding in (n,k) block code






The rows of G, are linearly independent.
mG U=
k n
k
k n
m m m u u u
m m m u u u
V V V
V
V
V
+ + + =
(
(
(
(

=
2 2 2 1 1 2 1
2
1
2 1 2 1
) , , , (
) , , , ( ) , , , (

Linear block codes contd
Example: Block code (6,3)

(
(
(

=
(
(
(

=
1
0
0
0
1
0
0
0
1
1
1
0
0
1
1
1
0
1
3
2
1
V
V
V
G
1
1
1
1
1
0
0
0
0
1
0
1
1
1
1
1
1
0
1
1
0
0
0
1
1
0
1
1
1
1
1
0
0
0
1
1

1
1
0
0
0
1
1
0
1

0
0
0
1
0
0
0
1
0
1
0
0
1
1
0
0
1
0

0
0
0
1
0
0
0
1
0
Message vector Codeword
Linear block codes contd
Systematic block code (n,k)
For a systematic code, the first (or last) k
elements in the codeword are information bits.
matrix ) (
matrix identity
] [
k n k
k k
k
k
k
=
=
=
P
I
I P G
) ,..., , , ,..., , ( ) ,..., , (
bits message
2 1
bits parity
2 1 2 1


k k n n
m m m p p p u u u

= = U
Linear block codes contd
Linear block codes contd
Linear block codes contd
For any linear code we can find an
matrix , which its rows are
orthogonal to rows of :

H is called the parity check matrix and
its rows are linearly independent.
For systematic linear block codes:
n k n ) (
H
G
0 GH =
T
] [
T
k n
P I H

=
Linear block codes contd
Linear block codes contd
Two required properties of the parity
check matrix:
1. No column of H can be all zeros, or else an error
in the corresponding codeword position would not
affect the syndrome and would be undetectable.
2. All columns of H must be unique. If two columns
of H were identical, errors in these two
corresponding codeword positions would be
indistinguishable.
Linear block codes contd






Syndrome testing:
S is syndrome of r, corresponding to the error
pattern e.
The syndrome is the result of a parity check
performed on r to determine whether r is a valid
member of the codeword set.

Format
Channel
encoding
Modulation
Channel
decoding
Format
Demodulation
Detection
Data source
Data sink
U
r
m
m

channel
or vector pattern error ) ,...., , (
or vector codeword received ) ,...., , (
2 1
2 1
n
n
e e e
r r r
=
=
e
r
e U r + =
T T
eH rH S = =
Linear block codes contd
Example
Suppose that codeword U = 101110 is transmitted and the
vector r = 001110 is received; that is, the leftmost bit is
received in error. Find the syndrome vector value S = rH
T

and verify that it is equal to eH
T
.
Solution


Linear block codes contd
Standard array
1. For row , find a vector in of minimum
weight which is not already listed in the array.
2. Call this pattern and form the row as the
corresponding coset

k k n k n k n
k
k
2 2
2
2 2
2
2 2 2 2
2
2 1
U e U e e
U e U e e
U U U

zero
codeword
coset
coset leaders
k n
i

= 2 ,..., 3 , 2
n
V
i
e th : i
Linear block codes contd
Standard array and syndrome table decoding
1. Calculate
2. Find the coset leader, , corresponding to .
3. Calculate and corresponding .

Note that
If , error is corrected.
If , undetectable decoding error occurs.
T
rH S =
i
e e =
S
e r U

+ =
m

)

(

e (e U e e) U e r U + + = + + = + =
e e =

e e =

Linear block codes contd

Example: Standard array for the (6,3) code
010110 100101 010001
010100 100000
100100 010000
111100 001000
000110 110111 011010 101101 101010 011100 110011 000100
000101 110001 011111 101011 101100 011000 110111 000010
000110 110010 011100 101000 101111 011011 110101 000001
000111 110011 011101 101001 101110 011010 110100 000000

Coset leaders
coset
codewords
Linear block codes contd
Example: Standard array for the (6,3) code
Linear block codes contd

111 010001
100 100000
010 010000
001 001000
110 000100
011 000010
101 000001
000 000000
(101110) (100000) (001110)

estimated is vector corrected The

(100000)

is syndrome this to ing correspond pattern Error

(100) (001110)
: computed is of syndrome The
received. is (001110)
ted. transmit (101110)
= + = + =
=
= = =
=
=
e r U
e
H rH S
r
r
U
T T
Error pattern Syndrome
Linear block codes contd
The Hamming weight of vector U, denoted by
w(U), is the number of non-zero elements in
U.
The Hamming distance between two vectors
U and V, is the number of elements in which
they differ.

The minimum distance of a block code is


) ( ) ( V U V U, = w d
) ( min ) , ( min
min i
i
j i
j i
w d d U U U = =
=
Linear block codes contd
Error detection capability is given by


Error correcting-capability t of a code, which is
defined as the maximum number of
guaranteed correctable errors per codeword, is





(

=
2
1
min
d
t
1
min
= d e
Linear block codes contd
For memory less channels, the probability
that the decoder commits an erroneous
decoding is

is the transition probability or bit error probability
over channel.
The decoded bit error probability is
j n j
n
t j
M
p p
j
n
P

+ =

|
|
.
|

\
|
s

) 1 (
1
j n j
n
t j
B
p p
j
n
j
n
P

+ =

|
|
.
|

\
|
~

) 1 (
1
1
p
Linear block codes contd
Discrete, memoryless, symmetric channel model




Note that for coded systems, the coded bits are
modulated and transmitted over channel. For
example, for M-PSK modulation on AWGN channels
(M>2):


where is energy per coded bit, given by
Tx. bits Rx. bits
1-p
1-p
p
p
( ) ( )
|
|
.
|

\
|
|
.
|

\
|
=
|
|
.
|

\
|
|
.
|

\
|
~
M N
R E M
Q
M M N
E M
Q
M
p
c b c
t t
sin
log 2
log
2
sin
log 2
log
2
0
2
2 0
2
2
c
E
1
0 0
1
Hamming codes
Hamming codes are a subclass of linear block codes
and belong to the category of perfect codes.
Hamming codes are expressed as a function of a
single integer .





The columns of the parity-check matrix, H, consist of
all non-zero binary m-tuples.


Hamming codes
2 > m
t
m n-k
m k
n
m
m
1 : capability correction Error
: bits parity of Number
1 2 : bits n informatio of Number
1 2 : length Code
=
=
=
=
Hamming codes
Example: Systematic Hamming code (7,4)


] [
1 0 1 1 1 0 0
1 1 0 1 0 1 0
1 1 1 0 0 0 1
3 3
T
P I H

=
(
(
(

=
] [
1 0 0 0 1 1 1
0 1 0 0 0 1 1
0 0 1 0 1 0 1
0 0 0 1 1 1 0
4 4
=
(
(
(
(

= I P G
Cyclic block codes
Cyclic codes are a subclass of linear
block codes.
Encoding and syndrome calculation are
easily performed using feedback shift-
registers.
Hence, relatively long block codes can be
implemented with a reasonable complexity.
BCH and Reed-Solomon codes are cyclic
codes.
Cyclic block codes
A linear (n,k) code is called a Cyclic code
if all cyclic shifts of a codeword are also a
codeword.



Example:

) ,..., , , , ,..., , (
) ,..., , , (
1 2 1 0 1 1
) (
1 2 1 0
+

=
=
i n n i n i n
i
n
u u u u u u u
u u u u
U
U
i cyclic shifts of U
U U U U U
U
= = = = =
=
) 1101 ( ) 1011 ( ) 0111 ( ) 1110 (
) 1101 (
) 4 ( ) 3 ( ) 2 ( ) 1 (
Cyclic block codes
Algebraic structure of Cyclic codes, implies expressing
codewords in polynomial form


Relationship between a codeword and its cyclic shifts:





Hence:
) 1 ( degree ... ) (
1
1
2
2 1 0
n- X u X u X u u X
n
n

+ + + + = U
) 1 ( ) (
...
..., ) (
1
) 1 (
) 1 (
1 1
) (
1
2
2
1 0 1
1
1
2
2
1 0
1
) 1 (
+ + =
+ + + + + + =
+ + + =

n
n
X u
n
n
n
X
n
n n
n
n
n
n
X u X
u X u X u X u X u u
X u X u X u X u X X
n
n
U
U
U

) 1 ( modulo ) ( ) (
) (
+ =
n i i
X X X X U U
By extension
) 1 ( modulo ) ( ) (
) 1 (
+ =
n
X X X X U U
Cyclic block codes
Basic properties of Cyclic codes:
Let C be a binary (n,k) linear cyclic code
1. Within the set of code polynomials in C, there
is a unique monic polynomial with
minimal degree is called the
generator polynomials.

2. Every code polynomial in C, can be
expressed uniquely as
3. The generator polynomial is a factor of



) (X g
) ( . X n r g <
r
r
X g X g g X + + + = ... ) (
1 0
g
) (X U
) ( ) ( ) ( X X X g m U =
) (X g
Cyclic block codes
4. The orthogonality of G and H in polynomial
form is expressed as . This
means is also a factor of

5. The row , of generator matrix is
formed by the coefficients of the cyclic
shift of the generator polynomial.



(
(
(
(
(
(

=
(
(
(
(

r
r
r
r
k
g g g
g g g
g g g
g g g
X X
X X
X

1 0
1 0
1 0
1 0
1
) (
) (
) (
0
0
g
g
g
G
1 ) ( ) ( + =
n
X X X h g
1 +
n
X ) (X h
k i i ,..., 1 , =
" 1 " i
Cyclic block codes
Systematic encoding algorithm for an
(n,k) Cyclic code:
1. Multiply the message polynomial by

2. Divide the result of Step 1 by the generator
polynomial . Let be the reminder.

3. Add to to form the codeword
) (X m
k n
X

) (X g ) (X p
) (X p ) (X X
k n
m

) (X U
Cyclic block codes
Example: For the systematic (7,4) Cyclic code
with generator polynomial
1. Find the codeword for the message





) 1 1 0 1 0 0 1 (
1 ) ( ) ( ) (
: polynomial codeword the Form
1 ) 1 ( ) 1 (
: ( by ) ( Divide
) 1 ( ) ( ) (
1 ) ( ) 1011 (
3 , 4 , 7
bits message bits parity
6 5 3 3
) ( remainder generator
3
quotient
3 2 6 5 3
6 5 3 3 2 3 3
3 2


=
+ + + = + =
+ + + + + + = + +
+ + = + + = =
+ + = =
= = =

U
m p U
g m
m m
m m
p g q
X X X X X X X
X X X X X X X X
X) X X
X X X X X X X X X X
X X X
k n k n
X (X) (X)
k n
k n
) 1011 ( = m
3
1 ) ( X X X + + = g
Cyclic block codes
2. Find the generator and parity check matrices, G and H,
respectively.
(
(
(
(

=
= + + + =
1 0 1 1 0 0 0
0 1 0 1 1 0 0
0 0 1 0 1 1 0
0 0 0 1 0 1 1
) 1101 ( ) , , , ( 1 0 1 1 ) (
3 2 1 0
3 2
G
g g g g g X X X X
Not in systematic form.
We do the following:


row(4) row(4) row(2) row(1)
row(3) row(3) row(1)
+ +
+
(
(
(
(

=
1 0 0 0 1 0 1
0 1 0 0 1 1 1
0 0 1 0 1 1 0
0 0 0 1 0 1 1
G
(
(
(

=
1 1 1 0 1 0 0
0 1 1 1 0 1 0
1 1 0 1 0 0 1
H
4 4
I
3 3
I
T
P
P
Cyclic block codes
Syndrome decoding for Cyclic codes:
Received codeword in polynomial form is given by


The syndrome is the reminder obtained by dividing the
received polynomial by the generator polynomial.


With syndrome and Standard array, error is estimated.

In Cyclic codes, the size of standard array is considerably
reduced.
) ( ) ( ) ( X X X e U r + = Received
codeword
Error
pattern
) ( ) ( ) ( ) ( X X X X S g q r + =
Syndrome
Example of the block codes
8PSK
QPSK
[dB] /
0
N E
b

B
P