Image Compression

Lecture 7
Sankalp Kallakuri
elsanky@gmail.com
 Fundamentals

• Data compression can be achieved if there is

redundancy in data.
1
Relative Data Redundancy RD  1
CR
n1
Compression Ratio CR 
n2

There are 3 kinds of redundancies used in image processing

. Coding
. Interpixel
. Psychovisual
 Coding Redundancy
• Grey levels can be coded in order to compress the
image.
nk
pr (rk )  k  0,1,2....., L  1
n
L 1
Lavg   l (rk ) pr (rk )
k 0

MxN image will need MNLavg number of pixels which may be less
than the Number of pixels decided by the number of grey levels as per
an m bit binary code.

Variable length codes assign shorter codewords to more frequent

symbols and longer codewords to less frequent symbols hence reducing
the average number of bits required.
 Example Variable Length Code
Text sequence

CEACDABABCEABADACADABABADEACBADABCADAEE

Table for frequency of occurrence of each symbol

Frequency to
Symbol
appear
A 15
B 7
C 6
D 6
E 5
 Example Variable Length Code
Bit requirement for fixed length code

Symbol Frequency to appear Bit Pattern Total bits

A 15 000 15*3=45

B 7 001 7*3=21

C 6 010 6*3=18

D 6 011 6*3=18

E 5 100 5*3=15

Total 117bits
 Example Variable Length Code
Huffman code
Symbol Frequency to appear Total bits
code length
A 15 0 1 15*1=15
B 7 100 3 7*3=21
C 6 101 3 6*3=18
D 6 110 3 6*3=18
E 5 111 3 5*3=15
Total
87bits

0110110 Coding redundancy is always present for a standard

M bit code
A
D
D
 Interpixel Redundancy

• Based on the ability to predict the value of a pixel based

on its neighborhood.
• Codes based on histograms or grey level choices do not
capture spatial redundancy brought about by the
structure of the image.

The two images shall have the same histogram but different
autocorrelation functions.
 Interpixel Dependency

• Run length coding is useful when there are large

stretches of similar spatial data.
Color/ grey level Thresholded to B&W

1 2 3 4 5

Run length representation 255(60),0(5),255(70),0(5),255(20)

 Psychovisual Redundancy

• The psychovisually redundant information isn’t needed

hence can be removed.
• Example : reducing the spatial or the grey scale
resolution of the image that has to be quantized.

Used to generate The last bit plane

random numbers gets randomised

Improved Grey Scale Quantization

 Fidelity Criteria
• Data loss due to psychovisual redundancy coding may need to be
checked.Fidelity criteria are a measure for such loss.

• Two kinds of fidelity criteria

1) subjective and 2) objective

• Objective Criteria
1) Root Mean Square Error
2) Mean Square Signal to Noise Ratio

• Subjective
{-3,-2,-1,0,1,2,3}
{much worse, worse, slightly worse, the same, slightly
better, better, much better}
{excellent, fine, passable, marginal, inferior, unusable}
 Image Compression Models
f(x,y) F(x,y)
Source Channel Channel Channel Source
Encoder Encoder Decoder Decoder

Source encoder
f(x,y) channel
Mapper Quantizer Symbol
encoder

Source decoder

channel Symbol Inverse F(x,y)

Decoder Mapper

From gonzalez and woods pg 422

 Channel Encoder and Decoder

• The channel encoders and decoders usually have parity or

checksum like corrections.
• Hamming(7,4) 3 code bits 4 data bits
Distance between any valid code words is 3
Hence any single bit error can be detected and corrected.
Parity bits Data Bits
h1  b3  b2  b0 h3  b3
h2  b3  b1  b0 h5  b2
h3  b2  b1  b0 h6  b1
Error Checking
c1  h1  h3  h5  h7
h7  b0
c2  h2  h3  h6  h7
c4  h4  h5  h6  h7
 Information Theory

• The information is modeled as a random process.

I ( E )   log P( E )

I – Information P - probability

A is the alphabet of the source all symbols belong to A.

J

 P(a )  1
j 1
j

Entropy is the quantity defining the average information produced by

J sources.
J
H ( z )   P(a j ) log P(a j )
j 1
 Information Channel

Let B be the channel alphabet with k possible symbols

J
P(bk )   P(b k | a j )P(a j )
j 1

Channel matrix

 P(b1 | a1 ) P(b2 | a1 ) … P(bk | a1 ) 

 P(b | a ) 
… … …. 
Q 1 2
 …. …. …. …. 
 …. …. P(b | a )
 1 k
P (b | a ) k k 

v=Qz is the distribution of the output alphabet .

 Skipped topics

• Shannons first theorem for noiseless

coding
• Shannons second theorem for noisy
coding.
• Source Coding Theorem
 Example: Information Theoryfrom Gonzalez and
Woods

21 21 21 95 169 243 243 243

21 21 21 95 169 243 243 243
21 21 21 95 169 243 243 243
21 21 21 95 169 243 243 243

Image with 8 bits per pixel . One among 28x4x8 possible output images.

Grey level Count Probability

21 12 3/8
95 4 1/8
169 4 1/8
243 12 3/8
First order estimate of the entropy. For given mage 1.81bits/pixel. 58 bits total.
 Example
Gray-level pair Count Probability
(21,21) 8 1/4
(21,95) 4 1/8
(95,169) 4 1/8
(169,243) 4 1/8
(243,243) 8 1/4
(243,21) 4 1/8

Estimate of the grey level pairs. Second order estimate.

The entropy turns out to be 1.25 bits/pixel.
 Error Free Compression

• Usually consists of two steps

. devising an alternate representation where the redundancies are
reduced.
. coding the representation to eliminate redundancies.

Variable Length Coding

. Assigns the shortest code word to the most probable symbol.
. The inputs maybe images, run lengths, pixel differences etc.

Huffman Code
.The first step involves ordering all the symbols in the probability of
their occurrence.
.The two symbols with the lowest probabilities are merged to one
composite symbol.
. The step is repeated until there are only two symbols left.
 Huffman Coding
Original Source Source Reduction
Symbol Probability 1 2 3 4
a2 0.4 0.4 0.4 0.4 0.6
a6 0.3 0.1 0.3 0.3 0.4
a1 0.1 0.1 0.2 0.3
a4 0.1 0.1 0.1
a3 0.06 0.1
a5 0.04

The next step is to map the least frequent composite symbols

at each step to codes of successively larger word length.
Symbol Probabilit Code 1 2 3 4
y
a2 0.4 1 0.4 1 0.4 1 0.4 1 0.6 0

a6 0.3 00 0.1 00 0.3 00 0.3 00 0.4 1

a1 0.1 011 0.1 011 0.2 010 0.3 01
a4 0.1 0100 0.1 0100 0.1 011
a3 0.06 01010 0.1 0101
a5 0.04 01011
 Huffman Code

• Block code : as each source block is mapped to a fixed

sequence of symbols.

• Instantaneously decodable: the sequence can be

instantaneously decoded by reading in a left to right
fashion and using a look up table.

• Uniquely decodable: the sequences of code map to only

one valid original symbol.
 Other near optimal VLC

• Truncated Huffman
• B2-Code
• Shift Code
• Binary Shift
• Huffman shift