School of Elec. And Telecom.

Hanoi University of Science and Technology

Advanced Multimedia
Video Coding
Dzung Tien Nguyen, MSc. Tien Pham Van, Dr. School of Electronics and Telecommunications, Hanoi University of Science and Technology
Advanced Multimedia course 1

School of Elec. And Telecom. Hanoi University of Science and Technology

Video Format and Quality

Advanced Multimedia course

Video Capture (1)

School of Elec. And Telecom. Hanoi University of Science and Technology

Spatial & Temporal Sampling Spatial:

How affect the visual quality?

Defines a set of valued sampled points Occurs at intersection points on rectangular grid Periodically

Temporal:

Advanced Multimedia course

Video Capture (2)

School of Elec. And Telecom. Hanoi University of Science and Technology

Frames & Fields Frames: Progressive sampling Fields: Interlaced sampling Slices: a distinct region of frame, encoded seperately

Advanced Multimedia course

Sampling Formats
School of Elec. And Telecom. Hanoi University of Science and Technology

YCbCr:
4:4:4 4:2:2 4:2:0

4:4:4

4:2:2

4:2:0

Example: Image resolution: 720x576. Calculate the total bits to represent a frame in each sampling format (Y:8 bits)
Advanced Multimedia course 5

Video frame formats

School of Elec. And Telecom. Hanoi University of Science and Technology

Video frame formats

ITU-R BT.601-5 parameters

Advanced Multimedia course 6

Quality Measurement
School of Elec. And Telecom. Hanoi University of Science and Technology

Subjective Quality Measurement

Depends on the complex interaction between HVS, eye and brain PSNR

Objective Quality Measurement

(2n 1) 2 PSNRdB 10log10 MSE

Advanced Multimedia course

School of Elec. And Telecom. Hanoi University of Science and Technology

Video Coding Concepts

Advanced Multimedia course

Why Video coding?

School of Elec. And Telecom. Hanoi University of Science and Technology

Multimedia: is for storing, transmitting, playing etc. Video without compression

720 x 480 pixels per frame 30 frames per second Total 90 minutes Full color The full data quantity = 167.96 G bytes!!

Advanced Multimedia course

Desired Features
School of Elec. And Telecom. Hanoi University of Science and Technology

Better compression
Improved quality Interactivity and Manipulation of Content Error Resilience

Processing of content in the compressed domain

Identification and selective coding/decoding of the object of interest

Facilitate Search / Indexing (MPEG-7)

Time Table
School of Elec. And Telecom. Hanoi University of Science and Technology

VC-1/VC-2 H.26L H.263 H.261 MPEG4 H.264

MPEG2/H.262
MPEG1 JPEG
Year 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2010

Advanced Multimedia course

11

Evolution of Video Compression Standards

School of Elec. And Telecom. Hanoi University of Science and Technology

H.261 Video Telephony H.262/MPEG-2 Digital TV/DVD H.263 Video Conferencing

MPEG-1 Video-CD

MPEG-4 Visual Object-based Coding

H.264 MPEG-4 AVC

Advanced Multimedia course

12

Performance of codecs
School of Elec. And Telecom. Hanoi University of Science and Technology

Where used?
School of Elec. And Telecom. Hanoi University of Science and Technology

MPEG-1
Video-CD Usually .mpg or .mpeg files are MPEG-1 DAB Digital Radio is MP2 (MPEG-1 Layer 2) MP3 files (MPEG-1 Layer 3)

MPEG-2:
.vob, .m2v, rarely .mpg files Anything to do with DVD
Camcorders, DVD players, DVD recorders

Digital TV (DVB)

MPEG-4:
High Quality AVI files Video Phones DivX Some advanced audio players support MPEG-4 Advanced Audio Coding (AAC)

Where used?
School of Elec. And Telecom. Hanoi University of Science and Technology

H.263/+/++
NetMeeting and similar video-chat Network streaming application, video phone H.264
Video Conferencing: over different networks Multimedia Streaming: live and on-demand: Youtube, iTunes, Adobe Flash Player (web), etc. Multimedia Messaging Services (MMS) Blu-ray, Digital Video Broadcasting, iPod Video, HD DVD

VC-1, VC-2
Video on Internet, HDTV broadcast, UHDTV

Lossy and Lossless compression

School of Elec. And Telecom. Hanoi University of Science and Technology

Lossy: discard some of data.

Name some lossy compression techniques?

Lossless: remove statistical redundancy

Name lossless compression technique?

Tradeoff?

Advanced Multimedia course

16

Video and Image compression

School of Elec. And Telecom. Hanoi University of Science and Technology

What is the difference between video compression and image compression?

Temporal Redundancy

Coding method to remove redundancy:

Intraframe Coding
Remove spatial redundancy

Interframe Coding
Remove temporal redundancy

Advanced Multimedia course

17

Compression Techniques
School of Elec. And Telecom. Hanoi University of Science and Technology

Theorem
Spatial redundancy
DCT, Subband, wavelet

Temporal redundancy
MC/ME

Statistical redundancy
RLC, VLC

Perceptual redundancy
SQ, VQ, fractal

Advanced Multimedia course

18

Video Coding Concepts

School of Elec. And Telecom. Hanoi University of Science and Technology

Video CODEC Temporal Model Image Model Entropy Coder

Advanced Multimedia course

19

Video CODEC = COder+ DECoder

School of Elec. And Telecom. Hanoi University of Science and Technology

CO

Prediction model: to estimate the predictor of current frame, then calculate the residual. Output: Residual and Prediction parameters Spatial model: to reduce spatial redundancy from residual frame and quantize samples. Output: coefficients Entropy encoder: to encode coefficients together with prediction parameters and header information. Output: Coded video

DEC? Draw a functional block diagram of a DECoder

Advanced Multimedia course 20

10

Temporal Model
School of Elec. And Telecom. Hanoi University of Science and Technology

Motion prediction Changes due to previous frame Block-based motion estimation and compensation
Previous frame as predictor

Frame 1

+
Frame 2
Advanced Multimedia course

=
Residual frame
21

Temporal Model
School of Elec. And Telecom. Hanoi University of Science and Technology

Macro-block and motion estimation

Motion estimation
Advanced Multimedia course 22

11

Image model
School of Elec. And Telecom. Hanoi University of Science and Technology

Prediction Transformation Quantization Reordering

Spatial prediction

Scan order

Quantization
Advanced Multimedia course

4x4 DCT pattern

23

Entropy Coder
School of Elec. And Telecom. Hanoi University of Science and Technology

To put residual frame, motion vector and header to a compressed bitstream To reduce the amount of bits representing data

Advanced Multimedia course

24

12

Transform Coding
School of Elec. And Telecom. Hanoi University of Science and Technology

Advanced Multimedia course

25

DCT
School of Elec. And Telecom. Hanoi University of Science and Technology

Use the technique of the JPEG

DCT based coding scheme
DCT transform (2D)

13

Discrete cosine transform

School of Elec. And Telecom. Hanoi University of Science and Technology

Use the technique of the JPEG

Discrete cosine transform

DCT Transformation
School of Elec. And Telecom. Hanoi University of Science and Technology

28

14

DCT of Correlated and Uncorrelated Image

School of Elec. And Telecom. Hanoi University of Science and Technology

Advanced Multimedia course

29

LAB
School of Elec. And Telecom. Hanoi University of Science and Technology

Apply DCT to an array of double Apply DCT to an gray-scale image, observe and comment

Advanced Multimedia course

30

15