IPTV QoS

Monitoring

Fast Forward Your Development

www.dsp-ip.com

Overview
QoS and QoE
Measurement and Monitoring Goals
Lab, pre-deployment, Monitoring, failure analysis

Measurement Methods
Objective, Subjective, payload/codec based

Monitoring Levels
Transport, Transaction, content

Monitoring Domains
Head-end, network, Home

Fast Forward Your Development

www.dsp-ip.com

Challenges in providing QoE

Name

Challenge

Answer

network

Overcome Packet loss, jitter,

network congestion

Improved protocols
QoS mapping schemes

Bandwidth

No bandwidth in the network and at

the home

increased capacity inside the

home and the network level

Monitoring

Measure hundreds of video streams

in Gb Eth & IP environment

Video over IP monitoring,

new protocols & devices

CRM costs Lower MTBF, call center calls and

truck rolls

Fast Forward Your Development

Automatic maintenance,
Automatic firmware update
Home network monitoring

www.dsp-ip.com

User quality measurements

What does subscribers want?
Content
Technical
Factors

Content quality
Content pricing
Content availability

Viewing experience
Video Quality
Channel Zapping delay

Integration of services
Usability
One bill
New services

Customer services

Fast Forward Your Development

www.dsp-ip.com

Voice/Video Quality?
We can look at quality in different ways:
1. Perceptual subjective Quality (MOS)
2. Numerical Quality parameters (PSNR)
3. How to connect (relation matrix between 1
and 2)

Fast Forward Your Development

www.dsp-ip.com

MEASUREMENT &
MONITORING
In the Lab & In The Fields
For Pre-Deployment/monitoring/Failure Analysis

Fast Forward Your Development

www.dsp-ip.com

Measurement & Monitoring phases

Design

Lab
Testing

Deployment
Phase
Pre
Deployment
Testing

Pre-Deployment
Phase

Analysis
Problem Solving
Tuning

24/7
Monitoring

Fast Forward Your Development

www.dsp-ip.com

Measurement & Monitoring phases

Design & Lab testing
Simulation and Emulation of the network
Lab and testing tools

Pre Deployment Stage

Work on actual network
Load testing
Lab, testing, diagnosing and monitoring tools

Deployment (production) Phase

Mostly monitoring (probes) equipment, management
systems, data filtering and diagnostics equipment

Fast Forward Your Development

www.dsp-ip.com

MEASUREMENT TAXONOMY
Subjective
Objective
Payload based, codec aware, codec anaware

Fast Forward Your Development

www.dsp-ip.com

Measurement methods review

Subjective
Accurate
Expensive, not for monitoring

Objective
Repeatable
For both testing and monitoring

Fast Forward Your Development

www.dsp-ip.com

Multimedia monitoring methods

HSI and
Data World

Broadcast
World
Subjective

MOS
(Voice)

BT500
(Video)

Objective
Codec aware
Packet based

VQS
Telchemy
V-Factor

VQI
Testing

Fast Forward Your Development

Network
Monitoring
Delay, Jitter
Packet loss
Codec independent
Packet based

MDI
Monitoring

www.dsp-ip.com

Objective methods

Fast Forward Your Development

www.dsp-ip.com

Payload Based Methods

Fast Forward Your Development

www.dsp-ip.com

Full Reference: Video Quality Assessment

ITU-T J.144 and ITU-R BT.1683

Full-reference perceptual models

Digital TV
Rec. 601 image resolution (PAL/NTSC)
Bit rates: 768 kbps ~ 5 Mbps
Compression errors

Fast Forward Your Development

www.dsp-ip.com

Voice Quality Assessment with/out reference

ITU-T P.862 (Feb 2001) - Full Reference

Full-reference perceptual model (PESQ)

Signal-based measurement
Narrow-band telephony and speech codecs
P.862.1 provides output mapping for prediction on
MOS scale

ITU-T P.563 (May 2004)

No-reference perceptual model
Signal-based measurement
Narrow-band telephony applications

Fast Forward Your Development

www.dsp-ip.com

Voice Quality Assessment

ITU-T P.862.2 (Nov 2005):
Extension of ITU-T P.862
Wide-band telephony and speech codecs (5 ~7Khz)
ITU-T P.VTQ (on-going):
Targeted at VoIP applications
Minimum performance framework for no-reference
packet-based measurement
Models analyze packet statistics; speech payload is
assumed
Uses P.862 as a measurement reference

Fast Forward Your Development

www.dsp-ip.com

Codec Aware Methods

Codec aware
Packet based

VQS
Telchemy
V-Factor

VQI

Fast Forward Your Development

www.dsp-ip.com

Packet Codec Aware

Monitoring technique
Codec dependent
Incorporates network parameters data with
codec behavior data
Scales- could monitor thousands of channels
Examples:
The need a codec aware metrics
VQS (Telchemy)
VQI(Brix)
V-Factor (QoSMetrics)

Robust
codec
Problem area

Raw
codec

Fast Forward Your Development

www.dsp-ip.com

Packet Codec aware

Packet loss/discard typically
occurs in high density periods

Base quality level

depends on frame rate,
codec type, bit rate

Average can
be
misleading
Poor quality
during burst of
loss/discards

Impact of Burst
of
Packet Loss
5-8
seconds

Fast Forward Your Development

15-30
seconds

Subjective
compensation for
variance between
human and testing
equipment view of
loss

www.dsp-ip.com

Example V-Factor
Based on MPQM (Moving Picture Quality
Metrics) high quality video measurement
standard
V = f(QER, PLR, R)
QER relative video codec quality
PLR Packet loss ratio (based on actual packet
loss, jitter data and jitter buffer model)
R Image complexity factor (2-3)

Adopted by Spirnet

Fast Forward Your Development

www.dsp-ip.com

Packet Codec Independent

Monitoring only
Codec independent
Based on network parameters data only
Scales - could monitor thousands of channels
Examples:
MDI
IneoQuest
standardized by IETF

Fast Forward Your Development

www.dsp-ip.com

MONITORING LEVELS

Fast Forward Your Development

www.dsp-ip.com

Measurement Levels
Transport Level
Service (transaction) Level
Media Quality Level
Add Your Text

Add Your Text

Add Your Text

Add Your Text

Fast Forward Your Development

Content Quality

Video Stream Quality

Transport Quality

Transaction Quality

www.dsp-ip.com

Transport Level
Example:

Packet Loss
Loss Patterns
Jitter
Delay

Well understood
Defined by ITU and IETF

Fast Forward Your Development

www.dsp-ip.com

Transaction Level
Examples
Post-dial delay in PSTN/mobile networks
Video start time

Requires understanding in both network

monitoring and signaling (IGMP, SIP) and in
media coding (analysis of the media to discover
dial tone or I frame)

Fast Forward Your Development

www.dsp-ip.com

Channel zapping delay

Multicast saves bandwidth but creates
signaling delays:

Multicast Leave +
Multicast Join +
First I Frame +
Up to 2 seconds buffering time

First frame
viewed

Total Channel zapping Latency

Leave latency

First I
Frame

Buffering
latency

Join latency
Signaling Latency

Fast Forward Your Development

Media
Latency

www.dsp-ip.com

Microsoft improved channel zapping

Unicast server is used to serve the stream till the
second multicast stream has an I Frame
Immediate I Frame
Shorter buffering time
I Frame unicast

First frame
viewed

I Frame Multicast

Total Channel zapping Latency

Leave latency

unicast stream
Join latency
Multicast stream

Shorter buffering time

Fast Forward Your Development

www.dsp-ip.com

VoD Signaling
Unicast server is used to serve the stream till the
second multicast stream has an I Frame
Immediate I Frame
Shorter buffering time
First frame
viewed
Total VoD Signaling Latency
Play to server

Buffering
latency

Fast Forward Your Development

www.dsp-ip.com

Content Level
Content quality is a payload based measurement.
Requires decoding of the video stream
Understanding of the buffering and error concealment algorithms
of the decoder

CPU intensive Does not scale

Accurate
Used mostly is Lab equipment and diagnostic equipment
Examples:
PSNR
ITU-T J.144

Usually requires the reference (original) stream

Tests:
Source artifacts
Source quality
Fast Forward Your Development

www.dsp-ip.com

Standardization landscape

ATIS

ITU

IIF IPTV
Interoperability
Forum

Study Group 12

QoS Metrics
Standardization

Algorithms for
end-to-end
transmission
performance

Used for
Monitoring

Fast Forward Your Development

ITU
VQEG Video
Quality Expert
Group
Video performance
measurement based
on Subjective tests
Database

DSL
DSL Forum
TR-64, TR-69
LAN and WAN
monitoring
standards

Used in
Diagnostics /
Lab
www.dsp-ip.com

Example: ATIS IIF Quality Metrics

VSTQ - Video Service Transmission Quality
Transmission Quality - codec/ content independent
Based on the rate and distribution of effective packet loss and discard

VSPQ - Video Service Picture Quality

Estimated viewing quality
Considers the impact of VSTQ, video codec type and rate, resolution

VSAQ - Video Service Audio Quality

audio listening quality
Considers the impact of VSTQ, audio codec type, sample rate, ..

VSMQ - Video Service Multimedia Quality

overall user experience
Combined effect of VSPQ, VSAQ, audio-video synchronization..

VSCQ - Video Service Control (Plane) Quality

Considers responsiveness and reliability of control plane (trick play)
Fast Forward Your Development

www.dsp-ip.com

Monitoring levels
J.144 and PSNR examines the video content only (payload
measurements)
TR101290 examines only transport stream data and coherence without
examining the video content
V-Factor and VQS looks at packet loss, jitter and loss patterns data and
incorporate it with codec information and video header information
MDI Examines only packet loss and packet loss patterns without
considering the codec or video information

MDI

V-Factor, VQS

TR101290
MPEG2TS
Headers
J.144, PSNR
Video
payload

Fast Forward Your Development

www.dsp-ip.com

MEASUREMENT DOMAINS
Headend
Network
VoD
Home

Fast Forward Your Development

www.dsp-ip.com

DVB TR 101290
DVB Transport stream monitoring standard
Alerts and KPI:
TS sync loss
CRC error
Sync byte error
PCR error
PAT error
PCR
Continuity counter errors
discontinuity
PMT error
PCR accuracy
PID error
error
PTS error
Transport error

Fast Forward Your Development

www.dsp-ip.com

Network measurement points

Head-end

VoD Servers
Encoders
TS muxers

Network core
Routers

Network Probe interface

GBE, OC3/12, STM1/4

Raw Video input

MPEG2 TS - DVB ASI, DVBoIP

Digital and analog video

Network edge

ATM switches
GBE Switches

Access

DSLAM / BRAS

CPE

STBs
Home network

Source: Brix-Video Quality Measurement Algorithms WP

Fast Forward Your Development

www.dsp-ip.com

Headend
Video source quality
Video encoding quality
MPEG2 Transport stream parameters
TR100290
1st-3rd level priority alerts

Inputs

Raw analog video in a variety of forms (SD/HD)

DVB-ASI
DVBoIP
DVBoATM (AAL1, AAL5)

Fast Forward Your Development

www.dsp-ip.com

VoD
Pre-deployment
RTSP signaling latency (delay)
Performance under stress (user load)
Trick Mode (FF, REW)
Signaling delay
Video quality
Buffer state (overflow and underflow)

Monitoring
Loss of data (under stress)

Interfaces:
GBE
10GBE

Fast Forward Your Development

www.dsp-ip.com

Home network monitoring challenges

Scalability Millions of STBs
Cost Monitoring hardware and/or software costs
must not add to STB costs (multiply by number of
STBs)
Data Large amounts of data which needs RT
processing, filtering, logging and analysis

Unclear network structure

Non standard equipment
Installation, support and debugging problems

Fast Forward Your Development

www.dsp-ip.com

Home network monitoring roots

Home network monitoring is currently a minor
feature in IPTV Support Automation
Support Automation handles:

Installation
Preventive maintenance
STB and CPE remote management (TR69 and more)
Service verification for field technician
CRM support
Interaction logging
Home Network monitoring and repair

Fast Forward Your Development

www.dsp-ip.com

CPE network management

2Wire solution -Three data views
OSS/BSS
Customer care
Data Analysis

Other companies in this field

SupportSoft
Enure
Motive

Fast Forward Your Development

www.dsp-ip.com

Low level network monitoring

Fast Forward Your Development

www.dsp-ip.com

Vendors specialization
Lab Testing

Monitoring

Incumbent

Telecom
Agilent
Ixia
Spirnet

Telecom
Agilent
JDSU/
Acterna
Spirnet

New
entrants

Telecom
Shenick

Cable
Rhode &
Schwartz

Fast Forward Your Development

Cable
Thomson
Snell&Wilcox

Telecom
Telchemy
Bridge Technologies
iBrix
QoSMetrix
www.dsp-ip.com

DSP-IP Contact information

For Course materials & lecture request contact:
Adi Yakov
Training and Courses Manager

adi@dsp-ip.com
+972-9-8651933

For projects development services contact:

Alona Ashkenazi
Technology Management Services :
alona@dsp-ip.com
+972-9-8850956
www.dsp-ip.com
Mail : info@dsp-ip.com
Phone: +972-9-8850956,
Fax : +972-50- 8962910

Fast Forward Your Development

www.dsp-ip.com