Cisco QoS

QoS
Mechanisms
NSA/spicard 2001, Cisco Systems, Inc. 1

What Is QoS ?
Acronym for Quality of Service

For certain traffic
Controlled latency
Dedicated bandwidth
Improved loss characteristics
QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 2

Who Cares ?
Network telephony
Interactive applications
Transaction applications

IntServ or DiffServ
Integrated Services (RFC1633)

Request for resources per flow with
a signalling protocol (i.e. RSVP)
Differentiated Services (RFC2475)
Manage available resources based
on a tag associated per flow (IP
prec or new DSCP)

Integrated Services IntServ
Applications signal their QoS requirements via RSVP

to the network
Every network node along the path must check to
see if the reservation request can be met
Resources are reserved if the service constraints can
be met
An Error message is sent back to receiver if the
constraints cannot be met
Network nodes make sure there are no violations of
the traffic contract
Non conforming packets are either marked down or
dropped
Differentiated Services DiffServ
Technique for providing QoS in TCP/IP

No need for per hop signaling and flow state
maintenance as required by RSVP
Each network device classifies, polices and
schedules packets in a flow
Uses the Type of Service (ToS) byte in the IP header
to identify or set the priority level
6 most significant bits of the ToS byte are called
DSCP (DiffServ Code Point)
3 of these DSCP bits identify the IP Precedence

IntServ or DiffServ
Integrated Services
Network needs to maintain each
reservation
all or nothing mechanism
real time traffic oriented
Differentiated Services
Define limited flow classes
more scalable, but provisioning
DiffServ
Flow Classification and policing

Classifier
Conditioner
Forwarding
Queuing (Per Hop Behaviour)

Cisco QoS components
Classifier (ACL, NBAR,CB-marking)

Conditioner
Policy based Routing (PBR)

Committed Access Rate (CAR, CB policing)
Traffic Shaping (GTS, FTRS, CB-shaper)

Queuing/Scheduling
Congestion management(PQ, CQ, WFQ, CBWFQ, LLQ)
Congestion avoidance (wRED)

Fragmentation and Interleaving (MLPPP, FRF11/12)
Policy Based Routing
interface Ethernet0/0
ip policy route-map lab
access-list 101 permit tcp any host 10.22.1.10

route-map lab permit 10
match ip address 101
set ip precedence 4

Network-based Application
Recognition (NBAR)
Classification based on protocol type, L4 ports,

application requests (url, sqlnet )
Additional Protocols Easily Added
NBAR engine is driven by Packet Description
Language (PDL)
No changes required to the engine
PDL files will be read from flash - no IOS upgrade

required to adopt new protocols

Network-based Application
Recognition (NBAR)
A New IP Packet Classifier Capable of

Matching on
Dynamically-assigned TCP/UDP port numbers
URL matching using regular expressions

Sub-port criteria such as transaction types

Committed Access Rate (CAR)
Two functions
Packet ClassificationIP precedence
and QoS group setting
Access Bandwidth Management
through rate limiting

Committed Access Rate (CAR)
Rate Limiting (Policing)

Packet Classification
Similar to Traffic Shaping, but no
Buffering
Matching specification: Confirm Action:

1) All traffic Drop, Transmit, Recolor
2) IP precedence Exceed Action:
3) MAC address Drop, Transmit, Recolor
4) IP access list
Traffic Traffic
Action
Matching Measurement
Policy
Specification Instrumentation

CAR Policy Examples
Recolor
Drop
Drop Multimedia
Recolor
Mission-Critical
Per Application CAR

Traffic Shaping
Internet Service
Provider (ISP)
128 Kbps
Cloud
T1
Branch Office Central

Bottleneck Site
Reduces outbound traffic flow to avoid

congestion (via buffering)
Eliminates bottlenecks in topologies with
data rate mismatch
Provides mechanism to partition interfaces to
match far-end requirements
Traffic Shaping
Traffic Transmit Output

Destined Queue Line
for Interface Match
Classify
No Match
Configured
Queuing (e.g.
WFQ, CQ, etc.)
Classification by: Leaky

Extended Access Bucket
List Functionality Shaping
(GTS only)

Shaping Q vs Output Q
Shaping Queue
FIFO, LLQ
SubIntf 1.1 CIR=48 kbps Output

Tx Q ring
TS
Always FIFO
Main Intf 1
TS
SubIntf 1.2 Output
CIR=48 kbps Interleaving
Dual fifo if
Output FRF12
Shaping Queue
Interface
FIFO, LLQ FIFO, WFQ

Traffic shaping packet path
Packet path:
When a packet arrives at the interface for transmission, the following happens:
if the SHAPING queue is empty, the arriving packet is processed by the traffic
shaper.
1. If possible, the traffic shaper sends the packet to the OUTPUT queue.
(Means if number of bits allowed during Tc is not reached)
2. Otherwise, the packet is placed in the SHAPING queue and sent in next Tc.
If the SHAPING queue is not empty, the packet is placed in the shaping queue.
When there are packets in the SHAPING queue, the traffic shaper removes the number of
packets it can transmit from the SHAPING queue every time interval.

Token/leaky bucket
B - Burst size max in bits

= bucket size
R R - Token arrival rate.
In practice: add
Overflow an increment of
Tokens
B tokens every Tc
excess or every time a
packet arrive
Packets Enough
token in YES: Conform
arriving
bucket ? (transmit at line rate)
NO: Exceed (drop, enqueue or use excess)

Token/leaky bucket
bucket is filled at defined rate with

tokens (at each Tc or elapsed time between
incoming pkts)
Incoming Packets take available tokens in
bucket
Packets can to take up to burst bits (excess
burst is just a 2nd bucket mechanism)
If no credits in bucket, packet gets dropped
(policer) or queued (shaper)
Burst must take into account max packets length and TCP window
size (slow start).
Policing vs. Traffic Shaping
Both ensure that traffic does not exceed a

(contracted) BW limit
Both limit BW but with different impact on traffic
Policing drops more often - more retransmits
Shaping adds variable delay (buffering)
Policer causes TCP Retransmits
Oscillation of Windows in TCP
Policer can be a Marker also

Policer on input or output interface; Shaper on output

interface
Shaper use regular Tc interval, policer use elapse time
between packet to credit bucket
Shaper smooth traffic, policer allows bursts
Shaper can adapt to Network congestion (BECN,
FECN)
Shaper create shaping queues (can be use as a
congestion mechanism in virtual intf like tunnel)

Policing
Traffic
Traffic
Traffic Rate Traffic Rate
Time Time
Shaping
Traffic
Traffic
Traffic Rate
Traffic Rate
Time Time

Queuing Algorithms
Congestion management algorithms

First In First Out
Priority Queuing
Custom Queuing
Weighted Fair Queuing (WFQ)
Class based WFQ (CB-WFQ) and LLQ

Queuing Algorithms
Congestion avoidance algorithms

Weighted Random Early Detection
(wRED)

tx-ring vs tx-queue
Forwarder
Interface Yes
Congested?
FIFO/PQ/CQ/WFQ (output hold-queue
interface hold compromise latency/
No buffering )
Tx-Queues
Transmission
Tx-ring Always FIFO Queue (tx-ring)
(high-end txq=2+ BW/512k)
(in particles or packets, platform dependant !) When intf queue doesnt use FIFO
CSCdk17210, CSCdm75220, CSCdk70536 TxQ is automatically reduced (based on intf BW)
Changed w/ Tx-ring-limit or in order to activate fancy queuing earlier
show vip tx-polling-high <interface> If configured too small (high level of interrupts)
<new-tx-polling-high>
If configured too large (high latency)
tx-ring vs tx-queue
Tx-ring is the driver queue (L2 queues)
Tx-queues used ONLY when tx-ring is full
(fancy queuing is activated L3 queues)
Tx-ring configurable (platform/adapter
dependant) high enough to accommodate
line rate, short enough to reduce latency
Tx-ring tuned automatically when fancy
queuing configured (CSCdk17210)
Tx-queue default 64 on low-end, default Bw/
4000 (min=2) on VIP (500ms for 250B)
First In First Out (FIFO)
Transmit
Queue
Output Line
Simplest Queuing Algorithm

packets leave in order of arrival
Fixed Queue Lengths (default 40)
Result in dropping from tail of queue
under load

FIFO
Order of Arrival completely determines the

bandwidth, promptness and buffer allocation
Does not provide protection against ill-
behaved sources
Bursty sources may cause high delay in
delivering some time sensitive control/
signaling messages

Priority Queuing (PQ)
Interface Hardware
Ethernet
Frame Relay
ATM
Serial Link
High Etc.
Traffic
Destined Medium Transmit Output
for Interface Queue Line
Classify
Normal
Low
Q Length Defined
by Q Limit Absolute Priority
Scheduling
Interface Buffer
Classification by: Resources
Protocol (IP, IPX, AppleTalk,
SNA, DecNet, Bridge, etc.)
Incoming Interface
(EO, SO, S1, etc.)

Priority Queuing
Forwarder
Priority Classification
(Access)
List
Low Medium High

Priority Priority Priority
Transmission
Queue

Priority Queuing
Four priority queues -

High, Medium, Normal, Low
Packets not classified by priority-list
mechanism fall into normal queue

Priority Queuing
Can cause traffic lockout if

configured incorrectly
FIFO within priority
Within priority, may still be unpredictable
Resource allocation
lower priority queues may starve

Custom Queuing (CQ)
Interface Hardware
Ethernet
Frame Relay
1/10 ATM
Serial Link
2/10 Etc.
Traffic
Destined 3/10
for Interface Transmit Output
2/10 Queue Line
Classify
3/10
Up to 16
Link
Q Length Utilization Weighted Round
Deferred by Ratio Robin Scheduling
(byte count min
Queue Limit default 1500, each packet
started is sent)
Classification by: Interface Allocate

Protocol (IP, IPX, AppleTalk, Buffer Proportion of
SNA, DecNet, Bridge, etc.) Resources Link Bandwidth
Incoming interface
(EO, SO, S1, etc.)

Custom Queuing
Forwarder
Priority
(Access)
Control Traffic List
Queue #0
Queue #1 Queue #9
Queue #2 Queue #10
Queue #3 Queue #11
Queue #4 Queue #12
Queue #5 Queue #13
Queue #6 Queue #14
Queue #7 Queue #15
Queue #8 Queue #16
Transmission Queue
Custom Queuing
Identify the traffic

Distribute into FIFO queue by class
Dequeue in some rotation
Remove some number of bytes at most
Move to next queue in round robin order

Priority vs. Custom Queuing
PQ CQ
4 Queues: High, Medium, 16 Queues
Normal, Low
Packets classified by protocol or
Packets classified by protocol interface
or interface
Weighted round robin
Absolute priority scheduling scheduling
Lower priority queues may WRED and RSVP not supported
starve
Guarantees BW per queue, not
delay

Q Classification:
Source address 1 1 Reserved queues
Dest address
...
Source port (RSVP and RTP Reserve)
Dest. Port 2 2 2
Tos bits (not prec!)
Weight: 3
IP Precedence ... IP Precedence 7
RSVP/RTP Reserve De-
W=32384/(prec+1) 4 4
queue
...
5 5
IP Precedence 0
...
6 6 6 (Best Effort/Hash queues)

8 fixed system queues:

LMI, ARP, PPP control, Routing protocol
2n dynamic queues (hashing)
IP Flow = 109-bit string = {Sadd, Dadd,
PID, Sport, Dport, 5-bit ToS} (not prec !)
16 -> 4096, default is 256 (11.2)
x RSVP queues 0 -> 1000 [def: 0]
system queues has fixed weight (1024)

dynamic queues use IP prec (w=32384/(prec
+1)) (since 12.0.5T)
non ip flow use default 32384
internal traffic (CDP) use 4096
RSVP use w=4*(max_rsvp_bw/flow_bw)

Output scheduling based on a TDM
mechanism.
Tail_time = tail_time(t-1) + packet_size*W
F(A1)=0+100
F(B1)=50+300 A1[100]
B1[300]
F(A2)=100+20 A2[20]
F(B2)=350+300 F(A3)=120+10 A3 [10]
B2[300]
t 100 70 60 50 0
Hence the resulting scheduling is:
B2 B1 A3 A2 A1

Output Dropping based on a threshold.

Congestive Discard Threshold (CDT)
Not related to precedence
run-downleft#sh queue s 0/1

Input queue: 0/75/0 (size/max/drops); Total output drops: 0
Queueing strategy: weighted fair
Output queue: 7/1000/64/0 (size/max total/threshold/drops)
Conversations 5/5/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)

Upon enqueue of a new packet,

IF the sum of all subqueues > Max-limit (default 1000),
THEN, if the tail time of the new packet is > current system tail time,
then the new packet is dropped
else the system tail-time packet is dropped and the new one is
enqueued
IF CDT(default 64) < the sum of all subqueues < Max-limit,
THEN, if the new packets tail time > current system tail time
then drop the new packet
else enqueue the new packet
Exception: a packet classified into an empty sub-queue is never dropped

Weighted Fair Queuing
Provides relative bandwidth

guarantees and dropping mechanism
Fair Queuing (FQ) allocates equal share
of bandwidth to each active queue
Weighted Fair Queuing (WFQ) allows
for unequal allocation of bandwidth
sub_Queue BW= BW*(TOS+1)/ (TOS+1)

WFQ
1 FTP - Tos 0 get 1/16*48K = 3 Kbps

1 FTP - Tos 0 1/16*48K = 3 Kbps
1 Telnet - Tos 4 5/16*48K = 15 Kbps
1 HTTP - Tos 2 3/16*48K = 9 Kbps
1 Voice - Tos 5 6/16*48K = 18 Kbps
______________

(Tos+1)= 16

PQ-WFQ (IP RTP Priority)
To prioritise Voice traffic (on FR, PPP)

Create a priority queue (weight=0) + BW limit
RTP only (range of UDP ports)
Reduce voice latency
Router Exhaustive
Voice 1 1 Queuing
(Highest)
Data (High, 2 2 V V
PQ
e.g. IP Prec 4) 4 3 2 1 1
WAN
Data (Medium, Circuit
3 3 3 3
e.g. IP Prec: 2) WFQ
4 4 4 4

Class-Based WFQ (CBWFQ)
Class queues
1 1 Max: 255 since 12.2(6) CSCdv49675
(256 including the default class-queue)
...
2 2 2
De-queue
3
Default class-queue
OR
5 5
Classify
...
WFQ System
6 6 6 (unclassified traffic)

CBWFQ
Forwarder
Per class class-default

(ACL)
Per flow
Class #1 (hash)
Class #2
Flow #3
Class #3
Flow #2
Class #64
Flow #1
Scheduling based on Finish Time

CBWFQ
256 Classes (255 + default)

Bandwidth x (min guarantee)
wRED within classes
default class : class-default

fair-queue & queue-limit
fair-queue & wred
bandwidth & queue-limit
bandwidth & wred

CBWFQ
By default sum of BWclasses <=75% BW (for
L2 overhead+ default class) can be change with
max-reserved bandwidth xx on main intf or
PVC (CSCdv06837)
FIFO queuing per class (default 64 pkts)
Hellos, keepalives, (non-TCP updates) use
PAK-flag
routing updates (TCP) use default queue !!
Output scheduling is still weighted
between classes:
w = f(1/class_BW ; 1/#hashed_queues)
PQ-CBWFQ or
Low Latency Queuing LLQ
Forwarder
Per class class-default

(ACL)
Per flow
Class #1 (hash)
Class #2
Flow #256
Class #3
Flow #2
Class #63
PQ Flow #1
Scheduling based on Finish Time

PQ-CBWFQ (LLQ)
Priority Classes
priority and max BW (not limited to RTP flows)
burst size (200ms of traffic by default)
Expedite Forwarding class in diffserv model
low latency queuing classes
Only 1 fifo PQ (even if several priority classes !!)
CAR like policed if congestion (exept LLQ in
FR is ALWAYS policed but drop if congested)

WFQ vs LLQ
WFQ LLQ
%BW min guarantee per
class
Shared BW, no guarantees Pclass, BW max allowed
Prec define weight, BW (CAR like limiting)
PQ-WFQ for RTP only BW define weight (not prec)

wRED per class (prec
oriented)

Random Early Detection (RED)
Packets Queue
Arriving
Queue
Pointer
Without RED when the queue fills up ALL packets that

arrive are droppedTail drop
With RED as oppose to doing a tail drop
the router monitors the average queue
size and using randomization choose connections to
notify that a congestion
is impending

RED algorythm
for each packet arrival

calculate the average queue size (avg)
if min_threshold > avg
queue arriving packet
else if min_threshold <= avg < max_threshold
calculate packet-drop probability (p)
with packet-drop probability p
drop the arriving packet
or
queue the arriving packet
else if max_threshold <= avg
drop arriving packet

RED
The average queue size is calculated

based on the previous average and the
current size of the queue
Avg = (old_avg * (1 - 1/2^n)) +
current_queue_size * 1/2^n)
n exponential-weight-constant keyword
P: drop probability
prob = mark_prob * (avg - min_th) / (max_th - min_th)
default mark_prob = 1/10

weighted RED
Two drop slopes are

shown; Up to eight
can be defined
Prec 0
Probability drop
of Packet profile
Discard Prec 1
Slope is
adjustable drop
profile
Queue Length
IP Prec 0 IP Prec 1
Max0/max 1
Min. Min.
Configure min and max threshold per IPprec

(or per DSCP drop pref)
wRED
(TCP behavior slow start)
Sends as much as credit allows

Starts credit small
Avoid overloading network queues
Increases credit exponentially
To gauge network capability

wRED
(TCP behavior)
If ACK acknowledges something

Update credit and send
If not, presume it indicates a lost packet
Send first unacknowledged message right
away
Halve current credit
Increase linearly to gauge network
throughput
TCP and tail drop
global synchronization
Queue
100%
Utilization
Time
Tail Drop
3 Traffic Flows Start Another Traffic Flow

at Different Times Starts at This Point

wRED
Dropping a message is a way of

telling the sender to slow down
Randomly drop (instead of tail drop)
avoid global synchronization
Weighted drop thresholds based on
IP Prec

wRED
(non TCP behavior)
wRED will be unable to force the application
to back off, however, wRED will drop its
packets in proportion of its sending rate
Flow RED can discard per flow for non-
adaptive flows (ie UDP), with Random-
detect flow (IOS non-vip only)
FRED maintain flow queues and drop based
on: avg*scaling_factor < Flow_queue_depth
Means that this flow takes too much ressources
(is considered as non-adaptive)
RED must be activate first!!
Others ...
Selective Packet Discard (SPD)

discard input packets based on
precedence (router protection)
QPPB flow classification based on IP
prec, set and carried in BGP
attribute (community-list).

Link Fragmentation and
Interleaving (LFI)
Traffic
Destined Transmit Output
for Interface Queue Line
IP Voice
Jumbogram
WFQ
Large Packet
Multlink PPP Fragmentation:
with LFI Fragment Size Based
Or Frame on Required Delay
Realy with
FRF 11/12
LFI
Fragment Large Size Packets on Slow

Links (<768kb/s)
i.e.1500 Byte takes 215 ms on a
56kbs link (packet/rate)
Optimize voice delay an jitter
same mechanism for Frame Relay

MultilinkPPP / FRF.11 & .12
Voice 2 Voice 1 Jumbogram
Fragment 4 Fragment 3 Voice 2 Fragment 2 Voice 1 Fragment 1
Line overhead
Segmentation/reassembly overhead (ppp
multilink or FRF12 overhead)
Fragment all packets greater than fragment
size defined
Interleave packets from OTHER queues
PQ-WFQ for Frame Relay
Dual Interface FIFO is turned on automatically when FRTS& FRF12 is

configured
Interface Queue operation before PQ-WFQ in 12.0(5)T:
LMI and unfragmented packets (VoIP, VoFR) to Hi queue
All fragmented packets (data) to Lo queue
Interface Queue operation with PQ-WFQ in 12.0(5)T:
LMI and PQ contents (VoIP and VoFR) to Hi queue
Everything else, regardless of fragmentation (data) to Lo queue
Per PVC Per Interface dual FIFO

V V Router
PQ - voice Interface
1 1
High
PQ V V WAN Circuit
2 L L 1 1
V V
3 3 3 3 4 3 2 4 3 2 L 1 1
WFQ
Low
4 4 4 4

PQ-WFQ FR (detailed)
Voice flow
Data flow
Inter-
WFQ leaving
Data flow de-queuing Fragmentation PVC A
Interface queues
Voice flow
High FIFO
Data flow PVC B
Inter-
WFQ Tx Ring
Data flow leaving
de-queuing Fragmentation Normal FIFO
Voice flow
Data flow PVC C

Inter-
WFQ
Data flow leaving
de-queuing Fragmentation
Frame Relay traffic-shaping with FRF 12 then FRF11

LLQ for FR updates
BW in the priority class
given in Kbps
max allowable BW for priority classes is mincir
does NOT take cRTP into account
class is policed to given BW (cannot burst like the data classes)
Voice is no longer shaped in 12.1.2T
If a voice packet present, it is sent in the next Tc timeslot
Bc is decremented for voice (even if already negative)
Data will be held back until Bc is positive again
VoFR:
will also use the PQ of LLQ, but does not use the class configuration
priority classes can be configured in LLQ at the same time
this is not recommended
BW available to priority classes while VoFR is present is:
mincir - voice bandwidth
Do NOT mix non-VoFR real-time traffic with VoFR on the same PVC: it can NOT be
interleaved (due to FRF.11 Annex C fragging spec)

Queuing Summary (updated)
LLQ
PQ CQ WFQ PQWFQ CBWFQ
(PQ-CBWFQ)
IP Prec,
RSVP, RTP VoFR and
Protocol, Protocol, VoFR and
Classification interface interface
Reserve, IP RTP Mod CLI
Mod CLI
protocol, Priority
port
1 PQ + 64 1 PQ +
# queues 4 16 Per flow
WFQ classes CBWFQ
Fair: PQ: Strict

Strict Round- Fair (weight, PQ: Strict
Scheduling priority robin arrival time) WFQ: Fair
weight and CBWFQ:
BW Fair/BW
Delay
Yes No No Yes No Yes
guarantee
BW PQ: yes
No No No Yes Yes
Guarantee WFQ: No
Used for Last

No No Yes No Yes
Voice resort

QOS & ATM (Paris Project)
Implemented on PA-A3
(TX-queue per VC allows fancy queuing per VC)
Phase 1 - Per-VC WRED (single VC)
Phase 2 - Precedence Mapping (multiple VC)
Bundle Management
Phase 3 - Per-VC WFQ

Per-VC WRED : Ph 1 Traffic
Threshold Exceeded Shaping
VIP2-50 PA-A3-XX
VC1
VC2
VC3
No discard
Per-VC Per-VC on PA
WRED: Queues
Intelligent Discard
IP ATM COS-Ph2: Bundle
Prec: 5 - 7 VC1: VBR-nrt
Prec: 0 - 4 VC2: ABR
A single Bundle routing neighborship!

(several PVCs grouped in one intf)
Flexible IP CoS mapping to VCs
RED (WRED) runs on each VC queue
VC Bundle Mgmt
Two Modes:
Protected VC rule : when a protected VC goes
down, the bundle goes down
Protected group rule : when all members in the
protected group fail, the bundle is declared DOWN
When a bundle is declared down, no traffic is
forwarded out of the bundle

Bumping
VC bumping: possibility for a traffic

mapped to a VC X to be forwarded onto
another VC Y, in case of failure of X.
Implicit bumping rule : Y is the next lower
precedence level VC is selected,
Explicit bumping rule : Y is explicitely
specified.

Bumping (Cont.)
Traffic is restored to the original VC when it

comes back.
Reject Bumping : It is possible for a VC to
be configured not to accept the bumped
traffic
When no alternate VC for some bumped
traffic, the bundle will be declared down.

Bumping (Cont.)
To prevent from declaring a bundle DOWN

due to the failure of the lowest precedence
VC, explicit bumping should be configured on
the lowest precedence.
Should be used in conjunction with the
protected group rule
If the VC which carries the bumped traffic
fails also, the traffic will follow the bumping
rules specified for that VC.

Bundle example
TOS 6-7 <-> VC 1 - ATM VBR-nrt

Protected VC

Protected Group- bump explicit 7

Protected Group - bump implicit
TOS 0-1 <-> VC 4 - ATM UBR

Protected Group
Based on MPLS-EXP (since 12.2(8)T)

LLQ per VC:ph3
Allows LLQ per PVC (in bundle)

Differentiated Service Code Point (DSCP)
IPv4 ToS IP prec ToS bits U
DSCP ECN
Former ToS byte = new DS field
DS codepoint: a specific value of the DSCP portion of the DS field, used

to select a PHB
DS field (rfc2474): the IPv4 header TOS octet or the IPv6 Traffic Class
octet when interpreted in conformance with the definition given in
[DSFIELD]. The bits of the DSCP field encode the DS codepoint,
ECN bits used for host Congestion Notification (RFC3168, cscdu83511)

802.1p, IP Precedence, DSCP
3 bits used for CoS (User Priority)!
Layer 2!
802.1Q/p! TAG!
PREAM.! SFD! DA! SA! PT! DATA! FCS!
4 Bytes!
Layer 3 !
IPV4! Standard IPV4/IPV6: Type of Service field!
Version! ToS!
Len! ID! offset! TTL! Proto! FCS! IP-SA! IP-DA! Data!
Length! 1 Byte!
IP Prec (3 bits)!
DSCP (6 bits)! ECN!
Layer 2 mechanisms exist on LAN segment

Layer 3 mechanisms can provide end-to-end classification
DiffServ Code Point (DSCP) encoding compliant with RFC 2474 + ECN in RFC3168
Class Selector Code Points (CSC) - backward compatibility to IP Precedence
Keywords in CB marking CS0 to CS7 (DCSP= xxx000)
DS field (RFC 2474, 2597, 2598)
Per-Hop Behaviours (PHB) DiffServ Code Points (DSCP)

46
Expedited Forwarding EF 101110
Assured Forwarding
Low Drop Med Drop High Drop
Pref Pref Pref
10 12 14
Class 1 AF11 AF12 AF13 001010 001100 001110
18 20 22
Class 2 AF21 AF22 AF23 010010 010100 010110
26 28 30
Class 3 AF31 AF32 AF33 011010 011100 011110
34 36 38
Class 4 AF41 AF42 AF43 100010 100100 100110
0
Best Effort BE 000000
DS field pools
DiffServ Code Points (DSCP) pools(RFC2474)
Pool 1 DSCP=xxxxx0 standard DSCP
Pool 2 DSCP=xxxx11 Local usage
Pool 3 DSCP=xxxx01 Local usage

(reserved for future std)

Class-based Marking
DSCP and IP Prec
Router(config-pmap-c)#set ip dscp ?
<0-63> Differentiated services codepoint value
af11 Match packets with AF11 dscp (001010)

cs1 Match packets with CS1(precedence 1) dscp (001000)

default Match packets with default dscp (000000)
ef Match packets with EF dscp (101110)
Requires CEF on the interface

(see also CSCdt74738 for marking internal flows)

Modular QoS CLI (MQC)
Configuring Classification policies

Associate policies to class
Attaching policies to interface (in/out)

Modular QoS CLI
class-map PQ
match
access-group Access group
destination-address Destination address (mac)
input-interface Select an input interface to match
ip IP specific values (prec, DSCP)
cos 802.1p/Q
mpls Multi Protocol Label Switching (exp)
not do not match
protocol Protocol (nbar )
qos-group Qos-group
source-address Source address (mac)
(multicast supported since CSCdr34902 12.1(4)T)
Modular QoS CLI
policy-map test
class PQ
bandwidth Bandwidth in kbps or percent/percent remaining
priority Strict Scheduling Priority for this Class+ burst
(default 200ms of traffic)
queue-limit Queue Max Threshold for Tail Drop
random-detect Enable WRED as drop policy
(IPprec or DSCP based) + ECN bits option
service-policy Configure QoS Service Policy-------> hierarchical
set Set QoS values (prec, DSCP,cos, exp, FRde, CLP)
shape Traffic Shaping: avg (CIR) or peak (CIR*(1+be/bc))
police traffic policing (conform, exeed, violate)
+2 rate policer + multiple actions

Hierarchical policy
Allow multi level (2) policing
policy-map bandwidth-allocation-policy
class udp
bandwidth <bw specification>
service-policy udp-hierarchy
class tcp
service-policy tcp-hierarchy
policy-map udp-hierarchy
class nfs
class tftp
policy-map tcp-hierarchy
class http
class ftp
Hierarchical limitations
Any to any combination supported on vip

Other platform parent/child restrictions:
parent level Bandwidth & Bandwidth Shaping Police

Shaping
Child level Bandwidth Police Police Police
Priority
Bandwidth

Common classification
QoS feature ordering
Since 12.1T, avoid multiple cascading classification, improve perf
input common classification
input access lists
input marking
input policing
CEF/fast switching
output common classification
output access lists
output marking
output policing
Traffic shaping
queueing (CBWFQ, WRED)

High-end routers
QOS mechanisms
7500 series + VIP use:

CEF distributed QOS
features
GSR use:
CAR + VOQ and MDRR/WRED
7200-NSE,10000, 7600 (OSM modules) use:
PXF processors technology

7500 Distributed QOS
dWFQ supports
Flow-based WFQ
ToS-based
Qos group-based
dLLQ

Flow based dWFQ
Forwarder
Classifier - Compute Hash
Flow Queue Queue Queue Queue

Queues 0 1 2 511
Calendar Queues
Transmission Queue
Output scheduler aka calendar is round robin (weight=1)

ToS based dWFQ
Forwarder
lower 2bits
IP Prec-Based
Classifier
Flow Queue Queue Queue Queue

Queues 0 1 2 3
Weight
Calendar Queues
Transmission Queue
calendar has 4 default weighted classes (10,20,30,40%)
Qos group dWFQ
Forwarder
QoS group set using CAR or
QoS policy progation via BGP
Classifier
QOS-
Group Queue Queue Queue Queue
0 1 2 99
Queues
Weight
(1-100) Calendar Queues
Transmission Queue

dLLQ
Forwarder
Per class
(ACL)
Class #1 Class #2 Class #3
Per flow Per flow Per flow

(hash) (hash) (hash)
Flow #2 Flow #2 Flow #2
Flow #1 Flow #3 PQ Flow #1 Flow #3 Flow #1 Flow #3
Scheduling based on Calendar Queueing

dLLQ features
Class default 1% min guarantee

Class default BW always garanteed
(=totale_BW sum(classes_BW))
Fair queue option per class
Weight used in calendar queue <99

Distributed features
dCEF, dNBAR, dTS, dGRE, dLLQ,

dmlppp, dfrf12,dWRED, dLFI
Non distributed QOS (RSP based) no
longer supported

QOS Mechanisms in GSR
CAR
Meter
Inbound
Traffic
Classifier Marker Conditioner Queuing
Stream
(from dCEF MDRR
ToFab F
CAR CAR
Select dest xACL
or or A
slot/intf) Policy Routing WRED
B
R
I
Meter C
Outbound FromFab
Traffic Queuing Conditioner Marker Classifier
Stream
QOS features available depend on line card

GSR: input queuing (VOQ)
Receive Side of Input line card
Input
Ports Per Dest Slot VOQ, 8 Virtual
Each Slot has 8 CoS Output
Queues Assigned Queues
W-RED /dest slot MDRR
Crossbar Switch Fabric

dCEF

GSR: output queuing
Transmit Side of output line card
W-RED MDRR
Output
Ports
Crossbar Switch Fabric

Modified Deficit Round Robin
(MDRR)
All queues, except low-latency queue,

are serviced in round robin fashion
Two modes for servicing the low-latency
queue
Alternate Priority and Strict Priority

Modified Deficit Round Robin
(MDRR)
Strict Priority
In Strict Priority mode, the low
latency queue is always serviced if
packets are queued. Remaining
.. queues are serviced in round
robin fashion
Alternate Priority
In Alternate Priority mode, service
alternates between the low-latency
queue and the other queues.
.. Remaining queues are serviced in
round robin fashion

Operation of DRR
Weights can be applied to queues to further

control packet individual de-queue rate when
interface is congested
Packet de-queue quantum q is:
q= MTU + (weight - 1)*512
(1<weight<100 to avoid large delays)
For Tx side MDRR local MTU is used
For Rx side MDRR largest MTU of destination
interfaces is used
Operation of DRR
Dequeue packets until the queues

counter is zero or negative.
Move to next queue.
If negative, retain the value until the
next cycle.
If positive, reset to zero. (i.e. avoid
accumulation of credits)

Parallel eXpress Forwarding
overview
US Patent 6,101,599 Column Column
Memory Memory
0 1 2 3
IM IM IM IM
Ingress Demux
Packet Packet
Egress Mux
4 5 6 7
Headers In IM IM IM IM
Headers Out
8 9 10 11
IM IM IM IM
12 13 14 15
IM IM IM IM
Column Column
Memory Memory feedback
A PXF Processor is a 4x4 CPU Array

PXF overview
Only header (64B) of a packet are sent through

the PXF processor.
Each column of CPU provides a specific service
or set of services.
Each row is a pipeline where services are applied
simultaneously to multiple packets.
Multiple rows allow processing in parallel. (but
no out of order packets)
Ability to run multiple pass in the array for
complex feature (that requires more cpu cycles)

PXF cascading
PXF Services cascading
CEF
VTMS,
Accounting shaping,
ACLs, policing CBWFQ,
Classification WRED, LLQ
.
PXF Services are platform dependant

Versatile Traffic Management
Services (VTMS)
VTMS is a set of PXF services for

OSM modules (7600/6500)
allow classification, shaping,
queuing and congestion mngt

VTMS
Overview
Scheduling
RED Queues Wheel
Classification
Per logical interface, per direction
DSCP-based for up to 64 classes
per direction
Classify
(64+default +system)
One VTMS queue per class (2048)
Shaping per queue
Ingress, egress Congestion Management
CIR, EIR (future) WRR w/ POS, CHOC, GE WAN
FQ w/ ATM
CBWFQ w/ ATM (future)
Congestion Avoidance
RED per queue w/ current OSMs
WRED per queue w/ future OSMs

Updates 12.2 (T)
set FR de, ATM CLP, EXP

Dual rate policer
QOS in MPLS
Bw percent in LLQ
RSVP classification
QoS preclassify
FRTS interface congestion (12.2(4)T)

Frame Relay DE bit
class-map match-fr-de
match fr-de
FR
policy-map DE-ToS
class match-fr-de
set ip precedence 1
interface s4/0.1
service-policy input DE-ToS
policy-map rate-limit policy-map FRDE
class police class class-default
police 512000 1500 1500
set fr-de
conform-action transmit
exceed-action set-frde-transmit interface s4/0.1
! Only when be is used
service-policy output FRDE
interface ser4/0.1
service-policy output rate-limit
ATM CLP bit
ATM
policy-map SETCLP policy-map rate-limit

class class-default class police
police 512000 1500 1500
set atm-clp conform-action transmit
! exceed-action set-clp-transmit
!
interface atm1/0/0
interface atm1/0/0
service-policy output SETCLP
service-policy output rate-limit

Single Rate/3 colors Policer (RFC2697)
Tc: token available in bucket1

Te: token avail in bucket2

Dual Rate/3 Colors Policer (RFC 2698)
Tp: token available in bucket1

Tc: token avail in bucket2

Actions (colors)
Conform / Exceed / Violate Actions

drop
set-dscp-transmit
set-mpls-exp-transmit
set-prec-transmit
set-clp-transmit
set-frde-transmit
set-qos-transmit
transmit
(since CSCdt11400 multiple actions for each color)

QoS and MPLS
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Label | EXP |S| TTL |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Two methods are possible
Single LSP per QoS FEC: E-LSP

use EXP field in MPLS header to select Diff-Serv queue
By default IP prec copied in EXP labelS (cisco)
By default exp is not copied down (in label below or prec)
Multiple LSPs per QoS FEC: L-LSP

use Label to select Diff-Serv queue:
End to End QoS
Customer CE PE Customer
QoS QoS SP QoS QoS QoS
Uniform mode: QoS bits modified by each domain
Short pipe: Core QoS is independant of edge
Pipe: SP devices QoS is independant of ingress
End to End: Transparency is assured CE to CE

MPLS CoS transparency
LSP TE
LSP LDP MPLS

MPLS
exp 5 or n exp 2
IP: dsp 5 MPLS
MPLS exp 5 or n IP: dscp 5
MPLS
exp 5 or n exp 5 or n
MPLS
IP: dscp 5 MPLS exp 5 or n
exp 5 or n
IP: dscp 5
IP: dscp 5
By default ingress IP prec copied in EXP labels (or manually set to n new cli
set mpls exp imposition n replace set mpls exp n)
Classification behavior on egress PE use DSCP
PHP should not be configured (use explicit-null) and use QOS group to map
EXP to QOS-group for egress PE classification based on EXP

CoS transparency
Uniform mode
Assume something recolors the top-most label here to 0

set mpls experimental top-most (0-7)
IP: MPLS MPLS IP:

dscp exp 5 exp 0 dscp
5 0
MPLS MPLS MPLS
exp 5 exp 5 exp 0
IP: IP: IP:
dscp dscp dscp
5 5 5
in both the MPLS-to-MPLS and the MPLS-to-IP cases, the PHBs of the top-most
popped label is copied into the new top label or the IP DSCP if no label remains.
mpls propagate-cos CSCdv21062
Only works for 8 distinct PHBs. Requires specific mapping dscp->mpls-exp-
>dscp if the distinct PHBs are encoded using more than the 3 precedence bits
Egress PE classification based on IP DSCP
CoS transparency
Short-Pipe mode
IP: MPLS MPLS IP:

5 5
MPLS MPLS MPLS
exp 5 exp 5 exp 0
IP: IP: IP:
dscp dscp dscp
5 5 5
the PHBs of the top-most popped label is copied into the new top label
but not copied in IP DSCP
Egress PE classification still based on IP DSCP

CoS transparency
Pipe mode
IP: MPLS MPLS IP:

5 5
MPLS MPLS MPLS
exp 5 exp 5 exp 0
IP: IP: IP:
dscp dscp dscp
5 5 5
the PHBs of the top-most popped label is copied into the new top label but not
copied in IP DSCP
classification is based on mpls-exp field (use qos group) of the top-most
received MPLS frame
PHP should not be configured (use explicit-null)

CoS Transparency
End to End
CE
PE
MPLS MPLS
IP: MPLS exp 2
dscp exp 2 exp 2
5 IP: MPLS
IP: exp 2
dscp dscp
5 5 IP:
dscp
Push a Null-Label and 5
use its EXP field to
convey result of Copy EXP of popped Null-label
classification into EXP of pushed labels
Preserve DSCP
QOS policy is done on CE side, reduce load/complexity on PE

the mpls-exp of the received mpls frame is copied in memory and later
copied in all the mpls-exp field of the pushed labels.
mpls ip encapsulate exp-null on CE intf facing PE
Diff-Serv over MPLS:
coloring MPLS Frames (PE)
class-map match-all DSCP-VoIP
match ip dscp 40
class-map match-any DSCP-Business
match ip dscp 32
class-map match-all DSCP-Management
match ip dscp 48
!
policy-map UPC
class DSCP-VoIP interface Serial0/0.1
set mpls exp 2 service-policy input UPC
class DSCP-Management
set mpls exp 1 (since 12.1(5)T)
class DSCP-Business
police 40000 1500 1500
conform-action set-exp-transmit 1 (since 12.2(1)T)
exceed-action drop violate-action drop
class class-default
set mpls exp 0
(set mpls experimental top-most (0-7) CSCdv19041
+CSCdv26453+ mpls propagate-cos CSCdv21062)
Diff-Serv over MPLS:
matching MPLS Frames (P)
class-map match-all MPLS-VoIP
match mpls exp 2 (since 12.0(7)T)
class-map match-any MPLS-Gold
match mpls exp 1
!
policy-map MPLS-QoS
class MPLS-VoIP
priority 24
class MPLS-Gold
bandwidth 44
random
class class-default interface Serial0/1
bandwidth 22 bandwidth 125
random max-reserved-bandwidth 90
tag ip
service-policy output MPLS-QoS

Qos-group and discard-class
INPUT POLICY on Egress PEs OUTPUT POLICY on egress PEs
class-map AF1
class-map AF11 match qos-group 1
match mpls exp top 0
policy-map input policy-map output

class AF11 class AF1
bandwidth percent 20
set qos-group 1
random-detect discard-class-based
set discard-class 1
As egress PE remove all labels, if egress QOS is based on MPLS-EXP field

QoS-group and discard-class (new) are used for egress classification and droping,

RSVP Packet Classification
Egress Interface/PVC
Unclassified
Queuing (LLQ)
flows voice conforming, admitted flows
PQ!
non-voice conforming,
RSVP admitted flows Reserved!
Classifi- Queues!
cation
Class1!
non-admitted flows, Class2!
voice signalling traffic,
PATH and RESV msgs
Default/BE!
Since CSCdp21647 RSVP is able to recognize voice flow and put it in PQ

Or use ip rsvp pq-profile (12.1(3)T)
picks up bearer traffic only - another classification mechanism must be used to prioritize
signalling traffic (use ip rsvp signalling dscp nn to assign dscp value to sig)
LLQ Configuration
>> Absolute Percent Specification (CSCds25849)
policy-map Multiservice policy-map Multiservice 75% rule
class VoIP class VoIP applies
priority 240 priority 240
(OR priority percent 10) OR priority percent 10
class business class business
bandwidth percent 30 bandwidth 512
class data class data
bandwidth percent 20 bandwidth 256
>> Relative Percent Specification

policy-map Multiservice 75% rule
class VoIP doesnt
priority percent 10 apply
class business
bandwidth remaining percent 30 (75% rule always apply on PQ)
class class-default
bandwidth remaining percent 20
QoS pre-classify
when IP is in IP output intf cant

classify based on inner IP header
for GRE, IPSec, L2TP
QoS preclassify cmd applied on tunnel,
crypto-map or V template
Store a copy of initial IP header for
classification after encap
12.1(5)T for 7100/7200 12.2.(2)T for 2600/3600

Future enhancements
Voice Adaptive Traffic Shaping

Possibility to use CIR when no voice packets
And use minCIR when voice is active
Flow based QOS CBWFQ on low-end
Already available in 7500 via fair-queue cmd
CLI
Hierarchical CBmarking
Percent keyword for CBpolicing

Conclusions
QoS not inherent to network

no BW creation, requires
provisioning
Requires for each flow:
Classification, metering,
and congestion control

Conclusions
PQ, CQ (and FIFO) available for ages

but many side effects.
New mechanisms for real QoS,
available only since 12.1-12.2 (T ...)
Requires upgrade plan ...

Cisco QoS

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco QoS

Uploaded by

Copyright:

Available Formats

QoS

NSA/spicard 2001, Cisco Systems, Inc. 1

Acronym for Quality of Service

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 2

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 3

Integrated Services (RFC1633)

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 4

Applications signal their QoS requirements via RSVP

Technique for providing QoS in TCP/IP

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 6

Flow Classification and policing

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 8

Classifier (ACL, NBAR,CB-marking)

Policy based Routing (PBR)

Traffic Shaping (GTS, FTRS, CB-shaper)

Congestion management(PQ, CQ, WFQ, CBWFQ, LLQ)

Congestion avoidance (wRED)

access-list 101 permit tcp any host 10.22.1.10

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 10

Classification based on protocol type, L4 ports,

No changes required to the engine

PDL files will be read from flash - no IOS upgrade

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 11

A New IP Packet Classifier Capable of

URL matching using regular expressions

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 12

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 13

Rate Limiting (Policing)

Matching specification: Confirm Action:

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 14

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 15

Branch Office Central

Reduces outbound traffic flow to avoid

Traffic Transmit Output

Classification by: Leaky

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 17

SubIntf 1.1 CIR=48 kbps Output

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 18

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 19

B - Burst size max in bits

NO: Exceed (drop, enqueue or use excess)

bucket is filled at defined rate with

Both ensure that traffic does not exceed a

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 22

Policer on input or output interface; Shaper on output

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 23

Traffic Rate Traffic Rate

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 24

Congestion management algorithms

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 25

Congestion avoidance algorithms

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 26

Simplest Queuing Algorithm

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 29

Order of Arrival completely determines the

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 30

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 31

Low Medium High

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 32

Four priority queues -

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 33

Can cause traffic lockout if

QoS mechanisms 2001, Cisco Systems, Inc. www.cisco.com 34