Professional Documents
Culture Documents
Version 5.0
Session Objectives
http://www.testbells.com/400-101.html
Session Non-Objectives
Related Sessions
Session Id
Session Name
BRKARC-3470
BRKARC-3452
BRKDCT-1980
BRKRST-2509
http://www.testbells.com/400-101.html
Agenda
Introduction
Conclusion
Introduction
http://www.testbells.com/400-101.html
http://www.testbells.com/400-101.html
http://www.testbells.com/400-101.html
Manage
Congestion
Maximize
Throughput
10
Agenda
Introduction
Conclusion
22nd Anniversary
Policing
Identify and
Split Traffic
into Different
Classes
Discard
Misbehaving
Traffic to Maintain
Network Integrity
Marking
Queuing &
Scheduling
Mark Traffic
Prioritize,
According to Protect and
Behavior and Isolate Traffic
Business
Based on
Policies
Markings
Shaping
Control
Bursts and
Conform
Traffic
13
Classification
Marking
Traffic Categorization
Traffic Re-categorization
Policing
Identify traffic
DSCP
CoS
ACLs
IP PREC
Before:
Uncontrolled Bandwidth
lync
mgmt
Videop2p
VoIP
HTTP
email
App1
vmotion
backup
Remap Traffic
VoIP/Lync
After:
Controlled Bandwidth
Email/HTTP
SAP
Backup
Tokens
Burst
Limit
Buffering
Queuing
Scheduling
Shaping
Client to Server
Server to Storage
Aggregation Points
10 GE
Speed Mismatch
10 GE
1 GE
1 GE
1 GE
1 GE
http://www.testbells.com/400-101.html
18
http://www.testbells.com/400-101.html
19
CoS
Queues
Priority
5-7
PQ
No-Drop
3, 4
Q2
Q1
0, 1
Default-Q
Best-Effort
DSCP
Priority
CS6 (CS7)
Platinum
EF
Queues
PQ
Gold
AF41
Q7
Silver
CS4
Q6
No-Drop
CoS3
Q5
Bronze
AF21
Q4
CS2
Q3
Scavenger
AF11
Q2
Bulk Data
CS1
Q1
Best-Effort
Default-Q
Management
Priority Queue
always served first
COS
Queue
5, 6, 7
PQ1
3, 4
Q3
Q2
0, 1
Q-Default
Egress Port
Normal Queue
Drop Threshold 4
PQ1
Drop Threshold 3
Drop Threshold 2
data
L3
DSCP [24]
L2
CoS [3]
Q3
Scheduler
DWRR
Q2
Drop Threshold 1
3
Normal Queues
served after PQ is
empty according to
Scheduler
Q-Default
24
of
Line
Blocking
25
Red Cars
blocked!
No Congestion
http://www.testbells.com/400-101.html
26
Stadium
Full
Cars turned
back
27
Stadium
Full
Supermarket
Not Full
http://www.testbells.com/400-101.html
28
Ingress Module
Egress Module
buffer
Ingress Module
Ingress Module
Crossbar
Fabric
Egress Module
Egress Module
29
Ingress Module
Ingress Module
Ingress Module
Egress Module
Crossbar
Fabric
Egress Module
Egress Module
30
Ingress Module
Egress Module
buffer
Ingress Module
Ingress Module
Crossbar
Fabric
Egress Module
Egress Module
http://www.testbells.com/400-101.html
31
Agenda
Introduction
Conclusion
Policy-Map
What action do I take on these
classes?
Policing
Marking
Scheduling
Service-Policy
Where do I apply this policy?
System Wide
Vlan
Interface (in/out)
Port-channels
34
http://www.testbells.com/400-101.html
35
DCBXP (802.1Qaz)
36
Network-QoS Policy
http://www.testbells.com/400-101.html
37
Queuing
CoS
DSCP
Network-QoS
CoS
Protocol (fcoe)
Policy-map
QoS
Marking
Policing
Mutation
Queuing
Buffering
Queuing
Scheduling
Network-QoS
Congestion-Control
Pause / MTU per VL
Service-policy
QoS
Interfaces
Vlans
Port-channel
System-qos
Queuing
Interfaces
Port-channels
System-qos
Network-QoS
System-qos
38
Routed Port
Access Port
VLAN
802.1q Trunk
Routed Port
QoS
Polcies
VLAN
Access Port
802.1q Trunk
Service Policy
applied to the
VLAN Database
39
Service Policy
policy
applied to the
Switchport
switchport
Routed Port
Access Port
802.1q Trunk
QoS
Policies
Routed Port
or the
Routed Port
Port-channel
or a portchannel
40
Service Policy
policy
applied to the
Switchport
switchport
Routed Port
Access Port
802.1q Trunk
QoS
Policies
Routed Port
or the
Routed Port
Port-Channel
or a portchannel
41
42
Agenda
Introduction
Conclusion
F1 10G
F3 40G
F2E 10G
F2 10G
F2E 10G
To Fabric Modules
To Central Arbiters
Arbitration
Aggregator
Fabric 2 ASIC
LC
CPU
VOQs
VOQs
Forwarding
Engine
Replication
Engine
Forwarding
Engine
VOQs
Replication
Engine
Replication
Engine
Replication
Engine
VOQs
3
>Policy Type qos
(Mutation, Classification,
Marking, Policing)
Return Credit
to Pool
Arbiter
9
CoS to Queue-Mapping
>Policy Type queuing
(Scheduling, Buffering)
Forwarding
Engine
data
L3
DSCP [24]
L2
CoS [2]
Ingress Module
4
VoQ gets assigned
based on Dest +
CoS value
1 2 3 4 5 6 7 88
Crossbar
Fabric
Egress Module
L3
DSCP [24]
L2
CoS [2]
7 Traffic gets
Received on
Egress VoQ
6 Traffic gets
forwarded over
Fabric
Ingress
data
1 2 3 4 5 6 7 8
8q2t
Ingress port-ASIC
1 2 33 4
VOQ
1p3q1t
Ingress VoQ-ASIC
Egress
1p7q4t
11
Egress
FIFO
Egress VoQ-ASIC
10
Egress port-ASIC
3
>Policy Type qos
(Mutation, Classification,
Marking, Policing)
Return Credit
to Pool
Arbiter
9
CoS to Queue-Mapping
>Policy Type queuing
(Scheduling, Buffering)
Forwarding
Engine
data
L3
DSCP [24]
L2
CoS [2]
Crossbar
Fabric
Ingress Module
41
VQI gets assigned
based on 5-Tuple
flow information
1 2 3 4 5 6 7 88
Egress Module
L3
DSCP [24]
L2
CoS [2]
7 Traffic gets
Received on
Egress VoQ
6 Traffic gets
forwarded over
Fabric
Ingress
data
1 2 3 4 5 6 7 8
8q2t
Ingress port-ASIC
2
Default CoS-to-Queue map:
CoS [2] to Default-Q (8)
1 2 33 4
1p3q1t
Ingress
VOQ
42
VoQ gets assigned
VoQ-ASIC based on VQI + CoS
Egress
1p7q4t
11
Egress
FIFO
Egress VoQ-ASIC
10
Egress port-ASIC
24-port 10G
N7K-M224XP-23L
8q2t
5.2MB / port
4.5MB / port
6-port 40G
N7K-M206FQ-23L
8q2t
20.8MB / port
18MB / port
2-port 100G
N7K-M202CF-22L
8q2t
62.8MB / port
54MB / port
Egress
M2 Module
VOQ Structure
Egress Queue
Structure
Egress Port
Buffer
24-port 10G
N7K-M224XP-23L
1p4q
295KB / port
512KB / 6 ports
1p7q4t
5MB / port
6-port 40G
N7K-M206FQ-23L
1p4q
1.2MB / port
1MB / 3 ports
1p7q4t
20.7MB / port
2-port 100G
N7K-M202CF-22L
1p4q
3MB / port
1MB / port
1p7q4t
30.2MB / port
EOBC
6 X 110Gbps
FSA
CPU
To Central Arbiters
6 X 110Gbps
Arbitration
Aggregator
x6
1G per SOC
Fabric ASIC
LC Inband
Fabric ASIC
55Gbps
55Gbps
to LC
CPU
8 X 10G
SoC 1
1
3
2
8 X 10G
SoC 2
5
7
6
9
8
11
10 12
8 X 10G
SoC 3
13 15
14 16
17 19
8 X 10G
SoC 4
21 23
25 27
8 X 10G
SoC 5
29 31
18 20
22 24
26 28
30 32
Front Panel Ports (SFP/SFP+)
33 35
34 36
8 X 10G
SoC 6
37 39
38 40
41 43
45 47
42 44
46 48
LinkSec-capable
To Fabric Modules
EOBC
Arbitration
Aggregator
LC
CPU
Fabric 2
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
4 X 10G
SoC
13
17
21
25
29
33
37
41
45
3
2
7
6
11
10
12
14
15
16
18
19
20
22
23
24
26
27
28
30
31
32
34
35
36
39
38
40
42
43
44
46
47
48
EOBC
6 X 110Gbps
FSA
CPU
To Central Arbiters
6 X 110Gbps
Arbitration
Aggregator
6 x 1G
x 12
1G switch
Fabric ASIC
Fabric ASIC
1G per SOC
LC Inband
55Gbps
55Gbps
2 X 40G
SoC 1
2 X 40G
SoC 2
2 X 40G
SoC 3
2 X 40G
SoC 4
2 X 40G
SoC 5
2 X 40G
SoC 6
2 X 40G
SoC 7
2 X 40G
SoC 8
2 X 40G
SoC 9
2 X 40G
SoC 10
2 X 40G
SoC 11
2 X 40G
SoC 12
11
13
15
17
19
21
23
10
12
14
16
18
20
22
24
EOBC
6 X 110Gbps
FSA
CPU
To Central Arbiters
6 X 110Gbps
Arbitration
Aggregator
6 x 1G
x 12
1G switch
Fabric ASIC
Fabric ASIC
1G per SOC
LC Inband
55Gbps
55Gbps
1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G 1 X 100G
SoC 1
SoC 2
SoC 3
SoC 4
SoC 5
SoC 6
SoC 7
SoC 8
SoC 9
SoC 10 SoC 11
SoC 12
10
11
12
F3 SoC Architecture
To Central
Arbiter
To Local
Fabric
From Local
Fabric
Decision Engine
Egress Parser
Virtual
Queuing
Ingress
Buffer
Layer 2 Lookups
Layer 3 Lookups
Classification
Marking / Policing
Egress Buffer
Ingress Parser
Port Logic
F3 SoC
http://www.testbells.com/400-101.html
Central Arbiter
8K VOQs
(1K Dest + 8 priority levels)
4 ingress queues
per port
10G Port 1
Egress Buffer
DWRR
Ingress Buffer
VQI 1
SP
1.5MB
10G Port 8
1.5MB
DWRR
VQI 8
SP
FABRIC
Buffering / queuing
Egress Scheduling
Ingress SOC
10G Port 1
Egress SOC
10G Port 8
Central Arbiter
8K VOQs
(1K VQIs + 8 priority levels)
6MB
FABRIC
Buffering / queuing
Ingress Buffer Manages congestion toward
egress destinations (VQIs)
6MB
VQI 1
40G Port 1
VQI 2
40G Port 2
SP
40G Port 2
DWRR
Egress Buffer
SP
40G Port 1
Ingress Buffer
Egress Scheduling
Ingress SOC
8 priority
levels
DWRR
4 ingress queues
per port
Egress SOC
Central Arbiter
8K VOQs
(1K VQIs + 8 priority levels)
12MB
Buffering / queuing
DWRR
Egress Buffer
FABRIC
100G Port 1
Ingress Buffer
VQI 1
SP
4 ingress queues
per port
Egress Scheduling
Ingress SOC
Egress SOC
8 priority
levels
100G Port 1
Supervisor Engine
Buffer credit
granted
Credit Central
Request buffer
credit for dest +
priority
Arbiter
Fabric Module 1
Fabric Module 2
Fabric Module 3
Fabric Module 4
Fabric Module 5
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Check
VQ drop
thresholds
Fabric ASIC
Req
Decision Engine
PKT
e2/1
Virtual
Queuing
HDR
e1/1
Queue packet
descriptor in VOQ
(dest + priority)
Ingress
Buffer
Payload queued in
port buffer based on
COS / DSCP
CL TCAM
Layer 2 Lookups
Layer 3 Lookups
Classification
Marking / Policing
Ingress
Parser
PKT HDR
Receive
from fabric
CL TCAM contains
QoS classification
rules
All L2 + L3 lookups /
Ingress + egress QoS classification /
Ingress + egress marking and policing
SP
Packet headers
sent to DE
Buffer on egress
based on dest+ priority
Port Logic
Module 1
DWRR
e2/1
Egress Buffer
Port Logic
F2E/F3 SoC
Receive packet
from wire
Credit
Ingress packet
parsing
Schedule
and transmit
on wire
F2E/ F3 SoC
Module 2
PKT HDR
e1/1
e2/1
8e-4q4q Template
7e Template
6e Template
CoS 5-7
CoS 5-7
CoS 5-7
CoS 5-7
(Q1)
(Q1)
(Q1)
(Q1)
4e Template
CoS 5-7
(Q1)
CoS 0
(Q-Default)
CoS 3-4
CoS 2,4
CoS 0-2
(Q3)
(Q3)
(Q-Default)
CoS 4
(Q3)
CoS 0-4
CoS 2
CoS 0,1
CoS 4
(Q-Default)
(Q4)
(Q-Default)
(Q3)
CoS 0-1
CoS 3
CoS 3
(Q-Default)
(Q4)
(Q4)
High (Pause)
Threshold
CoS 1-3
(Q4)
Low (Resume)
Threshold
59
DWRR
DWRR
Prio
50%
50%
Q2
Prio
100%
Prio
DWRR
DWRR
PQ2 (H)
Q-Def
33%
Q3
Q-Def
33%
PQ1
Q3
33%
Egress Port
Q2
33%
PQ1
Q-Def
33%
Egress Port
Q3
33%
Prio
DWRR
DWRR
Q2
PQ1
Egress Port
DWRR
Priority
100%
DWRR
Q3
Priority
Egress Port
Priority
PQ2 (L)
DWRR
Q-Def
PQ1 Q3 Q-Def.
(5,6,7) (2,4) (0,1)
PQ1 (H)
Q2
(3)
Egress Port
Q3 Q-Def.
(2) (0,1)
PQ1
PQ1 Q2
(5,6,7) (3,4)
default-4q-4e-out-policy
PQ3 (L)
PQ1 Q2 Q3 Q-Def.
(5,6,7) (3,4) (2) (0,1)
default-4q-6e-out-policy
Q-Def
100%
72MB
4q1t
1.5MB / port
F3 48-port 10G
N77-F348XP-23
72MB
4q1t
1.5MB / port
F3 24-port 40G
N77-F324FQ-25
144MB
4q1t
6MB / port
F3 12-port 100G
N77-F312CK-26
144MB
4q1t
12MB / port
Egress
Module
1p7q1t
295KB / port
256KB / 4 ports
F3 48-port 10G
N77-F348XP-23
1p7q1t
295KB / port
512KB / 8 ports
F3 24-port 40G
N77-F324FQ-25
1p7q1t
1.1MB / port
512KB / 2 ports
F3 12-port 100G
N77-F312CK-26
1p7q1t
2.2MB / port
512KB / 1 port
default-nq-8e-policy is default
network-qos policy and attached
to system qos in Admin-/DefaultVDC
2 ingress queues
with buffer ratio
1:9 and DWRR
wieights 1:1
4 egress queues
with one priority
queue and DWRR
wieights 1:1:1
Default CoS to Queue Mapping for Nexus 7000/7700 (F- and M-Series I/O
Module)
DSCP to Queue Mapping for Nexus 7000/7700 (F- and M-Series I/O Module)
63
802.1q Trunk /
Access Port
L3
DSCP [24]
L2
CoS [2]
Egress
Queues
CoS
data
CoS
Ingress
Queues
q-n
q-1
q-default
Set CoS 0
(will only set CoS to 0)
data
L3
L3
L2
Set DSCP 0
(will set DSCP + CoS to 0)
data
L3
DSCP [0]
L2
CoS [0]
DSCP
DSCP
[24]
[0]
CoS [0]
65
66
Default Queuing and QoS policies are applied to all physical interfaces across
all VDCs
http://www.testbells.com/400-101.html
Agenda
Introduction
Conclusion
High Performance
High Scalability
Feature-Rich
10 GE and true 40 GE
L2 and L3 features
Up to 224,000 MAC
Up to 128,000 ARP
FC and FCOE
VXLAN Bridging and
Routing
70
Ingress
VOQ buffer
Egress
VOQ buffer
Egress
VOQ buffer
Ingress Module
Nexus 7000
F-Series
I/O Module
Crossbar
Ingress Module
Egress Module
Fabric
Ingress Module
71
Nexus 5000/6000 & Nexus 7000 F-Series I/O Modules share the Ingress Buffer
Model
http://www.testbells.com/400-101.html
72
Traffic classification
DSCP, CoS, IP Precedence and ACL
Packet marking
DSCP, CoS, and ECN
http://www.testbells.com/400-101.html
Supervisor
UPC 1
UPC 2
UPC 3
UPC 4
24xSFP+
24xSFP+
Slot 1
Unified Ports
UPC 5
UPC 6
6xQSFP+
Slot 2
UPC-0
CPU
Expansion Slot 2
Supervisor
Fabric
UPC 1
UPC 2
24x
SFP+
UPC 4
UPC 3
24x
SFP+
Slot 1
CPU
UPC 5
2x
QSFP+
UPC 6
2x
QSFP+
Packet Buffering
25MB packet buffer is shared by every three 40 GE ports or twelve 10 GE ports.
Buffer is 16MB at ingress and 9MB at egress.
Ingress
UPC
16MB
Egress UPC
224
Gbps
Unified
Crossbar
Fabric
448
Gbps
9MB
MAC
Traffic
Classification
Ingress
Cos/DSCP
Marking
Ingress
Policing
Forwarding
Per-class
Buffer Usage
Monitoring
Ingress Buffer
Egress Queues
MAC
Egress UPC
WRED/ECN
Marking
Strict Priority +
DWRR Scheduling
Egress
Scheduling
Egress Buffer
Crossbar
Fabric
VoQs
(8 per egress port)
SPAN
Control
Port 2
Dedicated
Port 3
Dedicated
switch(config)# hardware random-detect min-thresh 10g 10g-min-threshold 40g 40g-minthreshold max-thresh 10g 10g-max-threshold 40g 40g-max-threshold ecn qos-group qosgroup-number
Attach policy-map
queuing to
interface
Create policy-map
type qos and set
qos-group and/or
add policing rule
Create policy-map
type queuing and
create actions
Attach policy-map
type qos as input to
an interface
Create class-map
type queuing and
match on qosgroup
Buffering is achieved via policy-map type network-qos where queuelimit can be configured
Queuing will be configured with policy-map type queuing and does NOT
interfere with policy-map type qos
N5k# show interface ethernet 1/1 capabilities | i Model|QOS
Model:
N5K-C5672UP-SUP
QOS scheduling:
rx-(6q1t),tx-(1p6q0t)
82
Buffering Capacity
Ingress
Traffic Type
10 GE Port
40 GE Port
6q1t
64 KB
67 KB
38.4 KB
154 KB
100 KB
100 KB
13.2 MB
14.7 MB
6q1t
6q1t
6q1t
Egress
Traffic Type
Unicast
Multicast
Egress Queue
Structure
1p5q0t
1p5q0t
10 GE Port
40 GE Port
363 KB
4.3MB
6.6 MB
http://www.testbells.com/400-101.html
Agenda
Introduction
Conclusion
FEX Overview
Traffic classification
DSCP, CoS
ACL classification (FEX offload) only on 5600/6000
Queue-limit Carving
http://www.testbells.com/400-101.html
Ingress/egress
VLAN-based QoS
policy
Nexus Parent
FEX
Ingress FEX
Module
/Port
Ingress Module
Module
/Port
Egress Module
FEX
Egress FEX
Ingress queue /
egress schedule
based on COS 0
Egress schedule
based on COS 0
Ingress queue at
input port based on
COS 0
Ingress queue /
egress schedule
based on COS 0
Nexus Parent
PKT
DSCP 0
1Q trunk
COS 0
FEX
Ingress FEX
(FEX 101)
Module
Ingress Module
Fabric
Module
Egress Module
FEX
Egress FEX
Ingress queue /
egress schedule
based on DSCP 0
Egress schedule
based on COS 0
Ingress queue at
input port based on
DSCP 0
Ingress queue /
egress schedule
based on DSCP 0
Nexus Parent
PKT
DSCP 0
COS 0
FEX
Ingress FEX
(FEX 101)
Module
Ingress Module
Fabric
Module
Egress Module
FEX
Egress FEX
http://www.testbells.com/400-101.html
FEX QoS Packet Flow Example (With Ingress Marking Policy and DSCPto-Queue)
Ingress QoS policy:
policy-map type qos marker
class ClassA
set dscp cs3
!
interface ethernet 101/1/1
service-policy type qos input marker
Ingress queue /
egress schedule
based on DSCP 0
Egress schedule
based on COS 3
Ingress queue at
input port based on
DSCP 0
Ingress queue /
egress schedule
based on DSCP CS3
Nexus 7000
PKT
DSCP 0
COS 0
FEX
Ingress FEX
(FEX 101)
PKT
DSCP CS3
Module
COS 3
Fabric
Ingress Module
Classify and remark to
DSCP CS3/COS 3 at
ingress Decision Engine
Module
Egress Module
FEX
Egress FEX
FEX QoS Packet Flow Example (With Ingress Marking Policy and DSCPto-Queue)
Ingress QoS policy:
policy-map type qos marker
class ClassA
set dscp cs3
!
interface ethernet 101/1/1
service-policy type qos input marker
Ingress queue /
egress schedule
based on DSCP 0
Egress schedule
based on COS 3
Ingress queue at
input port based on
DSCP 0
Ingress queue /
egress schedule
based on DSCP CS3
Nexus 7000
PKT
DSCP 0
COS 0
FEX
Ingress FEX
(FEX 101)
PKT
DSCP CS3
Module
COS 3
Fabric
Ingress Module
Classify and remark to
DSCP CS3/COS 3 at
ingress Decision Engine
Module
Egress Module
FEX
Egress FEX
Both system level and interface level policies are offloaded to the FEX
Number of queues
MTU
On Nexus 7000 with FEX + M-Series parent modules, network-qos and F-series
ingress queuing class-maps still drive FEX queuing configuration
DSCP-to-Queue on FEX
http://www.testbells.com/400-101.html
FEX Queue-Limit
Agenda
Introduction
Conclusion
101
CoS
Queuing
(Scheduling)
Queue-Limit
(Buffer)
Character
Best Effort
0, 1
BW remaining
50%
60%
High Volume /
Less Important
vMotion / Live
Migration
BW remaining
20%
10%
Medium Volume /
Important
Multimedia
3, 4
BW remaining
30%
20%
Medium Volume
Very Important
Strict Priority
5
Priority Queue
Network Control
6,7
10%
Low Volume /
Important /
Delay Sensitive
Low Volume /
Very important
102
Topology
Core
M2 cards facing
core and F2/F3
cards facing
access
Nexus 7000
Nexus 7000
vpc peer-link
VPC from
access to
aggregation
Nexus 5000
Straight-through
FEX, No VPC
Nexus 5000
Host VPC,
Straight-through
FEX
Nexus 2000
Nexus 2000
Nexus 2000
Nexus 5000
Enhanced VPC
(evPC)
Nexus 2000
Nexus 2000
Nexus 2000
103
vpc peer-link
vpc peer-link
Between the
different Tiers,
all ports are
TRUSTED
vpc peer-link
vpc peer-link
Mark Traffic
from Servers
(policy-map
type qos)
or just TRUST
(default)
104
105
106
CM_N-QOS_MATCH_QG2_COS1
CM_N-QOS_MATCH_QG3_COS2
CM_N-QOS_MATCH_QG4_COS4
CM_N-QOS_MATCH_QG5_COS5
107
vpc peer-link
vpc peer-link
vpc peer-link
Traffic
Classification
based on CoS
happens here!
Traffic
Classification
based on ACL
happens here!
108
Queuing (M2-cards)
Core
vpc peer-link
vpc peer-link
Modify CoS to
Queue mapping
in Admin/Default-VDC
Assign ingress
policy-map to each
interface with
bandwidth and
queue-limit
vpc peer-link
vpc peer-link
109
CoS
Queuing
(Scheduling)
Queue-Limit
(Buffer)
Queue
(8q2t / 1p7q4t)
Character
Best Effort
0, 1
BW remaining
50%
60%
8q2t-in-q-default /
1p7q4t-out-q-default
High Volume /
Less Important
vMotion / Live
Migration
BW remaining
20%
10%
8q2t-in-q6 /
1p7q4t-out-q6
Medium Volume /
Important
Multimedia
3, 4
BW remaining
30%
20%
8q2t-in-q2 /
1p7q4t-out-q2
Medium Volume
Very Important
Strict Priority
5
Priority Queue
Network Control
6,7
10%
8q2t-in-q1 /
1p7q4t-out-pq1
Low Volume /
Important /
Delay Sensitive
Low Volume /
Very important
110
Core
match-any 8q2t-in-q1
match-any 8q2t-in-q2
match-any 8q2t-in-q6
AdminVDC
AdminVDC
match-any 8q2t-in-q-default
match-any 1p7q4t-out-pq1
match-any 1p7q4t-out-q2
match-any 1p7q4t-out-q6
match-any 1p7q4t-out-q-default
Core
Core
vpc peer-link
vpc peer-link
vpc peer-link
114
AdminVDC
AdminVDC
vpc peer-link
vpc peer-link
vpc peer-link
vpc peer-link
116
Best Effort
CoS
0,1
Queuing
(Scheduling)egress
Queue-Limit
(Buffer)-ingress
Queue
(Ingress/Egress)
Character
BW remaining
50%
50%
4q1t-8e-4q4q-in-q-default /
1p3q1t-8e-4q4q-out-q-default
High Volume /
Less Important
vMotion / Live
Migration
BW remaining
20%
10%
4q1t-8e-4q4q-in-q4 /
1p3q1t-8e-4q4q-out-q3
Medium Volume /
Important
Multimedia
3, 4
BW remaining
30%
30%
4q1t-8e-4q4q-in-q3 /
1p3q1t-8e-4q4q-out-q2
Medium Volume
Very Important
Strict Priority
5
Priority Queue
Network Control
6/7
10%
4q1t-8e-4q4q-in-q1 /
1p3q1t-8e-4q4q-out-pq1
Low Volume /
Important /
Delay Sensitive
Low Volume /
Very important
117
Core
match-any 4q1t-8e-4q4q-in-q1
match-any 4q1t-8e-4q4q-in-q-default
match-any 4q1t-8e-4q4q-in-q3
AdminVDC
AdminVDC
match-any 4q1t-8e-4q4q-in-q4
match-any 1p3q1t-8e-4q4q-out-pq1
match-any 1p3q1t-8e-4q4q-out-q2
match-any 1p3q1t-8e-4q4q-out-q3
match-any 1p3q1t-8e-4q4q-out-q-default
Core
119
120
CoS
Queuing
(Scheduling)
Queue-Limit
(Buffer)
Queue
(6q1t / 1p6q0t)
Character
Best Effort
0,1
BW percent
40%
remaining
(226kByte)
qos-group 0 (default)
High Volume /
Less Important
vMotion / Live
Migration
2,3
BW percent
20%
41kByte
qos-group 3
Medium Volume /
Important
Multimedia
BW percent
30%
41kByte
qos-group 4
Medium Volume
Very Important
Strict Priority
Network Control
6,7
BW percent
10%
20kByte
(min)
qos-group5 / priority
Low Volume /
Important /
Delay Sensitive
Low Volume /
Very important
121
Core
122
Core
system qos
service-policy type network-qos PM_N-QOS_SYSTEM
123
Core
vpc peer-link
vpc peer-link
vpc peer-link
vpc peer-link
124
Core
Agenda
Introduction
Conclusion
Conclusion
Manage
Congestion
Maximize
Throughput
128
Merchant+ Strategy
Mix of Merchant and Custom Silicon
http://www.testbells.com/400-101.html
Buffering Model
Nexus 9000 compared to Nexus 7000/Nexus 5000
Ingress
VOQ buffer
Egress buffer
Egress
VOQ buffer
Ingress Module
Nexus 7000
F-Series
I/O Module
Crossbar
Ingress Module
Egress Module
Fabric
Ingress Module
132
Traffic classification
DSCP, CoS, and ACL
Packet marking
DSCP, CoS, and ECN
Buffer boost
http://www.testbells.com/400-101.html
System QoS
unicast
Trust CoS/DSCP
L2/L3/L4 info with ACL
Ingress
Cos/DSCP
Marking
Traffic
Classification
MTU
checking
CPU
span
Fabric
Module
NFE
unicast
unicast
multicast
multicast
CPU
span
control
span
Egress Port Queues
System QoS
Ingress
Cos/DSCP
Marking
Traffic
Classification
unicast
Trust CoS/DSCP
L2/L3/L4 info with ACL
Ingress
Policing
multicas
t
MTU
checking
CPU
span
Fabric
Module
OOBFC signaling
ALE
NFE
Unicast
from EoQs
Unicast EoQs
(4 per egr port)
Non-hairpinned local
unicast traffic
unicast
multicast
unicast
unicast
multicast
multicast
Control
Span
Span
Egress Port Queues
CPU
span
System QoS
Fabric Modules
ALE
10 MB
Buffer
10 MB
Buffer
20 MB
Buffer
10 MB
Buffer
20 MB
Buffer
10 MB
Buffer
North Star
NFE
ALE
North Star
NFE
NFE
NFE
4:1 Mux
Network Interface
Network Interface
Buffer Boost
disabled port
40GE 40GE
40GE
40GE
Marking policy
set dscp 18
class fex-qos-class-3
set dscp 26
!
interface Ethernet101/1/1
Applying 8e-4q4q template to enable 4 ingress/egress queues on FEX with COS to queue
mapping (also enables 4 ingress queues on F-series modules, if present)
system qos
service-policy type network-qos default-nq-8e-4q4q-policy
Queuing:
queue
qos-group
cos
priority
bandwidth mtu
--------+------------+--------------+---------+---------+---ctrl-hi
n/a
PRI
2400
ctrl-lo
n/a
PRI
2400
0 1
WRR
30
1600
WRR
30
1600
5 6
WRR
10
1600
3 4
WRR
30
1600
congestion-control tail-drop
mtu 9216
system qos
Queuing:
queue
qos-group
cos
priority
bandwidth mtu
--------+------------+--------------+---------+---------+---ctrl-hi
n/a
PRI
2400
ctrl-lo
n/a
PRI
2400
0 1
WRR
30
9280
WRR
30
9280
5 6
WRR
10
9280
3 4
WRR
30
9280
Non-default F-series
ingress queuing
class-maps (COS and
DSCP match statements
modified)
Queuing:
queue
qos-group
n/a
PRI
n/a
PRI
2400
WRR
30
1600
WRR
30
1600
1 2 3
WRR
10
1600
4 5
WRR
30
1600
queue
bandwidth mtu
ctrl-lo
priority
ctrl-hi
<>
cos
--------+------------+--------------+---------+---------+----
DSCPs
-----
-----
02
0-7,
04
8-31,
03
48-63,
05
32-47,
2400
Example #1 N2K-C2248TP-1GE
fex 101
hardware N2248T queue-limit 50000
Example #2 N2K-C2232TM-E-10GE
fex 102
hardware N2232TM-E queue-limit 50000
FEX output (show queuing interface) after (configured queue-limit rounded to nearest hardware
supported value):
http://www.testbells.com/400-101.html
F3 Queuing Configuration
Examples
Steps to follow
1.
1.
Define new type queuing policy-map (you cannot modify the default
policies)
Modify class-map parameters
Apply new policy-map to interfaces
Define new type queuing policy-map
Modify COS-to-queue mapping for target port type
Apply new policy-map to interfaces
(Optional) Clone the default egress queuing policy
Shape the SP queue in the new (cloned) policy
Apply the new queuing policy to the target interfaces
2.
3.
1.
2.
3.
1.
2.
3.
Important: changing COS- or DSCP-to-queue mapping takes effect immediately and is disruptive to
all ports
Show current
mapping
Configure new
mapping
Show new
mapping
Important: applying new queuing policy takes effect immediately and is disruptive to any ports to
which the policy is applied
Create new
queuing policy
Modify class-map
parameters (resize queues)
Apply policy to
interface(s)
Important: changing COS/DSCP-to-queue mapping and takes effect immediately and is disruptive to all ports;
applying new queuing policy takes effect immediately and is disruptive to any ports to which the policy is applied
Modify class-map
parameters (resize queues)
Apply policy to
interface(s)
Important!
If you change the COS- or DSCP-to-queue mapping for a port type, make sure all ports of that type in
all VDCs have a queuing policy applied that defines behavior for all queues with COS/DSCP values
mapped
Important!
http://www.testbells.com/400-101.html
Important!
If you DONT, traffic arriving on ports with default policy (i.e., without all queues activated
that have COS/DSCP values mapped) will suffer packet drops, poor performance, etc.
Of course, you can have different non-default policies on different sets of interfaces, but all
interfaces in the system must use some policy that defines all activated queues!
Important: applying new queuing policy takes effect immediately and is disruptive to any
ports to which the policy is applied
Promote your favorite speaker through Twitter and you could win $200 of Cisco
Press products (@CiscoPress)
You can submit an entry for more than one of your favorite speakers
Table Topics
Related sessions
http://www.testbells.com/400-101.html
Thank you
Description
Cisco Certification
CCNA Industrial
Cisco Industrial
Networking Specialist
Networking Fundamentals
for Industrial Control Systems (INICS)
Description
Cisco Certification
Cisco Enterprise IT
Business Specialist
Builds skills to discover and address technology needs using a businessfocused, consultative sales approach
Cisco Transformative
Architecture Specialist
Description
Focuses on the design, implementation, and monitoring of a comprehensive
security policy, using Cisco IOS security features
Cisco Certification
CCNA Security
Configure Cisco perimeter edge security solutions utilizing Cisco Switches, Cisco
Routers, and Cisco Adaptive Security Appliance (ASA) Firewalls
Deploy Ciscos Next Generation Firewall (NGFW) as well as Web Security, Email
Security and Cloud Web Security
Deploy Ciscos Identity Services Engine and 802.1X secure network access
Protect data traversing a public or shared infrastructure such as the Internet by
implementing and maintaining Cisco VPN solutions
Designed for professional security analysts, the course covers essential areas of
competency including event monitoring, security event/alarm/traffic analysis, and
incident response
For official product training on Ciscos latest security products, including Adaptive
Security Appliances, NGIPS, Advanced Malware Protection, Identity Services
Engine, Email and Web Security Appliances see
www.cisco.com/go/securitytraining
Description
Cisco Certification
Configure, implement and troubleshoot local and wide-area IPv4 and IPv6
networks. Also available in self study eLearning format with Cisco Learning
Lab.
Description
CCNP Wireless
CCNA Wireless
Cisco Certification
Description
Cisco Certification
Description
SPROUTE covers the implementation of routing protocols (OSPF, IS-IS, BGP),
route manipulations, and HA routing features; SPADVROUTE covers advanced
routing topics in BGP, multicast services including PIM-SM, and IPv6;
Cisco Certification
CCNP Service Provider
SPEDGE covers network services, including MPLS Layer 3 VPNs, Layer 2 VPNs,
and Carrier Ethernet services; all within SP IP NGN environments.
The two courses introduce networking technologies and solutions, including OSI
and TCP/IP models, IPv4/v6, switching, routing, transport types, security, network
management, and Cisco OS (IOS and IOS XR).
The three courses (SPUMTS, SPCDMA, SPLTE) cover knowledge and skills
required to understand products, technologies, and architectures that are found in
Universal Mobile Telecommunications Systems (UMTS) and Code Division Multiple
Access (CDMA) packet core networks, plus their migration to Long-Term Evolution
(LTE) Evolved Packet Systems (EPS), including Evolved Packet Core (EPC) and
Radio Access Networks (RANs).
Description
Cisco Certification
CCIE Collaboration
CCNP Collaboration
CCNP Collaboration
CCNA Collaboration
Description
Cisco Certification
Prepare for your CCIE Data Center practical exam with hands on lab
exercises running on a dedicated comprehensive topology
Obtain the skills to deploy complex virtualized Data Center Fabric and
Computing environments with Nexus and Cisco UCS.
Learn basic data center technologies and how to build a data center
infrastructure.
Get a deep understanding of the Cisco data center product line including
the Cisco Nexus9K in ACI and NexusOS modes
Description
Cisco Certification
Description
Cisco Certification
FlexPod Design Specialist;
FlexPod Implementation &
Administration Specialist
http://www.testbells.com/400-101.html