BRKARC-3322 - Catalyst 6500 IP Multicast Architecture (2011 Las Vegas)

Session Assumptions
You already have an understanding of

standard-based (platform-independent)
IP Multicast technologies
 You already have an understanding of
hardware-based (platform-dependent)
Catalyst 6500 architecture
General IP Multicast architecture covered in:
IPM-1261: Introduction to IP Multicast
Advanced IP Multicast architecture covered in:
IPM-2008: Advanced Topics in IP Multicast Deployment
General Catalyst 6500 architecture covered in:
RST-3465: Cisco Catalyst 6500 Switch Architecture
Presentation_ID © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 2
Session Goals
To provide a thorough understanding

of Catalyst 6500 hardware IP Multicast
architecture, packet flow & monitoring
 To introduce several IPv4 / IPv6 Multicast

improvements & innovations that are only
available on the Catalyst 6500 platform
Session Agenda
 Multicast Overview (~5-10m)
 Multicast Architecture (~15-20m)
 Multicast Forwarding (~10-15m)
 Multicast Replication (~10-15m)
 Multicast Packet Walks (~20-30m)
 Multicast on VSS (~15-20m)
 Multicast Debugging (~10-15m)
This is YOUR session:

Please Ask Questions!
Multicast Overview
PIM Distribution Modes
• PIM Distribution Modes

• IGMP / MLD & Snooping
• Multicast VPN & Extranet
• Multicast High-Availability
• Multicast Management
• Catalyst 6500 & IP Multicast
PIM Dense Mode (PIM-DM)
 Flood & Prune (S,G based) multicast Source Tree Source Tree
Rooted at Rooted at
distribution protocol Source1 Source2
 Designed with the assumption that receivers Source1 Source2
are “Densely” populated in the network...

 Uses only Source-based Trees
• Distribution “trees” are unidirectional (RPF-based)
 Does NOT use Rendezvous Points (RP’s),

which makes it simple to config & deploy
 Data-driven Multicast state
• Requires Periodic “Flooding” & “Pruning” on all PIM
routers, DR Election / Asserts on each subnet
 Inefficient “Flood” style packet delivery

 Group members tracked via IGMPv1 / v2
 NOTE: PIM-SM “Auto-RP” mapping info is
distributed “densely”
Receivers
PIM Sparse Mode (PIM-SM)
 General Purpose (S,G and *,G based) Source Tree Source Shared Tree
multicast distribution protocol SPT: Rooted
RPT: Rooted
at the
at the Source
Rendezvous Point
 Designed with the assumption that receivers
are “Sparsely” populated in the network... RP
 Uses both Shared (RP) & Source Trees

• Distribution “trees” are unidirectional (RPF-based)

• Requires Source “Registration”, RP mapping, DR
Election & Asserts, and SPT switch-over
 Automatic “Source” discovery (via Registers)

 More efficient on-demand packet delivery
 Supports an arbitrary (non-symmetrical)
source and receiver distribution model
 Group members tracked via IGMPv1,
Receivers
v2, or v3 (IPv4) or MLD v1 or v2 (IPv6)
PIM Source-Specific Multicast (PIM-SSM)
Source Tree Source Tree
 Shortest Path (S,G based) distribution, Rooted at Rooted at
simplifies “one-to-many” delivery Source1 Source2
Source1 Source2
 Low-Latency variation of PIM-SM, which
assumes the “one-to-many” model
• Internet / inter-domain distribution
• Voice & Video distribution
 Uses only & Source-specific (SPT) Trees

• Requires only SSM Range, and DR Election / Asserts
 Hosts are responsible for “source discovery”

• Typically via some out-of-band mechanism
(web page, content server, etc.)
• Eliminates need for RP and Shared trees
• Eliminates need for MSDP
 Group members tracked via IGMPv3 (IPv4)

or MLDv2 (IPv6) S1,G1 S1,G1 S1,G1 S2,G1
S2,G1 Receivers
Bidirectional PIM (PIM-BiDir)
Shared Tree
 Shared Path (*,G) based distribution, ideal Source Bidirectional tree
rooted at the
for “many-to-many” applications Rendezvous Point
 Highly Scalable variation of PIM-SM, which

assumes LOTS of Sources & Receivers RP
• Intra-Domain multicast
• Database distribution
 Uses only Shared (RP-based) Trees

 Data-Flow Independent
• Pre-built (RP/DF-based) distribution trees, so there are
NO source registers, DR asserts, or non-RPF issues
 Drastically reduces network mroute state

• Eliminates ALL (S,G) mroute state in the PIM network
• Permits virtually UNLIMITED Sources
• Source multicast traffic can flow both Up & Down
Sources/Receivers
(Bidirectional) on the Shared (RP-based) tree
 Group members tracked via IGMPv1 or v2

Multicast Overview
IGMP / MLD Snooping

IGMP & MLD
 Purpose: Signal & refresh Multicast group

“membership”, on receiver subnets
 Internet Group Management Protocol (IGMP)
• host membership reporting in IPv4
 Multicast Listener Discovery (MLD)
• host membership reporting in IPv6
 IGMP v1 & v2 support for PIM-SM & PIM-BiDir

 IGMP v3 support for PIM-SSM
• Option for “SSM Mapping”, which translates IGMPv2
(*,G) joins into PIM-SSM (S,G) joins
 MLD v1 & v2 support for IPv6 PIM-SM

 MLD v2 protocol support for IPv6 PIM-SSM
• Option for “SSM Mapping” to translate MLDv1 (*,G) Remember: IGMP & MLD
joins into IPv6 PIM-SSM (S,G) joins actually operate on Layer 3
(PIM-enabled) IP interfaces
IGMP / MLD & Snooping
Source Source
 IGMP for IPv4 & MLD for IPv6

 Layer 3 IGMP / MLD on all PIM-enabled Interfaces
• IGMP v1/v2/v3
• MLD v1/v2
• SSM mapping
 Layer 2 IGMP / MLD “Snooping” constrains

multicast flooding on L2 switchports, Join / Leave
Reports
 based on “snooped” membership reports
 Remember: Snooping leverages both L2 & L3

and Hardware & Software details
• Support for IGMPv1/2/3 and MLDv1/v2
Receivers
• PFC/DFC performs hardware “redirection” of all IGMP
& MLD packets to SP CPU for processing
Multicast Overview
Multicast VPN & Extranet

Multicast over MPLS VPN (MVPN)
Receiver
CE
Data MDT
Provides Multicast
CE Used for distribution over
high-bandwidth
data stream
an MPLS core
Join “Overlay” model encapsulates
VPN multicast traffic in GRE
tunnels, for transport over core
Enterprise MVPN tunnels connect remote
PE MPLS Core PE’s via Multicast Distribution
Receiver Trees (MDT’s)
Default MDT Creates fully meshed PE-PE
“Default MDT’s” for control-
plane data & low-bandwidth
streams
Can also create on-demand
“Data MDT’s” to constrain high-
Join bandwidth streams to only PE’s
with interested receivers
Default MDT Receiver

Used for
Traffic low-bandwidth
& control-traffic
High-Rate only
Source Shared / Source Tree
Rooted at RP or Source:
controls multicast distribution
MVPN Source Tree within individual VPN/VRF
Rooted at Source PE:
controls multicast distribution
within MPLS VPN core
MVPN Extranet
Join Receiver
Extranet MVPN
Same mroute,
extranet OIF
MPLS Core
Join
Traffic
Receiver
Source
 Allows for Multicast content that originated in one MVPN to be

distributed to other (separate) MVPN’s
 Two basic configuration requirements:
Configure all relevant MVRF’s on Source PE or Receiver PE (or both)
Configure “Fallback” static mroute (Source VRF IP subnets: Source and / or
RP address) to allow RPF check between MVRF’s.
Multicast Overview
Multicast High-Availability

Multicast HA - SSO/NSF
 Multicast Non-Stop Forwarding (NSF) Multicast Data

with Stateful Switch-Over (SSO) Structures
Active
 Offers single-node Multicast HA on
Catalyst 6500 with Supervisor 720
 Synchronizes all critical multicast
forwarding state & entries from the
Active to Standby supervisor
Global
• Multicast (MMLS/MFIB) Forwarding Entries Sync Periodic Syncs
• Multicast Expansion Table (MET) - for MMLS Standby

• IGMP / MLD Snooping state
 Minimizes multicast traffic disruption

during supervisor switchover
 Works with PIM “Triggered Joins”
Multicast HA -Triggered PIM Joins
 Use-Case: Any SSO/NSF capable “highly RP

available” multicast deployment
 Helps rebuilds the multicast routing “state”
very quickly after switchover
 On switchover, new Active supervisor Active
Standby
changes “Generation ID” in PIM Hellos Active
Failure
 PIM neighbors use the new GenID as an
indication that the neighbor on that Interface
has no multicast state
 Neighbor sends “triggered” PIM Join
messages for all (*,G) & (S,G) mroutes,
which have that PIM Interface as the RPF
Interface
 Assumption is that all PIM neighbors know Periodic PIM Hellos
how to interpret GenID field in PIM hellos Periodic PIM Joins
Hello with new GenID
Triggered PIM Joins
Multicast Overview
Multicast Management

• IGMP/MLD & Snooping
Multicast Management -
NDE with NetFlow v9
NetFlow
 Provides wire-rate statistics Collector
collection for IPv4 Multicast
data flows
 Support for both ingress &
egress multicast statistics
• Ingress NetFlow tracks multicast
traffic received on the Incoming
S1,G1
Interface (IIF) NetFlow v9
Export
S2,G1
• Egress NetFlow tracks multicast
traffic replicated on each Outgoing S1,G2
Interface (OIF) Egress
S3,G1
 Netflow “Sampling” support, Voice vlan100
to reduce export load Ingress
 Important for PIM-BiDir Video vlan200

networks, where (S,G) g1/1
entries NOT in mroute table
Data Catalyst 6500 vlan300
Multicast Management -
SNMP MIBs & Traps
 Several Multicast-specific SNMP
SNMP MIB’s Server
 PIM, MROUTE, MSDP, MVPN &

IGMP/MLD
 Several Multicast-specific
SNMP Traps
• pimNeighborLoss
• ciscoPimRPMappingChange SNMP Traps
• ciscoPimInvalidReFaster S1,G1
• ciscoPimInvalidJoinPrune SNMP MIBs S2,G1
• ciscoPimInterfaceUp
S1,G2
• ciscoPimInterfaceDown Egress
• msdpEstablished1 S3,G1
• msdpBackwardTransition
• ciscoMvpnMvrfChange Voice vlan100
Ingress
 Special Trap called “Multicast
Heartbeat” can be used to track Video vlan200
availability of a Multicast group g1/1
 ciscoIpMRouteMissingHeartBeats
vlan300
Data Catalyst 6500
Multicast Overview
Catalyst 6500 & IP Multicast

• IGMP/MLD & Snooping
Catalyst 6500 - Cisco’s Premiere
Multicast Switching Platform
 Leveraging 10+ years of IPv4 & IPv6 Multicast

expertise, innovation & solutions
 Delivers unmatched Multicast performance & pricing
• 200+Mpps IPv4 multicast forwarding demonstrated in EANTC tests
• Up to 80G multicast “replication” per-slot, with WS-6908-10GE module
 Deployed in the world’s most business-critical networks

• Global Financials - Exchanges, Brokerages, FSP’s
• Video Providers - Domestic & Foreign IPTV and Triple-Play ISP’s
• Service Providers - L2 & L3VPN IP Multicast Services
• Transportation - Airlines, Buses and Trains
• Government - Federal and Defense Departments
Catalyst 6500 IP Multicast
 Implements both Centralized & Distributed

IPv4 & IPv6 multicast hardware-switching
WS-SUP32-GE-3B
Off-loads the majority of multicast routing & switching
decisions from the software control-plane
 Supports IPv4 / IPv6 PIM-SM & SSM (S,G)

mroute forwarding in hardware
 Supports IPv4 PIM-SM & BIDIR (*,G) mroute WS-SUP720-3B
forwarding in hardware
 Supports IPv4 MVPN forwarding in hardware
 Supports IGMPv1/v2/v3 Snooping in hardware
 Supports MLDv1/v2 Snooping in hardware
VS-S720-10G-3C
 Supports IPv4 PIM Snooping in hardware
 Supports both Centralized & Distributed
IPv4 & IPv6 multicast packet-replication
Off-loads the majority of data-plane packet forwarding burden
from the software control-plane
VS-SUP2T-10G
Sup2T/PFC4 - Enhancements
FIB-XL  Provides for unparalleled

 Up to 256K Multicast Entries in the FIB-
Hardware Multicast Scaling!
 PIM “Source Register” in Hardware: all register  Saves CPU & Memory usage &
frames are encapsulated & de-
de-encapsulated by PFC minimizes Source Register time!
 PIM “Dual
“Dual--RPF” in Hardware:
Hardware: which allows SPT  Saves CPU & Memory usage &
switch--over (RP to Source) to occur w/o software
switch minimizes SPT Switchover time!
 New MFIB & EDC (+ LTL & MET sharing) designs

designs::  Optimized HW Infrastructure,
which Incorporate the new “LIF & BD” capabilities designed for L2/L3 Scalability!
 New Global IGMP Snooping & Optimized Multicast  Enhanced Layer 2 multicast
Flood (OMF) designs: for L2 multicast processing & forwarding!
Egress-Replication support  Optimized MVPN Replication!

 Multicast VPN (MVPN) Egress-
 Store IPv6 (*,G) & (S,G) entries in FIB (vs. Netflow)  Much faster forwarding of IPv6!
 Hardware IGMPv3 & MLDv2 “Snooping”

“Snooping” support  Faster updates of L2 IPv4 &
IPv6 “PIM-
“PIM-SSM” host tables!
Sup2T/PFC4 - More Enhancements
 Some Other Enhancements (Not Listed Previously)…

Previously)…
 PFC4 supports 8 PIM-
PIM-Bidir RPDF entries (vs. PFC3 supports 4).
 IP based (vs. DMAC based) L2 lookups, eliminates IP-

IP-MAC “address aliasing”.
 CoPP has special exception cases & granular rate-

rate-limit support, for IP Multicast.
 NetFlow (V9 & FnF

FnF)) has special fields & processing support, for IP Multicast.
 Multicast HA uses new MFIB & EDC, LTL & MET Sharing, BD & LIF infrastructure.
 Multicast Service Reflection (MSR) support for up to 64K translations.
 Consistent (IPv4 & IPv6) MFIB-

MFIB-based CLI & MIB, for monitoring & statistics.
 Hardware Integration with VPLS, H-

H-VPLS, and EoMPLS features*.
 Enhanced Multicast over GRE processing & forwarding performance.
 Improved TCAM-
TCAM-FULL handling, and (last
(last--resort
resort)) SW-
SW-based forwarding.
 And Others...
Multicast Architecture
Multicast Control-Plane
• Multicast Control-Plane
• Supervisor Components
• Module Components
Supervisor 720 - Multicast Control Plane
 Route Processor - Layer 3  Switch Processor - Layer 2

control-plane functions: control-plane functions:
• Multicast Protocols: PIM, IGMP / MLD, • Managing PFC / DFC hardware tables
AutoRP / BSR, MSDP, MVPN, etc. • IGMP / MLD Snooping packet processing
• Calculating RPF interfaces • PIM Snooping / RGMP packet processing
• Managing software IP mroute table • IGMP / MLD Querier functions
• Downloads IP mroute entries to SP (for • Statistics Collection & Reporting
installation into PFC / DFC hardware)
IP Mroute
Table
RP
Group
1 Gbps (Route Processor) Membership
CPU
Inband RP & Source
Advertisement
Channels
SP IGMP / MLD
1 Gbps (Switch Processor) Snooping
CPU PIM
Snooping
Supervisor 720 - Multicast Components
720G Switch Fabric & PFC3 Daughter Card

Multicast Replication Engine
Route Processor (RP)
Switch Processor (SP)
Supervisor 2T - Multicast Control Plane
 Route Processes - Layer 3  Switch Processes - Layer 2

control-plane functions: control-plane functions:
• Multicast Protocols: PIM, IGMP / MLD, • Managing PFC / DFC hardware tables
AutoRP / BSR, MSDP, MVPN, etc.
Same Basic •Principle:
IGMP / MLD Snooping packet processing
• Calculating RPF interfaces • PIM Snooping / RGMP packet processing
Separate Routing & Switching IOS processes
• Managing software IP mroute table
(with Single• RP
IGMPCLI)/ MLD Querier functions
• Downloads IP mroute entries to SP (for • Statistics Collection & Reporting
installation into PFC / DFC hardware)
IP Mroute
Table
RP
Group
1 Gbps (Route Process) Membership
Single (Dual-Core)
FunctionsCore)
Inband RP & Source
Advertisement
Channel Control-Plane
SP
Processor IGMP / MLD
(Switch Process) Snooping
Functions
PIM
Snooping
Supervisor 2T “Multicast” Components
2T Switch Fabric &
Multicast Replication Engine PFC4 Daughter Card
Combined
(SP/RP) Control-
Control-
Plane Processor
Supervisor Components
• LAN Module Components
Supervisor 720 Multicast Architecture
PFC3 provides Hardware 4 entry Bidir RP RP & SP CPUs are
L2-L3/L4 forwarding, using table for DF housed on the same
TCAM’s & ASICs Interface check Daughter Board
SP CPU programs
FIB contains ADJ contains RP CPU maintains
hardware and
(S,G) & (*,G) entries, Rewrite info & routes & state, and
performs IGMP/MLD
and RPF VLAN MET index downloads tables
& PIM snooping
Supervisor 720 Baseboard 1GE Uplinks
Counter QoS FIB

ADJ
RP (MSFC3)
FPGA TCAM TCAM 1 Gbps CPU DRAM
Port ASIC
Bidir RP 1 Gbps SP (NMP) DRAM
ACL L3/4 CPU
TCAM NetFlow
Engine CPU Daughter Card
MET
Protocol L2 Engine PFC3
Redirection Integrated
L2 CAM
Daughter Fabric 20 Gbps
720 Gbps
Logic Card Interface & Switch Fabric
Replication
Engine … 18 x 20Gbps
Fabric Channels
Protocol 32 Gbps Bus
Redirection DBUS Central Replication Engine
captures RBUS
IGMP/MLD & PIM 64K L2 CAM contains
16 Gbps MET contains OIL’s for Bus-based cards and
multicast MAC entries for replication uplink ports
Bus
Supervisor 32 Multicast Architecture
RP and SP
PFC3B is the same CPUs perform
as on Supervisor 720 all control
plane functions
Supervisor Engine 32 Baseboard 10GE Uplinks
Counter QoS FIB

ADJ
FPGA TCAM TCAM 1 Gbps SP CPU DRAM
Port Port
ASIC ASIC
Bidir RP 1 Gbps
ACL L3/4 RP CPU DRAM
TCAM Engine NetFlow
FPGA
MSFC2a Daughter Card
MUX
Protocol L2 Engine PFC3B
Redirection Daughter Bus Interface &
Logic L2 CAM Replication
Card
Engine
No Switch Fabric
MET
32 Gbps Central Replication Engine

DBUS
Bus RBUS performs replication for
the ENTIRE system
Supervisor 720-10GE Multicast Architecture
RP and SP
PFC3C serves same CPUs perform
basic functions as all control
PFC3A/B plane functions
Supervisor 720-10GE Baseboard 1GE & 10GE Uplinks
Counter QoS FIB

ADJ
RP (MSFC3C)
FPGA TCAM TCAM 1 Gbps CPU DRAM
Port Port
ASIC ASIC
Bidir RP 1 Gbps SP (NMP) DRAM
ACL L3/4 CPU
TCAM NetFlow
Engine
MET CPU Daughter Card
Protocol L2 Engine PFC3C Integrated

Redirection Daughter Fabric 20 Gbps
Logic L2 CAM Card 720 Gbps
Interface &
Switch Fabric
Replication
Fabric Channels
32 Gbps Bus
DBUS Central Replication engine
96K L2 CAM for more RBUS for Bus-based cards and
multicast MAC entries uplink ports
Supervisor 2T Multicast Architecture
PFC4 builds upon PFC3 and 8 entry Bidir RP table RP (combined L2/L3)
provides new IFE/OFE for DF Interface CPU is housed on the
processing capabilities check MSFC5 Daughter Board
RP (L2) programs RP (L3) maintains
LDB contains ADJ / RIT contains
hardware and routes & state, and
Ingress & Egress Rewrite Info & MET
performs IGMP/MLD & downloads tables
LIF/BD mapping index
PIM snooping
Supervisor 2T Baseboard 1GE & 10GE Uplinks
QoS FIB
ADJ RIT
TCAM TCAM 1 Gbps RP DRAM
Port ASIC CPU
ACL
TCAM Bidir RP
L3/4
Counter Engine NetFlow
MSFC5 Daughter Card
FPGA
MET
LIF Database L2 Engine PFC4 Integrated

Daughter Fabric 20 Gbps
Protocol L2 CAM 2Tbps Gbps
Redirection Card Interface & Switch Fabric
Replication
Fabric Channels
Protocol 32 Gbps Bus

Redirection DBUS Central Replication engine
captures 128K L2 CAM for more
RBUS for Bus-based cards and
16 Gbps MET contains OIL’s
IGMP/MLD & PIM multicast MAC entries for replication
Bus uplink ports
Multicast on a “Crossbar” Switch Fabric
Stock Stock Stock Stock
Join
Join
Join
Join
Broker Broker Broker Broker
C D B A
Port
Port
Port
Port
Port
Port
N
N
0
0
(Tx)
(Rx)
(Tx)
(Rx)
(Tx)
(Rx)
FE
RE
FE
RE
FE
RE
CPU
CPU
CPU
Fabric ASIC Fabric ASIC Fabric ASIC
S
c
h
CPU e
Port
0
RE
(Tx) d
Fabric ASIC
u
Source Port RE l
N (Rx)
Stock e
Exchange
Host r
LAN Module Components
• LAN Module Components
Classic Module Architecture
DBUS Example: WS-X6416-GBIC

RBUS
Classic Module
Port Port Port Port

ASIC ASIC ASIC ASIC
12xGE 12xGE 12xGE 12xGE
Example: WS-X6148A-RJ-45
Port ASIC for Physical DBUS

Connectivity, Buffering, RBUS
and Queuing
Classic Module
Port
ASIC
48x10/100
No multicast-specific components on these modules

CEF256 Module Architecture
Example: WS-X6516-GBIC
8 Gbps
DBUS
RBUS Fabric Channel
CEF256
Module
Fabric / Bus
Interface
Fabric Interface LC-DBUS

Connects to
LC-RBUS
Fabric and Bus
Replication
MET
Engine Port Port Port Port
ASIC ASIC ASIC ASIC
MET 4xGE 4xGE 4xGE 4xGE
Contains
Multicast Replication
OILs for Port ASIC for Physical
Engine - Replicates Packets
Replication Connectivity, Buffering,
to each OIF
and Queuing
CEF256 Module Architecture + DFC
Layer 2 Engine for

Example: WS-X6516-GBIC with WS-F6K-DFC
MAC learning,
Security / QoS ACL
Lookups, and
Protocol Redirection 8 Gbps Layer 3
Fabric Channel Engine for
CEF256 FIB/ADJ and
Layer 2 L3/L4 Module NetFlow
Fabric / Bus with DFC Lookups
Engine Engine
Interface
DFC
LC-DBUS
LC-RBUS
Replication
MET
Engine Port Port Port Port
ASIC ASIC ASIC ASIC
4xGE 4xGE 4xGE 4xGE
CEF720 Module Architecture
Example: WS-X6748-SFP
DBUS
20 Gbps RBUS 20 Gbps
Fabric Channel Fabric Channel
Complex A Complex B CEF720
Module
Fabric Bus Bus Fabric
Interface & Interface Interface Interface &
MET Replication Replication MET
Engine Engine
Port Port CFC Port Port

ASIC ASIC ASIC ASIC
Port ASIC for
MET Contains Fabric Interface & Physical
local OIF’s for Replication Engine - Connectivity,
Replication Connects to Fabric & Bus Buffering &
(CFC), and Replicates Queuing
Packets to OIL’s
Bus Interface for Control Data Only!!
CEF720 Module Architecture + DFC3/4
Example: WS-X6748-SFP with WS-F6700-DFC3B
Layer 2 Engine
for L2 Lookups
and Protocol Layer3/4 Engine for FIB/ADJ,
Redirection ACL, QoS, and NetFlow Lookups
20 Gbps 20 Gbps
Complex A Complex B CEF720
Module
Fabric Layer 2 Layer 2 Fabric + DFC
Interface & Engine Engine Interface &
MET Replication Replication MET
Engine L3/4 Engine
Engine
Port Port DFC Port Port
ASIC ASIC ASIC ASIC
CEF2T Module Architecture + DFC4
Example: WS-X6908-10G with WS-F6K-DFC4-E
Layer 2 Engine
for L2 Lookups
and Protocol Layer3/4 Engine for FIB/ADJ,
Redirection ACL, QoS, and NetFlow Lookups
40 Gbps 40 Gbps
Fabric Complex A Complex B Fabric CEF2T
MET ASIC MET MET ASIC MET Module
Layer 2 + DFC
Fabric Fabric Engine Fabric Fabric
Interface & Interface & Interface & Interface &
Replication Replication Replication Replication
Engine Engine L3/4 Engine Engine
Engine
Port Port Port Port DFC4 Port Port Port Port
ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC
1x10G 1x10G 1x10G 1x10G 1x10G 1x10G 1x10G 1x10G
Multicast Forwarding
Hardware Forwarding
• Hardware Forwarding
• Multicast FIB Processing
• PIM BiDir Processing
• Multicast Expansion Table
Supervisor 720 - MMLS
RP (MSFC)  RP CPU derives 3 important data

IP Mrouting,, structures from the IP Multicast
PIM, IGMP
routing table
MMLS-RP MDSS & MLD
• Multicast FIB: Contains (S,G) & (*,G)
entries, RPF VLAN, and BiDir RP index
SCP / ICC • Adjacency Table: Contains L2 rewrite
MAC information and MET index
SP (PFC) • Multicast Expansion Table (MET):
MET Manager, Contains Output Interface Lists (OILs),
IGMP & PIM i.e. Interfaces requiring replication
MMLS-SP Snooping
 RP CPU downloads tables to SP
 SP CPU installs tables into the
SCP appropriate hardware
• MET installed in replication engines
LC (CFC) LC (DFC)
• Multicast FIB and Adjacency tables
MET FIB/ADJ MET installed in PFC/DFC hardware
 SP CPU also maintains L2 tables
MMLS-LC MMLS-LC
for IGMP/MLD & PIM snooping
Sup720 & 32 - Mroute CLI
When the RPF “Multicast

Fast Drop” (RPF_MFD) flag is
6500# sho ip mroute 239.1.1.101 set, no matching traffic will be
sent to RP CPU
IP Multicast Routing Table
The SP Cperiodically
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, - Connected,updates
multicast statistics allowing
L - Local, P - Pruned, R - RP-bit set, F - Register
RP flag,
to refresh activity timers
T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
U - URD, I - Received Source Specific Host Report,
Z - Multicast Tunnel, z - MDT-data group sender,
Y - Joined MDT-data group, y - Sending to MDT-data group
V - RD & Vector, v - Vector
Outgoing interface flags: H - Hardware switched, A - Assert winner
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode
(101.1.1.1, 239.1.1.101), 00:05:22/00:02:53, flags: sTI

Incoming interface: Vlan101, RPF nbr 0.0.0.0, RPF-MFD
Outgoing interface list:
Vlan201, Forward/Sparse, 00:02:33/00:02:41, H
6500# “H” Flag is set to show

that OIF is switching in
Hardware
Sup720 & 32 - MMLS CLI
6500#sho mls ip multicast group 239.1.1.101

Multicast hardware switched flows:
(101.1.1.1, 239.1.1.101) Incoming interface: Vlan101, Packets switched: 57926515914
Hardware switched outgoing interfaces:
Vlan201
RPF-MFD installed (S,G) OIF’s (S,G) IIF
Total hardware switched flows : 1

6500#
(S,G) Packet
6500#sho mls ip multi stat group 239.1.1.101 Statistics
Group: 239.1.1.101
Source: 101.1.1.1, bytes: 75073226997168, packets: 57926872683
6500#
(S,G) Byte &
Packet Statistics
Supervisor 2T - MFIB
MSFC  RP CPU derives 3 important data

IPv4 & IGMP & MET MGR
structures from the IP Multicast
v6 PIM MLD
LTL MGR routing table
• Multicast FIB: Contains (S,G) & (*,G)
entries, RPF VLAN, and BiDir RP index
MRIB EDC Server
• Adjacency Table: Contains L2 rewrite
MAC information and MET index
I • Multicast Expansion Table (MET):
P PFC Contains Output Interface Lists (OILs),
C EDC
MFIB
Client i.e. Interfaces requiring replication
&  RP builds MRIB state in software
I MET MGR LTL RAM  MFIB uses MRIB to program
C forwarding information in hardware
C
DFC • Multicast FIB & Adjacency tables
MET MGR LTL RAM installed in PFC/DFC hardware
 EDC uses MRIB to program
MFIB EDC Client hardware (OIF) port information
 MET defines per-module OIF’s
Sup2T - Mroute CLI
6500#sho ip mroute 239.124.1.1
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
L - Local, P - Pruned, R - RP-bit set, F - Register flag,
T - SPT-bit set, J - Join SPT, M - MSDP created entry, NOTE: the RPF_MFD flag
E - Extranet,
no longer applies, because
X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
all flows are completely
U - URD, I - Received Source Specific Host Report, hardware switched
Z - Multicast Tunnel, z - MDT-data group sender,
Y - Joined MDT-data group, y - Sending to MDT-data group,
V - RD & Vector, v - Vector
Outgoing interface flags: H - Hardware switched, A - Assert winner
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode
(*, 239.124.1.1), 00:00:24/stopped, RP 3.3.3.3, flags: SJC

Incoming interface: Null, RPF nbr 0.0.0.0
Vlan148, Forward/Sparse, 00:00:11/00:02:48
(124.1.1.1, 239.124.1.1), 00:00:24/00:02:35, flags: T

Incoming interface: Vlan124, RPF nbr 0.0.0.0
Vlan148, Forward/Sparse, 00:00:11/00:02:48
6500#
Sup2T - MFIB CLI
6500#sho ip mfib 239.124.1.1
Entry Flags: C - Directly Connected, S - Signal, IA - Inherit A flag,
ET - Data Rate Exceeds Threshold, K - Keepalive
DDE - Data Driven Event, HW - Hardware Installed
I/O Item Flags: IC - Internal Copy, NP - Not platform switched,
NS - Negate Signalling, SP - Signal Present,
A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,
MA - MFIB Accept
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts: Total/RPF failed/Other drops
I/O Item Counts: FS Pkt Count/PS Pkt Count
Default
(*,239.124.1.1) Flags: C HW
SW Forwarding: 0/0/0/0, Other: 0/0/0
HW Forwarding: 0/0/0/0, Other: 0/0/0 Hardware Flag
Tunnel1 Flags: A
Vlan148 Flags: F NS
Pkts: 0/0 (S,G) Byte &
(124.1.1.1,239.124.1.1) Flags: HW Packet Statistics
SW Forwarding: 0/0/0/0, Other: 6/0/6
HW Forwarding: 182188/1001/1318/10307, Other: 0/0/0
Vlan124 Flags: A (S,G) IIF
Vlan148 Flags: F NS denoted by
Pkts: 0/0 (S,G) OIF’s the A flag
denoted by the
F & NS flags
6500#
Multicast FIB Processing
Multicast FIB TCAM Lookup
Compare 3
Generate
Lookup
Key
2010::1, FF05::1
2000::1, FF02::1 1
Lookup Key 2
2001::1, FF02::1 2
FFFFFFFF FFFFFFFF 2003::1, FF05::1 3

(S,G)
1 2010::1, FF05::1 2002::1, FF05::1 4
5 Result
Multicast 2003::1, FF06::1
Packet HIT! 46 RPF VLAN
2010::1, FF05::1 Adj Index
2010::1, FF06::1 7
S,G compares 2020::1, FF05::1 8
all bits in Masks FIB TCAM Patterns
Source IP and
Group IP
Replication Engine(s)
MAC, MET Index
OIL #1
OIL #2 MAC, MET Index
6 5
OIL #4
MAC, MET Index
MET
Adjacency Table
Sup720 & 32 - Hardware Shortcuts
Multicast
6500-sp#sho mls cef ip multicast source 192.168.6.2 group 226.1.1.1 detail
Source
ource
Multicast CEF Entries for VPN#0
& Group
(192.168.6.2, 226.1.1.1) Primary Input Entry address
IOSVPN:0 (1) PI:1 (1) CR:0 (1) Recirc:0 (1) Flag
Vlan:1019 AdjPtr:376832 FibRpfNf:1 FibRpfDf:1 FibAddr:0x4060
RPF rwvlans:1019 rwindex:0x7FFA adjmac:000b.45a8.1280 rdt:1 E:0 CAP1:0 MET
VLAN fmt:Mcast l3rwvld:1 DM:0 mtu:1518 rwtype:L2&L3 met2:0x4 met3:0x1746 indexes
packets:0000750657305 bytes:000000549260903076
Starting Offset: 0x0004
V E C:1014 I:0x07F09 Adjacency index
Starting Offset: 0x1746 Packets & Bytes switched associated with the
V C:1022 I:0x00A2D P->2C4C on the adjacency Multicast FIB entry
V C: 11 I:0x0219F
V E C: 12 I:0x019CF MET blocks for SP
replication engine
Found 1 entries. 0 are mfd entries
6500-sp#sh mls cef adjacency entry 376832 detail

Index: 376832 smac: 000b.45a8.1280, dmac: 0000.0000.0000
mtu: 1518, vlan: 1019, dindex: 0x7FFA, l3rw_vld: 1
format: MULTICAST, flags: 0x2000004608
met2: 4, met3: 5958
packets: 839836755, bytes: 555325105676
6500-sp#
Sup2T - Hardware Shortcuts
6500#sho plat hard multic rout ip source 124.1.1.1 gro 239.124.1.1 det
IPv4 Multicast CEF Entries for VPN#0 Multicast
Source
ource
(124.1.1.1, 239.124.1.1/32) & Group
RPF address
FIBAddr: 0x1BE02 IOSVPN: 0 RpfType: SglRpfChk SrcRpf: Vl124 VLAN
CPx: 0 s_star_pri: 1 non-rpf drop: 0
MET
PIAdjPtr: 0x34334 Format: IP rdt: off elif: 0xC5408 indexes
fltr_en: off idx_sel/bndl_en: 0 dec_ttl: on mtu_idx: 2(1518)
PV: 1 rwtype: MCAST_L3_RWT_L2_EXPS
met3: 0x5 met2: 0x4
Packets: 346661 Bytes: 456899198 Adjacency indexes
associated with the
NPIAdjPtr: 0x34335 Format: IP rdt: on elif: 0xC5408 Multicast FIB entry
fltr_en: off idx_sel/bndl_en: 0 dec_ttl: off
PV: 0 rwtype: MCAST_L3_REWRITE
met3: 0x5 met2: 0x0 DestNdx: 0x7FF3 Packets & Bytes switched
Packets: 352132 Bytes: 464109976 on the adjacency
MET offset: 0x5
OIF AdjPtr Elif CR
+-------------+----------+-----------+--------+ MET blocks & ICROIF
Vl148 0x8094 0x94 1T2 for egress replication
MET offset: 0x4
OIF AdjPtr Elif CR
+-------------+----------+-----------+--------+
EDT-3C334 0x3C334 0x8400A 1T1/T2 5T1 7T1/T2
Found 1 entries.
6500#
PIM-BiDir Processing
• Multicast Forwarding (15-20m)

Bidir FIB TCAM Lookup
Compare 3 Bidir RP Table
Generate 5
Lookup 10.1.1.10,
XXXXXX, 239.1.1.1
239.1.1.1 RP #1 DF Set
Key xxxxxxxx, 239.1.1.99 1
Lookup Key 2 RP #2 DF Set
xxxxxxxx, 239.1.16.1 2
RP #3 DF Set
00000000 FFFFFFFF xxxxxxxx, 239.2.1.1 3
S,G RP #4 DF Set
1 10.1.1.10, 239.1.1.1 xxxxxxxx, 239.14.14.1 4
Multicast xxxxxxxx, 239.44.12.1 5
Packet Bidir RP
HIT! xxxxxxxx, 239.1.1.1 6 Index
4 Adj Index
xxxxxxxx, 239.1.11.1 7
Result
xxxxxxxx, 239.50.1.1 8
*,G compares
all bits in Masks FIB TCAM Patterns
Group IP
Replication Engine(s)
MAC, MET Index
OIL #1
7 6
OIL #4
MAC, MET Index
MET
Adjacency Table
Sup720 Bidir RP-to-DF Mapping Table
Step 4, from FIB TCAM lookup
RP
Index DF Interface Set
0 1 2 3 4 5 6 … 4093 4094
1 1 2 3 4 5 6 … 4093 4094
2 1 2 3 4 5 6 … 4093 4094
3 1 2 3 4 5 6 … 4093 4094
4K VLAN IDs
RP RP IP DF
Index Address Interfaces
RP #1 DF Set
0 10.1.1.1 VLAN 1 2 3
RP #2 DF Set
1 10.2.2.2 VLAN 3 4 5 6
RP #3 DF Set
2 10.3.3.3 VLAN 3 4093
RP #4 DF Set
3 10.4.4.4 VLAN 200 201
Bidir RP Table
(PFC/DFC) Mapping Table
(Software Data Structure)
Multicast Expansion Table
Multicast Expansion Table (MET)
Final Step of Forwarding Lookup
Index OIF VLAN Port List
Index A from ADJ A 100 Gi4/12,Gi4/19
101 Gi9/1 MET Block

102 Gi9/25,Gi9/29
Index B from ADJ B 100 Gi4/12,Gi4/22,Gi4/30
1019 Te1/1
Index C from ADJ C 700 Gi4/1,Gi4/2,Gi4/3
4030 Te8/1
Replication MET 4031 Te8/3

Engine
4032 Te2/1,Te2/2,Te2/9
MET Information
6500-sp#sh mls cef ip multi source 192.168.6.2 group 226.1.1.1 detail

(192.168.6.2, 226.1.1.1)
IOSVPN:0 (1) PI:1 (1) CR:0 (1) Recirc:0 (1)
rwvlans:1019 rwindex:0x7FFA adjmac:000b.45a8.1280 rdt:1 E:0 CAP1:0
fmt:Mcast l3rwvld:1 DM:0 mtu:1518 rwtype:L2&L3 met2:0x4 met3:0x39E
packets:0000716098029 bytes:000000632810218228
V E C:1014 I:0x07F09
Starting Offset: 0x039E MET Indexes
V C: 11 I:0x0219F P->2C48
V E C: 12 I:0x019CF
OIF VLAN Port Indexes
6500-sp#test mcast rd-met slot 1 addr 4

V E C:1014 I:0x07F09
6500-sp#
Replication Overview
• Replication Overview
• Ingress Replication Mode
• Egress Replication Mode
• Egress-Local Optimization
 It is the process of creating “copies” of

multicast packets, one for each Layer 3 OIF
• For example: PIM-SM (S,G) mroute, with 3 OIFs - replication
hardware creates three egress copies of every one ingress
packet received from source (S) & destined to group (G)
• There are 3 basic types of hardware replication
 Layer 3 multicast replication
• For Layer 3 Forwarding between (1 or many) interfaces
• Multicast “Replication Engine” does the replication
 Layer 2 multicast replication
• For Layer 2 Forwarding between ports in same LC
• Port ASIC does the replication
 Fabric multicast replication
• For Egress Forwarding between ports in different LC’s
• Fabric ASIC does the replication
Multicast Replication Modes
 The Replication “mode” refers to

exactly where within the system
multicast “replication” occurs
 Replication occurs in one or more
hardware “replication-engine” ASIC’s
 In a Classic (bus) system, replication
occurs centrally on the Supervisor
replication-engine ASIC
 In a Fabric-enabled system, there are
two possible replication modes:
•Ingress replication
•Egress replication
Ingress Replication Mode
• Egress Replication mode
Ingress Replication-Mode
 Supported on Supervisor 720’s

Three Packets Cross
& Supervisor 2 + SFM. Switch Fabric
 Requires Fabric-enabled modules

 Replication load is Distributed: B
Supervisor and / or Switching modules RE
can perform replication
A
 Replication engine on the ingress C
module performs replication for all Switch
RE
OIF’s (MET2/3) Fabric RE
 Input and replicated packets get D
lookup on PFC or ingress DFC
 Replicated copies pass over Fabric RE
to egress modules (MET2)
 Multiple MET tables, but MET on all
replication-engines are synchronized RE = Replication Engine
Ingress Replication-Mode
Three Packets Cross
Switch Fabric
6500#sho mls ip multicast capability
Current mode of replication is Ingress
Configured replication mode is Ingress B
RE
Egress Local is Disabled
Slot Multicast replication capability Egress Local A

1 Egress No
Switch C
2 Egress Yes
3 Egress No RE
Fabric RE
4 Egress Yes
6500#sho mls ip multicast summary D
15422 MMLS entries using 9869888 bytes of memory
Number of partial hardware-switched flows: 0
Number of complete hardware-switched flows: 15422 RE
Directly connected subnet entry install is enabled
Hardware shortcuts for mvpn mroutes supported
Current mode of replication is Ingress
Consistency checker is enabled RE = Replication Engine
Bidir gm-scan-interval: 10
IPv6 Multicast Hardware Uplinks: Disabled
6500-sp#sho mls cef ip multi gro 239.1.1.101 det
(101.1.1.1, 239.1.1.101)
rwvlans:101 rwindex:0x9D9 adjmac:0013.5f1d.cb80 rdt:1 E:0 CAP1:0
fmt:Mcast l3rwvld:1 DM:0 mtu:1518 rwtype:L3 met2:0x0 met3:0x3C
packets:0000000000000 bytes:000000000000000000
Starting Offset: 0x003C
V E C: 201 I:0x00A1F

6500-sp#
Egress Replication Mode
Egress Replication-Mode
 Supported on “egress-capable”
One Packet Crosses
Fabric modules Switch Fabric
• e.g. WS-X6700, 6516A-GBIC & 6548A-GETX
 Input packets get one lookup on ingress

(PFC/DFC), and replicated packets get B
lookup on egress (PFC/DFC) A RE
 For OIFs on ingress module, local RE C

Switch
performs the replication (MET2/3) RE RE
Fabric
 For OIFs on other (egress) modules, D
ingress RE replicates a single copy RE
(MET2) to Switch Fabric
 Uses an internal (Egress) VLAN, called
Internal Central Rewrite OIF (ICROIF),
which all egress modules are members of = Replication Engine
RE = Replication Engine
 The RE on the egress module will then
replicate only for local OIFs (MET3)
Egress Replication-Mode
One Packet Crosses
Switch Fabric
6500#sho mls ip multicast capability

Current mode of replication is Egress
Configured replication mode is Auto B
Egress Local is Enabled RE
A
Slot Multicast replication capability Egress Local C
Switch
1 Egress No RE
RE
2 Egress Yes Fabric
3 Egress No
D
4 Egress Yes
RE
6500-sp#show mls cef ip multicast group 225.10.10.10 detail
(172.16.25.1, 225.10.10.10)
rwvlans:20 rwindex:0x939 adjmac:001d.a29a.1f00 rdt:1 E:0 CAP1:0 = Replication
RE = Replication Engine Engine
fmt:Mcast l3rwvld:1 DM:0 mtu:1518 rwtype:L2&L3 met2:0x8A met3:0x8B
packets:0000000000049 bytes:000000000000005782
Starting Offset: 0x008A
V E L0 C:1015 I:0x0080B
Starting Offset: 0x008B
V E C: 10 I:0x0091B

rwvlans:1015 rwindex:0x7FFA adjmac:001d.a29a.1f00 rdt:1 E:0 CAP1:0
fmt:Mcast l3rwvld:1 DM:0 mtu:1518 rwtype:L3 met2:0x0 met3:0x8B
packets:0000000000000 bytes:000000000000000000
Starting Offset: 0x008B
V E C: 10 I:0x0091B

6500-sp#
Egress-Local Optimization
Egress Replication - Same Module
With pre-SXF IOS, Egress replication OIF entries Entry OIF Dest Index
are handled the same (symmetrical) among the Valid

replication-engines on the same module Index
1 2/1 2/1
Hence replication-engine usage is not optimized Module B

2 2/2 2/2
on dual-channel modules…
Receiver Green
RE 2/1
2/2
Module A DROP
Switch
Fabric 2/3
RE
RE 2/4
Source
Entry OIF Dest Index
MET entry for Non-Local port 1 2/1 2/1
RE = Replication
= Replication
EngineEngine Pre--SXF
Pre 2 2/3 2/3
MET Contents
Egress Local Replication
Egress Local replication optimizes the replication Entry OIF Dest Index
bandwidth within the same module by including only

the local (L3) ports of each Replication-Engine Valid 1 2/1 2/1
Index
2 2/2 2/2
Cat6500(config)# mls ip multicast egress local Module B
RE 2/1 Receiver Green

2/2
Module A
Switch
Fabric 2/3
RE
RE 2/4
Source
1 2/3 2/3
RE = Replication
= Replication
EngineEngine SXF--SXH
SXF
Egress Local for SVI & Etherchannel
Egress Local for SVI & Etherchannel extends the original Entry OIF Dest Index
optimization to Logical interfaces (with multiple ports), by

Valid
including only the physical port index that should receive... Index
1 2/1 2/1
2 2/2 2/2
* On by DEFAULT for Sup2T  Module B
RE 2/1 Receiver Green

2/2
VLAN
Module A
GREEN
Switch
Fabric 2/3
RE
RE 2/4
Source
1 2/3 2/3
RE = Replication
= Replication
EngineEngine Post--SXI
Post
MET Contents
Multicast Packet Walks
Putting it all together...
• Centralized Replication
• Ingress Replication
• Egress Replication
• IGMP/MLD/PIM Snooping
Centralized Replication (1)
Supervisor
Engine 32
L3/4 Engine
Orange Green Red Purple
MET R R R R
L2 Engine Port ASIC Port ASIC

2
3
Replication A B
Engine Classic
PFC3 Module B
DBUS
RBUS
Source S
Classic Receiver R
Port ASIC Port ASIC
Module A A B4
Blue VLAN
1 Orange VLAN
Blue S Blue R Green VLAN
Red VLAN
Purple VLAN
Entire Packet
Packet Header
Centralized Replication (2)
Supervisor
Engine 32
L3/4 Engine
MET R R R R
8 8 8 8
L2 Engine Port ASIC Port ASIC
6 Replication A B
7
Engine Classic
PFC3 5 Module B
DBUS
RBUS
Source S
Classic Receiver R
Port ASIC Port ASIC
Module A A B
Blue VLAN
Orange VLAN
Blue S Blue R Green VLAN
Red VLAN
Purple VLAN
Entire Packet
Packet Header
Ingress Replication (1)
R R R R
Supervisor
Engine 720 Port ASIC Port ASIC
L3/4 Engine
A B
LC-RBUS
LC-DBUS
L2 Engine 720 Gbps
3 Switch Fabric Replication CEF256
8 Gbps MET
PFC3 Fabric Interface Engine Module B
DBUS
RBUS
2
Fabric CEF256 Source S

8 Gbps Module A
Interface
4
Receiver R
Blue VLAN
LC-DBUS
Orange VLAN
LC-RBUS
Green VLAN
Replication Red VLAN
MET
Engine Port ASIC Port ASIC Purple VLAN
A B
1
Blue S Blue R Entire Packet
Packet Header
Ingress Replication (2)
R R R R
Supervisor
Engine 720 Port ASIC Port ASIC
L3/4 Engine
A B
LC-RBUS
LC-DBUS
L2 Engine 720 Gbps
7 Switch 9
Fabric Replication
8 Gbps MET CEF256
PFC3 Fabric Interface Engine Module B
DBUS
RBUS
6
Fabric CEF256 Source S
8 8 Gbps Module A
Interface Receiver R
Blue VLAN
LC-DBUS
Orange VLAN
LC-RBUS
Green VLAN
5
Replication Red VLAN
MET
Engine Port ASIC Port ASIC Purple VLAN
A B
Blue S Blue R Entire Packet

Packet Header
Egress Replication (1)
R R R R
Port ASIC Port ASIC CEF720

DFC3 Module B
Supervisor Engine 720 L3/4 Engine w/DFC3
MET
720 Gbps 20 Gbps Fabric Interface &
PFC3 Switch Replication Engine L2 Engine
20 Gbps
Fabric
20 Gbps
20 Gbps
Source S
CEF720 Receiver R
2
Module A Blue VLAN
Fabric Interface & L2 Engine w/DFC3 Orange VLAN
MET 3
Replication Engine Green VLAN
4 Red VLAN
L3/4 Engine Purple VLAN
Port ASIC Port ASIC DFC3
Internal VLAN
1 Entire Packet
Blue S R Blue Packet Header
R R R R

DFC3 Module B
MET
720 Gbps 20 Gbps Fabric Interface & 9
20 Gbps
Fabric 8
20 Gbps
20 Gbps
Source S
CEF720 Receiver R
7
5
Module A Blue VLAN
MET 6
Red VLAN
Internal VLAN
Entire Packet
R R R R

DFC3 Module B
MET 12 12
720 Gbps 20 Gbps Fabric Interface & 11
20 Gbps
Fabric 10
20 Gbps
20 Gbps
Source S
CEF720 Receiver R
Module A Blue VLAN
MET
Red VLAN
Internal VLAN
Entire Packet
Multicast Packet Walks
Putting it all together...
• Centralized Replication
• Ingress Replication
• Egress Replication
• IGMP/MLD/PIM Snooping
IGMP / MLD Snooping
 Purpose: Constrain unnecessary (default)

multicast “flooding” on Layer 2 switch-ports
 IGMP/MLD Snooping code leverages both
Hardware & Software information:
• Switch “snoops” L3 contents of IGMP / MLD packets
• PFC/DFC ASIC’s recognize IGMP / MLD packets and
“redirects” them to SP CPU (protocol redirection logic)
• Switch installs static Layer 2 forwarding entries for each
multicast group MAC address
• Multicast data traffic forwarded to appropriate Interfaces
according to MAC address table entries (per VLAN)
 NOTE: Snooping does NOT affect the

performance of multicast data traffic
 Redirection logic ONLY redirects IGMP/MLD
packets, not UDP (multicast data) packets
IGMP Snooping - CLI
6500#sho ip igmp group vlan 201
IGMP Connected Group Membership
Group Address Interface Uptime Expires Last Reporter Group Accounted
239.1.1.101 Vlan201 05:09:00 stopped 201.1.1.2
6500#sho ip igmp snoop stat int vlan 201
Current number of Statistics entries : 41

Configured Statistics database limit : 32000
Configured Statistics database threshold: 25600
Configured Statistics database limit : Not exceeded
Configured Statistics database threshold: Not exceeded
Snooping statistics for Vlan201

#channels: 1
#hosts : 1
Source/Group Interface Reporter Uptime Last-Join Last-Leave

0.0.0.0/239.1.1.101 Vl201:Te4/1 201.1.1.2 05:10:36 00:00:00 -
6500#
PIM Snooping
 Purpose: Constrain unnecessary multicast

(default) “flooding” on Layer 2 switch-ports
 Mainly used in PIM-connected Layer 2 “Core”
 PIM Snooping code also leverages both
Hardware and Software information:
• Switch “snoops” the L3 contents of PIM packets
• PFC/DFC ASIC’s recognize PIM packets and “redirects”
them to SP CPU (protocol redirection logic)
• Switch installs / removes Layer 2 forwarding entries for
interfaces which send PIM Joins / Prunes
• Multicast data traffic only forwarded to appropriate
Interfaces according to PIM Snooping mroute table entries
(per VLAN)
 Protocol redirection ONLY redirects PIM

packets, not UDP (multicast data) packets
PIM Snooping - CLI
6500#sho ip pim snoop neigh vlan 146
IP Address Mac address Port Uptime/Expires Flags
VLAN 146: 3 neighbors
1.46.1.6 000b.45b7.0900 Gi2/3/16 00:21:14/00:01:42
1.46.1.4 0005.74e2.b340 Po202 01:50:22/00:01:41
1.46.1.1 0001.6444.5fc0 Router 01:50:23/00:01:31 DR DF
6500#sho ip pim snoop mroute vlan 146
Flags: J/P - (*,G) Join/Prune, j/p - (S,G) Join/Prune
SGR-P - (S,G,R) Prune
VLAN 146: 2 mroutes

(*, 239.135.1.5), 00:32:28/00:02:31
Downstream ports:
Upstream ports:
Outgoing ports: 15/1 Gi2/3/16
(1.35.1.51, 239.135.1.5), 00:32:28/00:02:31

1.46.1.6->1.46.1.1, 00:32:28/00:02:31, j
Downstream ports: Gi2/3/16
Upstream ports: 15/1
Outgoing ports: 15/1 Gi2/3/16
IGMP/MLD/PIM Snooping Packet Flow
Supervisor
CPU Daughter Card
RP (MSFC)
Port ASIC CPU
5
SP (NMP) SNOOPING
4
CPU PROCESS
MET
Fabric Interface Integrated

& Replication 720 Gbps
Engine Switch Fabric
CEF720
3 Module
DFC
L2 Engine Fabric Interface & w/DFC
2 Replication Engine
Protocol
L3/4 Engine Redirection
Logic
Port ASIC Port ASIC
1
Multicast on VSS
VSS Control-Plane
• VSS Control-Plane
• VSS Benefits
• VSS FIB & MET Programming
• VSS Packet Walks
Virtual Switching System - Overview
Multicast Control Plane Details
Control Plane operates ONLY on the VSS Active Supervisor
Multicast Control Plane traffic ( IGMP / MLD, PIM hellos, PIM Joins / Prunes, etc... )
received by VSS Standby are redirected to VSS Active via VSL
Control traffic is redirected to Active VSS SP/RP
Catalyst 6500 #1 Catalyst 6500 #2

(VS Active) (VS Standby)
VSL
Supervisor Supervisor
SP RP RP SP
PIM
Hellos
MEC MEC
Multicast on VSS
VSS Control-Plane
• VSS Benefits
Benefits of Multicast with VSS
Stand-Alone Virtual Switch

DR Non--DR
Non Switch1: Active Switch2: Standby
DR
STP/HSRP VSL
 Better multicast traffic load-distribution using MEC

 Inter chassis SSO – (*,G) & (S,G) and L2 snooping entries
synced between chassis, for faster convergence
 Simplified topology eliminates PIM DR election process, as
well as Non-RPF traffic across inter-switch links
Multicast Re-convergence Scenarios
Multicast state: 500 PIM groups @ 10 sources
per-group = 5000 mroute entries
Uplinks Failure Switch Interlink Failure
With Egress replication-mode = 10,000
shortcuts in hardware
RP
L3 L3
2 3
L2/L3 L2/L3
DR
4
Supervisor Failure Downlinks Failure
Receivers Receivers
Multicast on VSS
VSS Control-Plane
• VSS Benefits
VSS Multicast: FIB Programming
VS ACTIVE VS STANDBY
• The VS Standby SP is treated MET CFC

the same as a DFC module
DFC FIB/ADJ
from a FIB & ADJ forwarding
MET
perspective. DFC
FIB/ADJ
• The VS Active SP installs MET
multicast forwarding shortcuts

CFC CFC
onto the Active PFC, Standby MET MET
PFC and all other DFC’s in the

VS System.
FIB/ADJ FIB/ADJ
Active Standby
MET SP SP MET
VSL
HA FIB-CAPABLE
ICC MCAST GROUP
VSS Multicast: MET programming
VS ACTIVE VS STANDBY
• The VS Standby SP is CFC MET
treated as a Supervisor
from a MET programming DFC MET
perspective
DFC MET
• The SP on each Chassis
manages & programs it’s CFC MET CFC MET
own MET, and also MET’s

of all local CFC modules
Active Standby
• DFC modules manage and SP MET SP MET
program their own MET’s VSL

ACTIVE SCP MET STANDBY SCP MET
MCAST GROUP MCAST GROUP
Multicast on VSS
VSS Control-Plane
• VSS Benefits
VSS Ingress Forwarding Model
All forwarding lookups & replication occurs on the Ingress modules

Subsequent modules only perform L2 egress lookup. Multiple
replications cross the Ingress Core’s Switch Fabric and / or VSL
Ingress Core Egress Core
M0 R R M0 R3
E E
1 M1 R R M1
S MET2 LC2 LC 1 LC 3
MET3
2
M0 R R M0
R1 E R4
6 7 8
E VSL E
LC1 (Ingress) R5
3 M1 R R M1 9 10
LC3 4 LC 4
LC2
Switch 1 Switch 2
MEC
Access Switch I-E-E-E (Ingress-VSL-VSL-Egress)

•Multicast packets may cross up to 4 DFC’s
5
•Ingress DFC performs L2-L4 forwarding lookup
R2 •Up to 3 DFC’s (Cross VSL) perform L2 egress lookup
VSS Egress Forwarding Model
Ingress modules replicate locally & one copy to Egress modules

Egress modules (on Ingress Core) replicate locally, and also for all
receivers on Egress Core. Multiple replications cross the VSL
Ingress Core Egress Core

M0 R R M0 R3
E E
M1 R R M1
1
MET3 LC 3
S 2 LC2 (Egress) LC 1
MET2
M0 R R M0 R4
E
R1 6 7 8
E VSL E
LC1 (Ingress) R5
3 M1 R R M1 9
4 10
LC3 (Egress) LC2 LC 4
Switch 1 Switch 2
MEC
I-E-E-E (Ingress-VSL-VSL-Egress)
MET2 MET3
Access Switch
•Multicast packets may cross up to 4 DFC’s
5 •Ingress DFC performs L2-L4 forwarding lookup
•Up to 3 DFC’s (Cross VSL) perform L2 egress lookup
R2
Multicast Debugging
High-Level Flow
• High-Level Flow
• Verifying Multicast Routing
• Verifying Group Membership
• Scalability Notes
Multicast Related Entries
Protocols EIGRP/OSPF PIM/IGMP IGMP snooping
Unicast Routing Multicast Routing Layer 2 Forwarding

Table Table Table
Software
Tables
CEF MFIB CEF – MFIB Subsystem
S/W adjacency S/W FIB

Table Table
Hardware H/W adjacency H/W FIB Replication

Tables Table Table Table
Multicast Related Entries
Protocols EIGRP/OSPF PIM/IGMP IGMP snooping

show ip mroute show ip igmp snooping group
show ip mrib show ip igmp group
Unicast Routing Multicast Routing Layer 2 Forwarding

Table Table Table
Software show ip mfib
Tables
CEF MFIB CEF – MFIB Subsystem
show platform software multicast routing
S/W adjacency S/W FIB

Table Table
show platform hardware multicast routing
Hardware H/W adjacency H/W FIB Replication

Tables Table Table Table
show platform software met statistics
show platform software met module <#> detail
Multicast Debugging
Verifying Multicast Routing
• High-Level Flow
Verify Multicast Configurations
 Verify that ip multicast-routing is enabled!

 If MVPN, verify “ip multicast-routing” is enabled “per-VRF”
 Verify PIM enabled on applicable Interfaces (including Loopbacks)
 If PIM-SM and / or PIM-BiDir, then verify the RP
configuration (AutoRP, BSR, Phantom, Anycast, Static)
 Hardware MMLS/MFIB and IGMP Snooping enabled
(enabled by default, but can be disabled)
 Watch out for TTL, Multicast Boundary, Router Guard, IGMP
Filters, RACL’s, VACLs’, QoS Policers, etc.
 Watch for proper SSM / BiDir address-range configuration
(Requires prior address planning & coordination)
 Verify that Unicast Routing is enabled & that the RPF

information is correct 
Verifying Group-to-RP Mappings
6500#show ip pim rp mapping
PIM Group-to-RP Mappings
Group(s): 224.0.0.0/4, Static
RP: 10.255.255.3 (6509.cisco.com)
6500#show ip pim rp
Group: 239.1.1.10, RP: 10.255.255.3, v2, uptime 00:01:10, expires never
6500#
Identifying the DR for a Segment
show ip pim interface - identifies DR for each Interface
6500#show ip pim Interface
Address Interface Ver/ Nbr Query DR DR

Mode Count Intvl Prior
10.255.255.2 Loopback0 v2/S 0 30 1 10.255.255.2
10.255.254.1 Loopback1 v2/S 0 30 1 10.255.254.1
10.10.1.2 GigabitEthernet4/3 v2/S 1 30 1 10.10.1.2
10.30.1.1 GigabitEthernet4/16 v2/S 1 30 1 10.30.1.2
10.100.1.1 Vlan100 v2/S 1 30 1 10.100.1.2
10.101.1.1 Vlan101 v2/S 1 30 1 10.101.1.2
10.200.1.1 Vlan200 v2/S 1 30 1 10.200.1.2
10.201.1.1 Vlan201 v2/S 1 30 1 10.201.1.2
6500# This router’s PIM
Compare with DR
interface addresses
address
Bidir RP-to-DF Interface Mapping Table
6500#show mls ip multicast rp-mapping df-cache

State: H - Hardware Switched, I - Install Pending, D - Delete
Pending,
Z - Zombie All DF Interfaces
should be in H state
RP Address State DF State

10.255.255.3 H Gi2/13 H
10.255.255.3 H Gi1/13 H
10.255.255.3 H Gi2/14 H
10.255.255.3 H Gi2/24 H
10.255.255.3 H Vl100 H
10.255.255.3 H Vl101 H
10.255.255.3 H Vl200 H
10.255.255.3 H Vl201 H
6500# DF Interfaces for
specified Bidir RP
Multicast Debugging
Verifying Group Membership
• High-Level Flow
Verifying IGMP Membership
 IGMP is enabled when PIM is configured on an Interface

 For all receiver segments, IGMP (or MLD) drives OIF presence
in mroute table
Use show ip igmp groups to see IGMP join status on mrouter
Use debug ip igmp <group> to monitor IGMP packet reception at router
Be aware of effect of IGMP snooping on Layer 3 IGMP behavior
(e.g. not all joins/leaves seen by router)
 Verify IGMP Querier consistency, and group membership for

mrouters on shared segments
 Watch for IGMP access groups, filters, and / or multicast boundary
 Verify IGMP snooping entries
Make sure entry exists & port membership correct
Should include the mrouter port
Verifying IGMP & Snooping -
Configuration Status
6500#show ip igmp interface vlan 100

Vlan100 is up, line protocol is up
Internet address is 10.100.1.2/24
IGMP is enabled on interface
Current IGMP host version is 2
Current IGMP router version is 2
IGMP query interval is 60 seconds IGMP L3
IGMP querier timeout is 120 seconds Interface State
IGMP max query response time is 10 seconds

Last member query count is 2
Last member query response interval is 1000 ms
Inbound IGMP access group is not set
IGMP Packet
IGMP activity: 3 joins, 2 leaves Statistics
Verifying IGMP & Snooping -
Configuration Status (2)
...
Multicast routing is enabled on Interface
Multicast TTL threshold is 0
Multicast designated router (DR) is 10.100.1.3 DR & Querier
Information
IGMP querying router is 10.100.1.2 (this system)
No multicast groups joined by this system
IGMP snooping is globally enabled
IGMP snooping is enabled on this interface
IGMP snooping fast-leave is disabled and querier is disabled
IGMP snooping explicit-tracking is enabled
IGMP snooping last member query response interval is 1000 ms
IGMP snooping report-suppression is disabled
6500#
IGMP L2 Snooping State
Viewing Multicast Routers
6500#show ip igmp snooping mrouter
vlan ports
-----+----------------------------------------
100 Gi3/7,Gi3/16
101 Gi3/7,Gi3/16 List of Multicast
Router Ports, per VLAN
200 Gi3/7,Gi3/16
201 Gi3/7,Gi3/16
6500#
VLAN for Multicast
Router Ports
Verifying IGMP Entries
6500#show ip igmp groups
IGMP Connected Group Membership

Group Address Interface Uptime Expires Last Reporter
239.1.2.0 Vlan200 00:00:12 00:00:00 10.200.1.101
239.1.2.1 Vlan200 00:00:12 00:00:00 10.200.1.101
239.1.2.2 Vlan200 00:00:12 00:00:00 10.200.1.101
239.1.2.3 Vlan200 00:00:12 00:00:00 10.200.1.101
239.1.1.10 Vlan201 Interfaces with
00:00:13 00:00:00 10.201.1.101
Connected Receiver
224.0.1.40 Loopback0 1w4d 00:02:18 10.255.255.1
6500#
IP Address of Last
Multicast Group Joined Receiver to Report,
for the Group on the
Interface
Verifying IGMP Snooping Entries
6500#show mac-address-table multicast igmp
vlan mac address type learn qos ports

-----+---------------+--------+-----+---+--------------------------------
200 0100.5e01.0101 static Yes - Gi3/1,Gi3/7,Gi3/16
6500#
List of Receiver Ports
(Also Includes Any
Multicast Router Ports
in VLAN)
Group Destination MAC
VLAN Where Group

MAC Is Learned
Multicast Debugging
Scalability Notes
• High-Level Flow
Scalability Limits
1. Ensure L2 & L3 entry capacity is not exceeded

2. Monitor MET utilization (per replication-mode)
3. Monitor RP (and SP) CPU utilization
4. Monitor Fabric and Bus utilization
Take Away’s…
 Catalyst 6500 has over 10 years of Multicast innovations
 Optimized Hardware IPv4 & IPv6 Multicast Forwarding
 Scale to 32K mroutes @ ~30Mpps with Sup720 & PFC3
 Scale to 128K mroute @ ~60Mpps with Sup2T & PFC4
 Optimal forwarding capacity with Virtual Switching
 Catalyst 6500 Multicast Rocks!!! 
Complete Your Online
Session Evaluation
 Give us your feedback &

you could win Fabulous
prizes…
• Winners announced daily!
 Receive 20 Passport Points

for each session evaluation
that you complete.
 Complete your session
evaluation online now:
• Open a browser through our wireless Don’t forget to activate your CiscoLive 2011
network to access our portal “virtual account” for access to all session
materials, communities, as well as both live &
• Or visit one of the Internet stations on-demand activities throughout the year.
throughout the Convention Center
Activate your account at the Cisco booth in the
World of Solutions or visit www.ciscolive.com.

BRKARC-3322 - Catalyst 6500 IP Multicast Architecture (2011 Las Vegas)

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BRKARC-3322 - Catalyst 6500 IP Multicast Architecture (2011 Las Vegas)

Uploaded by

Copyright:

Available Formats

Session Assumptions

You already have an understanding of

To provide a thorough understanding

 To introduce several IPv4 / IPv6 Multicast

This is YOUR session:

• PIM Distribution Modes

 Designed with the assumption that receivers Source1 Source2

are “Densely” populated in the network...

 Does NOT use Rendezvous Points (RP’s),

 Inefficient “Flood” style packet delivery

 Uses both Shared (RP) & Source Trees

 Data-driven Multicast state

 Automatic “Source” discovery (via Registers)

 Uses only & Source-specific (SPT) Trees

 Hosts are responsible for “source discovery”

 Group members tracked via IGMPv3 (IPv4)

 Highly Scalable variation of PIM-SM, which

 Uses only Shared (RP-based) Trees

 Drastically reduces network mroute state

 Group members tracked via IGMPv1 or v2

• PIM Distribution Modes

 Purpose: Signal & refresh Multicast group

 IGMP v1 & v2 support for PIM-SM & PIM-BiDir

 MLD v1 & v2 support for IPv6 PIM-SM

 IGMP for IPv4 & MLD for IPv6

 Layer 2 IGMP / MLD “Snooping” constrains

 Remember: Snooping leverages both L2 & L3

• PIM Distribution Modes

Default MDT Receiver

 Allows for Multicast content that originated in one MVPN to be

• PIM Distribution Modes

 Multicast Non-Stop Forwarding (NSF) Multicast Data

• Multicast Expansion Table (MET) - for MMLS Standby

 Minimizes multicast traffic disruption

 Use-Case: Any SSO/NSF capable “highly RP

• PIM Distribution Modes

 Important for PIM-BiDir Video vlan200

 PIM, MROUTE, MSDP, MVPN &

• PIM Distribution Modes

 Leveraging 10+ years of IPv4 & IPv6 Multicast

 Deployed in the world’s most business-critical networks

 Implements both Centralized & Distributed

 Supports IPv4 / IPv6 PIM-SM & SSM (S,G)

FIB-XL  Provides for unparalleled

 New MFIB & EDC (+ LTL & MET sharing) designs

Egress-Replication support  Optimized MVPN Replication!

 Hardware IGMPv3 & MLDv2 “Snooping”

 Some Other Enhancements (Not Listed Previously)…

 IP based (vs. DMAC based) L2 lookups, eliminates IP-

 CoPP has special exception cases & granular rate-

 NetFlow (V9 & FnF

 Multicast Service Reflection (MSR) support for up to 64K translations.

 Consistent (IPv4 & IPv6) MFIB-

 Hardware Integration with VPLS, H-

 Enhanced Multicast over GRE processing & forwarding performance.

 Route Processor - Layer 3  Switch Processor - Layer 2

720G Switch Fabric & PFC3 Daughter Card

Route Processor (RP)

Switch Processor (SP)

 Route Processes - Layer 3  Switch Processes - Layer 2

Supervisor 720 Baseboard 1GE Uplinks

Counter QoS FIB

Supervisor Engine 32 Baseboard 10GE Uplinks

Counter QoS FIB