Back2Basics B k2B i

October14

DataCenterTechnicalOverviewSeries Data Center Technical Overview Series

Nexus7000 Nexus 7000

IMPORTANT:Audioisbeingbroadcastdirectlytoyour
computerspeakers,somakesuretheyarefunctional.No needtodialinseparately.

Back2Basics
Today sPresenter Todays Presenter
Current
CiscoDataCenterConsultingSystems EngineerspecializinginNexus7000partner E i i li i i N 7000 enablement.LocatedinRosemont,Chicago

Past
CiscoSecuritySystemsEngineer,Sr.Network EngineeratIPG (InterPublicGroup),Network Engineerat3com/USRobotics andMotorola Engineer at 3com/USRobotics and Motorola

RajChacko Raj Chacko

Cisco Nexus 7000 Series Cisco Nexus 7000 Back to Basics Switch & NX-OS Roadmap

Raj Chacko CCIE R&S Security Chacko, R&S, rajchack@cisco.com

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

The Evolving Data Center and New Challenges

Emerging Challenges Impact
Network/ Storage Sophisticated Virtualization
Higher I/O requirements Greater east-west bandwidth Rapid provisioning/

Application Complexity

Physical Infrastructure
10G ready wiring Server/cabling density

Cloud Computing and XaaS

Application Performance
WAN optimization Application Acceleration

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Transforming the DC with New Technologies

Consolidation Utility Automation A t ti Market

Virtualization

MultiMulti-SP Cloud Private Cloud Unified Computing Unified Fabric Architecture SP Cloud

Data Center Networking

HA with ISSU VPC

2008 Today 2011+

Presentation_ID

VDC Unified Fabric Fabric Extender OTV FabricPath Cloud-centric Networking Services LISP
5

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

The Cisco Nexus Switching Family

Complete switching portfolio Consistent operating system across all platforms Infrastructure scalability, transport flexibility and operational manageability
Nexus 7010 Nexus 7018 Nexus 1000V Virtual Switch

Nexus 4000

Nexus 5000

Nexus 2000 Fabric Extender

2008

Cisco Nexus 1000V

1K
x86

NX-OS Operating System Data Center Network Manager

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Introducing the Cisco Nexus 7000

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Nexus 7000 Platform

Industrys First Data Center Class Platform Nexus 7000 and NX-OS 10 & 18 Sl t versions Slot i 15+ Terabit System Unified Fabric Ready Modern Modular OS Modern, Device Virtualization Cisco TrustSec Continuous Operations p

Nexus 7010
8 I/O Slots + 2 Supervisor Slots Front to Back Airflow 256 10GbE (4:1) / 64 Ports line rate 384 10/100/1000 Ports

Nexus 7018
16 I/O Slots + 2 Supervisor Slots Side to Side Airflow 512 10GbE (4:1) / 128 Ports line rate 768 10/100/1000 Ports

Cisco NX-OS Multi-protocol Operating System Data Center Network Manager (DCNM)
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Nexus 7010 Chassis

Integrated cable management with cover Optional locking front doors d Locking ejector levers e es Supervisor slots (5-6) Payload slots (1-4, 7-10)

System status LEDs

ID LEDs on all FRUs

Front-toback airflow

Air exhaust

System fan trays Fabric fan trays

21RU

Two chassis per 7 rack Crossbar fabric modules

Power supplies Air intake with optional filter

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved.

Front

N7K-C7010
Cisco Confidential

Rear

Common equipment removes from rear

9

Nexus 7018 Chassis

Integrated cable management

System status LEDs Optional front door Side-to-side airflow

System fan trays

Supervisor slots (9-10)

25RU

Crossbar fabric modules

Payload slots (1-8, 11-18)

Common equipment removes from rear

Power supply air intake

Power supplies

Front
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

N7K-C7018

10

Rear

10

Supervisor Engine
Performs control plane and management functions D l Dual-core 1 66GH Intel Xeon processor with 4GB DRAM 1.66GHz I l X ih 2MB NVRAM, 2GB internal bootdisk, compact flash slots Out-of-band 10/100/1000 management interface Always-on Connectivity Management Processor (CMP) for lights-out management Console and auxiliary serial ports USB ports for file transfer
N7K-SUP1

ID LED Status LEDs

Presentation_ID

AUX Port Console Port Management Ethernet

Cisco Confidential

USB Ports Compact Flash Slots Reset Button

CMP Ethernet

2009 Cisco Systems, Inc. All rights reserved.

Management Interfaces
Management Ethernet
10/100/1000 interface used exclusively for system management Belongs to dedicated management VRF
Prevents data plane traffic from entering/exiting from mgmt0 interface Cannot move mgmt0 interface to another VRF Cannot assign other system ports to management VRF

Connectivity Management Processor (CMP) Ethernet

Connects to standalone, always-on microprocessor on supervisor engine i i i
Runs lightweight software with network stack Completely independent of NX-OS on main CPU

Provides lights out remote management and lights out disaster recovery via 10/100/1000 interface
Removes need for terminal servers
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Nexus 7000 Line Module Portfolio

( (1 Gig and under) g ) M1 Series 1GbE
$15K $27K TBD $27K

N7K-M148GT-11
(Shipping)

N7K-M148GS-11
(Shipping)

N7K-M148GT-11L
(Target Cairo 2HCY10)

N7K-M148GS-11L
(FCSd May 25th)

48 x 10/100/1000 46 Gbps Fabric Copper

48 x 1GigE 46 Gbps Fabric SFP

48 x 10/100/1000 46 Gbps Fabric Copper

48 x 1GigE 46 Gbps Fabric SFP

XL Capable

60 Mpps Forwarding Capacity L2 / L3

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

13

Nexus 7000 Line Module Portfolio 10 Gig

M1 Series 10 Gigabit Ethernet
$70K $70K $44K

N7K-M132XP-12
(Shipping)

N7K-M132XP-12L
(Cairo Target Oct 2010)

N7K-M108X2-12L
(FCSd May 25)

32 x 10GigE 4:1 Oversubscribed SFP+ 60 Mpps 80 Gbps Fabric

32 x 10GigE 4:1 Oversubscribed SFP+ 60 Mpps 80 Gbps Fabric XL Capable FEX Support

8 x 10GigE 1:1 Line rate X2 120 Mpps 80 Gbps Fabric

L2 / L3
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

14

F1 Series: High Performance High-Performance Layer 2 10GbE Modules

32-port 10G SFP+ F1 module Oct 2010

SFP+ and RJ-45 10G DCB I/O RJ 45 modules 1G/10G dual-speed switch-on-chip design Layer 2 with L3/L4 services
L3 Routing provided by M1 Modules

Hi h performance High f
230 Gbps fabric connectivity 320 Gbps local switching 480 Mpps forwarding per module 7.68 Billion pps per 7018

Multi-protocol Classic Ethernet, vPC, L2MP, DCB, FCoE

SKU N7K-F132XP-15=

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

15

Integrated Forwarding Engine

Advanced hardware forwarding engine Up to 60Mpps IPv4 unicast, 30Mpps IPv6 unicast throughput M1 Series Forwarding Engine
Equal to Cat 6K EARL 8

Integrated on every I/O module (NOT a FRU)

Table sizes
FIB TCAM IPv4 Routes IPv6 Routes Classification TCAM (ACL and QoS) NetFlow TCAM (Ingress and Egress) MAC table Bridge Domains (VDC + VLAN) Non-XL 128K 128K 64K 64K 512K 128K 16K XL Up to 1M Up to 1M Up to 500K 128K 512K 128K 16K

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Crossbar Switch Fabric Module

Each fabric module provides 46Gbps per I/O module slot
Up to 230Gbps per slot with 5 fabric modules

Initially shipping I/O modules do not leverage full fabric bandwidth

Maximum 80G per slot with 10G module Future modules leverage additional available fabric bandwidth

Access to fabric controlled using QoS aware central QoS-aware arbitration with VOQ

N7K-C7010-FAB-1

N7K-C7018-FAB-1
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Current I/O Module Capacity

Fabric Modules

1Gbps I/O modules

Requires 1 fabric for full bandwidth b d idth Requires 2 fabrics for N+1 redundancy

46Gbps/slot

Crossbar Fabric ASICs

46Gbps/slot

Crossbar Fabric ASICs

46Gbps 184Gbps 138Gbps 92Gbps 230Gbps

per slot bandwidth 4th and 5th fabric modules provide additional redundancy for current M1 cards, and full bandwidth for F1
46Gbps/slot 46Gbps/slot 46Gb / l t

Crossbar Fabric ASICs ASIC

Crossbar Fabric F bi ASICs

10Gbps I/O modules

Requires 2 fabrics for full bandwidth Requires 3 fabrics for N+1 redundancy
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

46Gbps/slot

Crossbar Fabric ASICs

18
18

Access to Fabric Bandwidth

Access to fabric controlled using central arbitration
Arbiter ASIC on supervisor engine provides fabric arbitration

Bandwidth capacity on egress modules represented by Virtual Output Queues (VOQs) t i (VOQ ) at ingress t fabric to f b i
I/O modules interface with arbiter to gain access to VOQs

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

NX-OS: NX OS: Purpose Built for the Data Center

IOS Catalyst

NX-OS Nexus

SAN-OS SAN OS

Release 4.1 MDS

Cisco Confidential

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

20

NX OS NX-OS Modular Architecture

Layer 3 Protocols Layer 2 Protocols
VLAN PVLAN STP LACP UDLD CDP 802.1X CTS

Storage Protocols
VSANs FCIP IVR Zoning FSPF

HA Manager M

OSPF BGP EIGRP PIM

GLBP HSRP IGMP SNMP

Future

System Infrastructure

Kernel (Linux)

Based on MDS-9000 Series SAN-OS Every process runs in protected memory for fault containment Automatic stateful process restart Modular code only runs in DRAM when invoked
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Data Center Class Availability

In Service Software Upgrade (ISSU)
Upgrade and reboot Initiate stateful failover Upgrade and reboot Upgrade and reboot I/O CPU

In Service Software Upgrades Minimize Planned Downtime Upgrades are possible between minor and major software releases Critical components for LAN + SAN vision

Active
OSP PF OSP PF BGP P PIM M etc c.

Standby
BGP P PIM M e etc.

Release Releas 4.2 e 4.1

Release Releas 4.2 e 4.1

HA Manager

HA Manager Kernel Kernel N7K Data Plane

I/O Module Images

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Release Releas R l 4.2 e 4.1

22

Data Center Class Availability

Stateful Process Restart

Stateful Process Restart Avoid Network Re-convergence Processes can restart in milliseconds and maintain state from state database (PSS) Net effect is zero impact to neighbor relationships Supported for all L2 protocols as well as OSPFv2
Restart process!
TCP/UDP P HSRP OSPF LACP IPv6 STP BGP PIM etc

HA Manager Kernel N7K Data Plane

PSS

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

23

Unified Fabric
Increased Efficiency, Simplified Operations Efficiency

Mgmt Network Front-End Network Backup Network Unified Fabric Storage Network Back-End Network

Unified Fabric

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

24

Delivering Unified I/O

Fibre Channel over Ethernet
Data Center Ethernet Standards Unified I/O Transport Mapping FC frames over Ethernet Transport Enables Fibre Channel to run over a lossless Ethernet medium Single Adapter, less device proliferation, lower power consumption NO gateways required
Ethern net Header E FCoE Header FC Header CRC C EOF F

Ethernet Eth t Fibre Channel Traffic

FC Payload

FCS S
25

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Network Stack Comparison

SCSI FCP iSCSI FCIP TCP IP Ethernet Physical Wire
SCSI
Presentation_ID

Less Overhead than FCIP or iSCSI

FC

FCoE

iSCSI
Cisco Confidential

FCIP

FCoE

FC
26

2009 Cisco Systems, Inc. All rights reserved.

Virtualization with VDC 1 to Many

VDC 1 VDC 2 Layer 2 Protocols Layer 2 Protocols
VLAN PVLAN STP LACP UDLD CDP 802.1X CTS

Layer 3 Protocols
OSPF BGP EIGRP PIM GLBP HSRP IGMP SNMP

Layer 3 Protocols
VLAN OSPF BGP EIGRP PIM GLBP HSRP IGMP SNMP UDLD CDP 802.1X CTS

VDC 1 VDC 2 VDC 3 VDC 4

PVLAN STP LACP

Infrastructure Kernel VDC Virtual Device Context (Up to 4) Fl ibl separation/distribution of h d Flexible i /di ib i f hardware resources and software components d f Complete data plane and control plane separation, Physical ports allocated to VDCs Complete software fault isolation Securely delineated administrative contexts Forwarding engine scalability with appropriate interface allocation
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

27

VDC - Enabling Network Consolidation

=
VDC 4 VDC Extranet VDC DMZ

VDC 2

VDC Prod

Device Partitioning into Multiple Contexts

Lower Capital Expenditure

Consolidate multiple devices Remove interconnect links

Reduce Operational Costs

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Fewer number of devices to manage Lowers overall data center power draw
28

Virtual Device Contexts

VDC Resource Utilization (Layer 2)
Layer 2 learning with multiple active VDCs also has an impact on resource utilization - MAC addresses learnt in a VDC are only propagated to other linecards when that linecard has a port in that VDC VDC

Switch Fabric
X

Linecard 1
MAC Table

Linecard 2
MAC Table

Linecard 3
MAC Table

MAC A A
1/1 1/2 1/3 1/4 2/1

MAC A A
2/2 2/3 2/4 3/1 3/2 3/3 3/4

VDC 10

VDC 20

VDC 20

VDC 10

VDC 30

VDC 20

MAC Address A

MAC A is propagated to linecard 2 and 3 but only linecard 2 installs MAC due to local port being in VDC 10
Cisco Confidential

VDC 30

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

29

Virtual Device Contexts

VDC Resource Utilization (Layer 3)
VDC 10
Linecard 1
FIB TCAM

VDC 20
Linecard 3
FIB TCAM

VDC 30
Linecard 5
FIB TCAM

FIB and ACL TCAM resources are more effectively utilized utilized
Linecard 6
FIB TCAM

Linecard 2
FIB TCAM

Linecard 4
FIB TCAM

Linecard 7
FIB TCAM

Linecard 8
FIB TCAM

128K ACL TCAM

128K ACL TCAM

128K ACL TCAM

128K ACL TCAM

128K ACL TCAM

128K ACL TCAM

128K ACL TCAM

128K ACL TCAM

64K

64K

64K

64K

64K

64K

64K

64K

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

30

Virtual Device Contexts

VDC Administration
Super User can access all global configuration commands, can create/delete VDCs and perform , p resource allocation across VDCs

VDC Administrator can change any configuration for resources allocated to that VDC and can also create user roles specific to that VDC with a subset of configuration commands b t f fi ti d

VDC User Role is a restricted role based access for a given VDC and can perform configuration as defined by the VDC Administrator

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

31

Enhancing Layer2 Scalability Multi-Chassis Solution

Virtual Port Channel (vPC)
L2
Si Si

Non-vPC

vPC

Physical Topology

Logical Topology

Bi-sectional Bi sectional BW with vPC

Virtual Port Channel

vPC is a Port-channeling concept extending link aggregation to two separate physical switches h i l i h Allows the creation of resilient L2 topologies based on Link Aggregation. Aggregation Eliminates the need for STP in the access-distribution Layer
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Uses all available uplink bandwidth y, Enable seamless VM Mobility, Server HA Clusters Scale Available Layer 2 Bandwidth Grow the size of the layer 2 network Simplify Network Design
32

Presentation_ID

Feature Overview & Terminology

vPC Terminology
vPC peer-keepalive link vPC peer-link

vPC peer a vPC switch, one of a pair vPC member port one of a set of ports (port channels) that form a vPC vPC the combined port channel between the vPC peers and the downstream device

CFS protocol

vPC peer-link Link used to synchronize state between vPC peer devices, must be 10GbE
vPC peer vPC

vPC vPC member b member port port

vPC peer-keepalive link the keepalive link between vPC peer devices, i.e., backup to the vPC peer-link vPC VLAN one of the VLANs carried over the peer-link and used to communicate via vPC with a peer device. non vPC VLAN One of the STP VLANs not non-vPC carried over the peer-link

vPC non-vPC device

CFS Cisco Fabric Services protocol, used for state synchronization and configuration validation between vPC peer devices

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Building a vPC Domain

Steps to setup vPC
1. Configure globally a vPC domain on both vPC devices 2. Configure a Peer-keepalive link on both vPC peer switches (make sure is operational) NOTE: When a vPC domain is configured the keepalive must be operational to allow a vPC d PC domain t successfully f i to f ll form. 3. Configure (or reuse) an interconnecting port-channel between the vPC peer switches 4. Configure the inter-switch channel as Peer-link on both vPC devices (make sure is operational) 5. Configure (or reuse) Port-channels to dual-attached devices 6. Configure a unique logical vPC and join port-channels across different vPC peers

vPC peerkeepalive link

vPC peer-link

vPC peer

Standalone Port-channel
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved.

vPC
Cisco Confidential

vPC member port

34

Attaching to a vPC Domain

IEEE 802 3ad and LACP 802.3ad
Definition:
Port-channel for devices for devices dual-attached to the vPC pair. Provides local load balancing for port-channel members STANDARD 802.3ad port channel

Access Device Requirements

STANDARD 802.3ad capability STANDARD LACP Optional

Recommendations:
Use LACP when available for better failover and misconfiguration protection
vPC Regular member Portport channel port

vPC

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Overlay Transport Virtualization

Technology Pillars
OTV is a MAC in IP technique to MAC IP extend Layer 2 domains OVER ANY TRANSPORT

Dynamic Encapsulation
No Pseudo-Wire State Maintenance Optimal Multicast Replication Multipoint Connectivity Point-to-Cloud Model
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Protocol Learning
Nexus 7000 First platform to support OTV starting with 5.0(3) release! Preserve Failure Boundary Built-in Loop Prevention Automated Multi-homing Site Independence
36

OTV Data Plane: Inter-Site Packet Flow Inter Site

4. The Edge Device on site East receives 1. Layer 2 lookup on the destination MAC. and decapsulates the packet. MAC 3 is reachable through IP B. 5. Layer 2 lookup on the original frame. 2. The Edge Device encapsulates the frame. MAC 3 is a local MAC MAC. 3 The Core deli ers the packet to the Edge 3. delivers Device on site East. Core 6. The frame is delivered to the destination.

3 MAC TABLE VLAN

100

MAC TABLE IF
Eth 2 Eth 1 IP B IP B
OTV

MAC
MAC 1

IP A

1 Layer 2 L Lookup

100 100 100

OTV

MAC 2 MAC 3 MAC 4

2 Encap
IP A IP B

Decap IP B 4
OTV

VLAN
100 100 100 100

MAC
MAC 1

IF
IP A

OTV MAC 2 IP A

5 Layer 2 L Lookup

MAC 1 MAC 3

MAC 1 MAC 3

IP A IP B

MAC 3 MAC 4

Eth 3 Eth 4

MAC 1 MAC 3
Presentation_ID

MAC 1

West Site
Cisco Confidential

East Site

MAC 1 MAC 3

6
37

MAC 3

2009 Cisco Systems, Inc. All rights reserved.

OTV Data Plane Encapsulation

OTV adds a 42 Byte IP encapsulation. y The outer IP header is followed by an OTV shim header, which contains information about the overlay (vlan, overlay number, etc). The 802.1Q header is extracted from the original frame and the VLAN field copied over into the OTV shim header. The OTV Edge Device can also map the 802.1p CoS bits to the outer IP headers DSCP field as well as to the OTV Shim header.
802.1Q

DMAC 802.1Q Co oS IP Header

VLAN

SMAC

Eth

Payload

DMAC 6B

SMAC 6B

Ether Type 2B

OTV Shim 8B
Original Frame

CRC 4B

ToS ToS

20B

42 Byte encapsulation
(same as VPLSoGRE)
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

38

OTV Control Plane

Neighbor Discovery in a Multicast-Enabled Core Multicast Enabled
OTV Adjacencies are established over the mcast group in the core
OTV Control Plane OTV Control Plane

OTV

OTV

West

East

Multicast-enabled Core
OTV

The mechanism
Edge Devices (EDs) join an ASM multicast group in the core. They join as hosts (no PIM on EDs) OTV hellos and updates are encapsulated in IP and sent to the multicast group lti t EDs are both sources and receivers
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved.

OTV Control Plane Pl

The end result

Emulation of a multi-access linklayer multicast environment Link-local Neighbor Discovery South Adjacencies are maintained over the multicast g p group A single update reaches all neighbors

Cisco Confidential

39

OTV Control Plane

Neighbor Discovery in a Multicast-Enabled Core (1) Multicast Enabled

1
OTV Hello

2 Encap

OTV
OTV Hello

OTV Control Plane

Multicast-enabled Core 3 Core

Replication ASM Group S G oup

OTV Control Plane

OTV Hello

5 4 Decap

OTV

IP A Mcast G

OTV Hello

IP A Mcast G

OTV Hello

IP A Mcast G

IP A

IP B
IGMP Report IGMP Repor rt

West

IGMP Report

East

IP C 4 Decap

OTV
OTV Hello IP A Mcast G

OTV Control Plane

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

OTV Hello

South
40

OTV Control Plane

Neighbor Discovery in a Multicast-Enabled Core (2) Multicast Enabled
10 The West Site sees that the hello contains its ID. The OTV Adjacency is Established OTV Control Plane ASM Group
OTV Hello IP C Mcast G

OTV Hello

Multicast-enabled Core 8 Core

Replication OTV Control Plane

OTV Hello

9 Decap p

OTV

OTV

9 Decap

IP A

IP B
IGMP Report p

West

IGMP Report p

East

IGMP Rep port

IP C 7 Encap

OTV
OTV Hello IP C Mcast G

From Bottom to Top

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved.

OTV Control Plane

Cisco Confidential

OTV Hello

South

6 The South Site sends its hello with Wests address in the TLV
41

OTV Control Plane

MAC Address Advertisements Multicast Core
Every time an Edge Device learns a new MAC address, the OTV control plane will advertise it together with its associated VLAN IDs and IP next hop. The IP next hops are the addresses of the Edge Devices through which these MACs addresses are reachable in the core. A single OTV update can contain multiple MAC addresses for different VLANs. A single update reaches all neighbors, as it is encapsulated in the same ASM multicast group used for the neighbor discovery.
VLAN MAC
MAC A MAC B MAC C

4
IF
IP A IP A IP A

1
3 New MACs are learned on VLAN 100 Vlan 100 Vlan 100 Vlan 100 MAC A MAC B MAC C

100

OTV update is replicated by the core 3 2 IP A

100 100

Core
VLAN

East
MAC
MAC A MAC B MAC C

IF
IP A IP A IP A

West
3

100 100 100

South-East
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

42

Configuration
OTV CLI Configuration
Connects to the core. Used to join the Overlay network core network. Its IP address is used as source IP for the OTV encap ASM/Bidir group in the core used for the OTV Control Plane.

interface Overlay0 otv join-interface Ethernet1/1 otv control-group 239.1.1.1 otv data group 232 192 1 0/24 data-group 232.192.1.0/24 otv extend-vlan 100-150 otv site-vlan 99

SSM group range used to carry the sites site s mcast traffic data.

Site VLANs being extended by OTV

VLAN used within the Site for communication between the sites Edge Devices

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

43

Cisco NXOS Software

Layer 2 Multipathing - Feature Sets
16-Way Equal Cost Multipathing (ECMP) at Layer 2 tL Mac-in-Mac Hierarchical Addressing with built in protocol checks (RPF,TTL) Optimized MAC Learning learn based on conversations. ISIS Control Plane leveraging an established routing protocol for Layer 2 routing Interoperability with existing classic ethernet networks
VPC + allows VPC into a L2MP cloud STP Boundary Termination

Mac-inMac-in-Mac

Up to 16-Way L2 ECMP

Mac-inMac-in-Mac

Up to 16Way L2 ECMP

M lti T Multi-Topology providing t ffi l idi traffic engineering capabilities

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

44

Cisco FabricPath Overview

Cisco FabricPath
Data Plane Innovation Control Plane Innovation

No MAC learning via flooding Routing, not bridging Built-in loop-mitigation Time to Live Time-to-Live (TTL) RPF Check

Plug-n-Play Layer 2 IS-IS Support unicast and pp multicast Fast, efficient, and scalable Equal Cost Multipathing (ECMP) VLAN and Multicast Pruning Cisco NX-OS NX-

Cisco Nexus Platform

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

45

Data Plane Operation

Encapsulation to creates hierarchical address scheme FabricPath header is imposed by ingress switch Ingress and egress switch addresses are used to make Routing decision No MAC learning required inside the L2 Fabric
FabricPath Header
S42 S11 C A DATA S11

FabricPath Routing
S11 S42

STP Domain FabricPath

AC

S42

Ingress Switch
C A DATA A

Egress Switch

L2 Bridging

STP Domain 1 AC
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

STP Domain 2 AC
46

Control Plane Operation

Plug-N-Play L2 IS-IS is used to manage forwarding topology Assigned switch addresses to all FabricPath enabled switches automatically (no user configuration required) Compute shortest, pair-wise paths Support equal-cost paths between any FabricPath switch pairs
S1 S2 S3 S4

FabricPath Routing Table

Switch S1 S2 S3 S4 S12 S42 IF L1 L2 L3 L4 L1, L2, L3, L4 L1, L2, L3, L4

L1 L2 S11 L3 L4 S12 S42

L2 Fabric

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

47

Unicast with FabricPath

Forwarding decision based on FabricPath Routing Table Support more than 2 active paths (up to 16) across the Fabric Increase bi-sectional bandwidth beyond port-channel High availability with N+1 path redundancy
S1 S2 S3 S4

Switc h S42

IF L1 L1, L2, L3, L4 MAC IF A C 1/1 S S42 L2 L3 S11 L4 S12 S42

L2 Fabric

1/1 /

A
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

C
48

Multicast with FabricPath

Forwarding through distinct Trees Several Trees are rooted in key location inside the fabric All Switches in L2 Fabric share the same view for each Tree Multicast traffic load-balanced across these Trees
Root for Tree #1 Root for Tree #2 Root for Tree #3 Root for Tree #4

Ingress switch for FabricPath decides which tree to be used and add tree number in the header

L2 Fabric

A
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

C
49

FabricPath Configuration
No L2 IS-IS configuration required New feature-set keyword introduced to allow multiple conditional services required by FabricPath to be enabled in one shot Simplified operational model only 3 CLIs to get FabricPath up and running
N7K(config)# feature-set fabricpath N7K(config)# vlan 10-19 N7K(config-vlan)# mode fabricpath N7K(config)# interface port-channel 1 N7K(config if)# N7K(config-if)# switchport mode fabricpath

L2 Fabric

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

FabricPath Port CE Port 50

N7000 + FEX Single Access Layer

Nexus 7000 N

Fabric Extender (2248 FEX)

Nexus 7000 + FEX is single management FEX/Nexus 7000 Nexus 2000 FEX is like a Line Card to the Nexus 7000 No Spanning Tree between FEX and Nexus 7000 Nexus 7000 maintains all management and configuration
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

51

Nexus 2248 Fabric Extender Shipping

48x1GE/100Mb

4x10GE

Beacon & Status LEDs Beacon & Status LEDs Power Supplies, Redundant & Hot Swappable

Redundant, Hot Swappable Fans

Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

52

Nexus 2000 with Nexus 7000

Benefits of the N7K + N2K
Core

Combines benefits of Top of Rack (ToR) and End of Row (EoR) network architectures Reduces cable runs Cross Nexus architecture provides p Investment protection Reduce management points in the network Solution for higher density 1G (i.e. 96 port 1G module) Ensures feature consistency across hundreds or thousands of server ports
VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM

Ac ccess

Aggre egation

N2K / N7K 1 GE

N2K / N7K 1 GE
Presentation_ID 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Major Competitors in this space

Juniper EX8208 is shipping Juniper EX8216 is getting positioned in the DC Touting Stratus architecture IBM teaming up with Juniper in Switching

HP with a bag of switches from various vendors. H3Cme too features of Nexus 7000 Series Leading with lower price Aggressive marketing against N7K with incorrect claims

Arista with its latest modular switch 7500 Big Claims like.. 5x Performance 1/10th the Power Consumption the footprint

Brocade / Foundry gaining market share in modular switching FCoE on DCX Aggressively taking Foundry boxes to channels

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

54

Presentation_ID

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

55