Brkcom 2003

UCS Networking Deep Dive
BRKCOM-2003
www.ciscolivevirtual.com
Agenda
Overview / System Architecture

Physical Architecture Logical Architecture
Switching Modes of the Fabric Interconnect Fabric Failover Ethernet Switching Modes Recommendations SAN / LAN Northbound Connections
FC/FCoE Storage direct attach, NAS appliance port Port Channel and vPC uplinks, traffic flows and failures
Adapter Offerings
UCS Generation-2 Hardware

BRKCOM-2003 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public
Overview
Unified Computing System (UCS)

Single Point of Management
Unified Fabric
Stateless Servers with Virtualised Adapters

UCS Building Blocks

UCS Manager Embedded manages entire system UCS Fabric Interconnect 20 Port 10Gb FCoE 40 Port 10Gb FCoE UCS Fabric Extender Remote line card
UCS Blade Server Chassis Flexible bay configurations UCS Blade or Rack Server Industry-standard architecture UCS Virtual Adapters Choice of multiple adapters
Cisco UCS Networking: Physical Architecture

SAN A
MGMT
ETH 1
ETH 2
SAN B
MGMT
Uplink Ports OOB Mgmt Fabric Switch Server Ports
6100 Fabric A
Cluster
6100 Fabric B
Fabric Extenders
I O M A
Chassis 1
I O M B
Cisco Public
I O M A
Chassis 20
I O M B
6
Virtualised Adapters
CNA
CNA
CNA
Compute Blades Half / Full width
B200
B250
BRKCOM-2003
2012 Cisco and/or its affiliates. All rights reserved.
Cisco UCS Networking: Physical Architecture

SAN A
MGMT
ETH 1
ETH 2
SAN B
MGMT
Uplink Ports OOB Mgmt Fabric Switch Server Ports
6100 Fabric A
Cluster
6100 Fabric B
Fabric Extenders
I O M A
Chassis 1
I O M B
Cisco Public
VIC
FEX
FEX
Rack Mount
Management Plane
Virtualised Adapters
CNA
Compute Blades Half / Full width
B200
BRKCOM-2003
Network Interface Virtualisation (NIV)

vNIC (LIF) Host presented PCI device managed by UCSM. VIF Policy application point where a vNIC connects to UCS fabric VNTag An id appended to the packet which contains the source and destination ID used for switching within the UCS fabric.
Fabric Interconnect
vFC 1
vEth 1
IOM
Adapter
vHBA 1
vNIC 1
Cable Virtual Cable (VNTag)
Service Profile (Server)

Blade
Abstracting the Logical Architecture

What you see
6100-A 6100-A
What you get

6100-A
Switch
Eth 1/1
vFC 1
vEth 1
vFC 1
vEth Dynamic, Rapid 1 Provisioning

State abstraction
IOM A Cable
10GE A
IOM A
10GE A
Location Independence Blade or Rack Adapter
Adapter
vHBA 1
vNIC 1
vHBA 1
vNIC 1
Physical Cable
Virtual Cable (VN-Tag)
Blade
BRKCOM-2003
Service Profile (Server)

Blade
2012 Cisco and/or its affiliates. All rights reserved. Cisco Public
(Server)
9
Hardware Components
UCS 6100 Hardware Architecture
BRKCOM-2003
Cisco Public
11
2104-IOM Architecture
Components
Woodside ASIC Aggregates traffic to/from 32 hostfacing 10G Ethernet ports from/to 8 network-facing 10G Ethernet ports CPU (also referred to as CMC) Controls Redwood and perform other chassis management functionality L2 Switch Aggregates traffic from BMCs on the server blades
FLASH DRAM EEPROM
Woodside Interfaces
HIF (Backplane ports) NIF (FabricPorts) BIF CIF
1-4 Fabric Ports to Interconnect
Chassis Management Controller
Control IO
Redwood ASIC
Switch
No local switching All traffic from HIFs goes upstream for Switching
BRKCOM-2003
Chassis Signals
8 Backplane Ports to Blades
Cisco Public
12
UCS IOM Slot to Uplink Pinning

Number of Active Fabric Links Blades pinned to fabric link
1-Link
2-Link 4-Link
All the HIF ports pinned to the active link

1,3,5,7 to link-1 2,4,6,8 to link-2 1,5 to link-1 2,6 to link-2 3,7 to link-3 4,8 to link-4
HIFs are statically pinned by the system to individual fabric ports. Only 1,2,4 links are supported for pinning. 3 is not a valid pinning configuration. On a link failure, only blades pinned to the NIF are brought down
BRKCOM-2003
Cisco Public
13
Individual Links
Static Pinning (IOM-FI)
Fabric Interconnect
Static Pinning done by the system dependent on number of fabric ports

1,2 4 (2^x) are valid links for initial pinning
Blade Server Ports 1
Fabric Ports
IOM
Blade 7
Blade 2
Blade 3
Blade 4
Blade 5
Blade 6
Blade 8
14
BRKCOM-2003
Cisco Public
Individual Links
Fabric Port Failure
Fabric Interconnect
Fabric Ports
Pinned HIFs are brought down

Other blades unaffected
IOM
Blade Server Ports 1
Blade 7
Blade 2
Blade 3
Blade 4
Blade 5
Blade 6
Blade 8
15
BRKCOM-2003
Cisco Public
Individual Links
Re-ack of chassis
Fabric Ports
Fabric Interconnect
Unused Link
Blade 2 Blade 3 Blade 4 Blade 5 Blade 6 Blade 7
Blades re-pinned to valid number of links 1,2 or 4 HIFs brought down/up for repinning
May result in unused links
IOM
Server Ports
Blade 1
Addition of links requires reack of chassis.

BRKCOM-2003 2012 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Blade 8
16
Ethernet Switching Modes
Switching Modes: End Host

Spanning Tree LAN
MAC Learning MAC Learning
Server vNIC pinned to an Uplink port No Spanning Tree Protocol

Reduces CPU load on upstream switches Reduces Control Plane load on 6100 Simplified upstream connectivity
6100 A vEth 3 Fabric A

VLAN 10
vEth 1
L2 Switching
UCS connects to the LAN like a Server, not like a Switch Maintains MAC table for Servers only
Eases MAC Table sizing in the Access Layer Doubles effective bandwidth vs STP
Allows Multiple Active Uplinks per VLAN

Prevents Loops by preventing Uplink-toUplink switching Completely transparent to upstream LAN Traffic on same VLAN switched locally
Cisco Public
VNIC 0
VNIC 0
Server 2
BRKCOM-2003
Server 1
18
End Host Mode Unicast Forwarding

Server to server traffic on the same VLAN is locally switched Uplink port to Uplink port traffic not switched Uplink Ports Each server link is pinned to an uplink port / port-channel 6100 Network to server unicast traffic is forwarded to server only if it arrives on pinned uplink port. This is termed as the Reverse Path Forwarding(RPF) check vEth 1
LAN
Server 2
Deja-Vu
RPF
VLAN 10
vEth 3
Packet with source MAC belonging to a server received on an uplink port is dropped (Deja-Vu Check)
VNIC 0
VNIC 0
Server 2
Server 1
19
End Host Mode Multicast Forwarding

Broadcast traffic is pinned on exactly one uplink port (or port-channel) i.e., it is dropped when received on other uplinks All multicast groups are pinned to same uplink port (port-channel)
Server to server multicast traffic is locally switched RPF and deja-vu check also applies for multicast traffic
LAN
B
B
Broadcast Listener All VLANs
Uplink Ports
6100
vEth 1 vEth 3
VNIC 0
VNIC 0
Server 2
Server 1
20
Switching Modes: Switch

Root
LAN
Fabric Interconnect behaves like a normal Layer 2 switch

Server vNIC traffic follows VLAN forwarding

VLAN 10
MAC Learning
Spanning tree protocol is run on the uplink ports per VLANRapid PVST+
Configuration of STP parameters (bridge priority, Hello Timers etc) not supported VTP is not supported currently MAC learning/aging happens on both the server and uplink ports like in a typical Layer 2 switch Upstream links are blocked per VLAN via Spanning Tree logic
Cisco Public
vEth 1
L2 Switching
VNIC 0
VNIC 0
Server 2
BRKCOM-2003
Server 1
21
Fabric Failover
Fabric Failover
Fabric provides NIC failover capabilities chosen when defining a service profile
6100-A
L1 L2 L1 L2
6100-B
vEth 1
Physical Cable Virtual Cable
10GE
vEth 1
IOM
10GE
Traditionally done using NIC bonding driver in the OS Provides failover for both unicast and multicast traffic
Works for any OS.
IOM
PHY Adapter
Cisco VIC M81KR Menlo M71KR
vNIC 1
VIRT Adapter
OS / Hypervisor / VM
23
Fabric Failover with Bare Metal OS

Slam Dunk: Cisco UCS simplifies the redundancy
No OS NIC Teaming configuration required
Simple single NIC design
6100-A
MAC A
L1 L2 L1 L2
6100-B
MAC A
Fabric failures hidden from OS

NIC stays UP
Server w/ Cisco VIC or Menlo
vNIC
MAC A
6100 sends gratuitous ARP Everything to gain Nothing to lose
WINDOWS / LINUX
Implicit MAC
Learned MAC
24
Fabric Failover with HYPER-V

Slam Dunk: Cisco UCS provides the missing redundancy
6100-A
MAC C
MAC A MAC B
L1 L2 L1 L2
6100-B
MAC C MAC A MAC B
Hyper-V soft switch only uses (1) NIC

Fabric Failover provides the missing redundancy
Server w/ Cisco VIC or Menlo
vNIC 1
MAC C
Everything to gain
Nothing to lose
VM
HYPER V soft switch

VM
MAC A Implicit MAC

BRKCOM-2003
MAC B Learned MAC

Cisco Public
25
Fabric Failover with Hypervisor Pass-Through

Default setting Cannot be disabled Single adapter per VM with Redundancy
6100-A
MAC B MAC A
L1 L2 L1 L2
6100-B
MAC B MAC A
Dynamic
Dynamic
Load Sharing
Alternating fabric Round Robin
vNIC 1
Server w/ Cisco VIC
vNIC 2
VM-FEX
VM 1
VM 2
MAC A Implicit MAC

MAC B Learned MAC

26
Ethernet Switching Modes Recommendations
Scalability
Spanning Tree protocol is not run in EHM hence control plane is unoccupied EHM is least disruptive to upstream network BPDU Filter/Guard, Portfast enabled upstream MAC learning does not happen in EHM on uplink ports. Current MAC address limitation on the 6100 ~14.5K.
Recommendation: End Host Mode

28
Pinning for Deterministic Traffic Flows

Dynamic pinning
Server ports pinned to an uplink port/port-channel automatically
DEFINED: PinGroup Oracle
Static pinning
Specific pingroups created and associated with adapters 6100 A
Fabric A
3 vEth 1
vEth 3
Pinning Switching
Static pinning allows traffic management if required for certain applications / servers
BRKCOM-2003
VNIC 0
VNIC 0
APPLIED: PinGroup Oracle
Server X
Cisco Public
Oracle
29
Fabric Failover
6100-A
L1 L2 L1 L2
6100-B
Fabric Failover is only applicable in EHM.

NIC teaming software required to provide failover in Switch mode.
Physical Cable Virtual Cable
vEth 1
IOM
10G E
vEth 1
IOM
10G E
PHY Adapter
Cisco VIC M81KR Menlo M71KR
vNIC 1
VIRT Adapter
OS / Hypervisor / VM
30
Active/Active use of Uplinks

End Host Mode
Primary Root Secondary Root
Switch Mode
Primary Root
LAN
Secondary Root
LAN
Active/Active Border Ports FI-A Server Ports FI-B
Border Ports FI-A Server Ports
Blocking
FI-B
BRKCOM-2003

Cisco Public
31
Disjoint L2 Upstream
Backup Production
EHM built on the premise that the L2 upstream is NOT disjoint. Incoming broadcast/multicast received only on 1 uplink for ALL VLANs
External LAN
Border Ports
Designated Bcast Receiver Fabric InterConnect
Recommendation: Switch Mode

32
Application Specific Scenarios

Certain application like MS-NLB (Unicast mode) have the need for unknown unicast flooding which is not done in EHM Certain network topologies provide better network path out of the Fabric Interconnect due to STP placement and HSRP L3 hop. Switch Mode is catch all for different scenarios. Recommendation: Switch Mode
33
Upstream Connectivity - Storage
SAN End Host NPV Mode

N-Port Virtualisation Forwarding
SAN A
FLOGI FDISC
SAN B
NPIV
F_Port VSAN 1 N_Proxy 6100-B vFC 1
Fabric Interconnect operates in N_Port Proxy mode (not FC Switch mode) simplifying managment
SAN switch sees Fabric Interconnect as an FC End Host with many N_Ports and many FC IDs assigned Server facing ports function as F-proxy ports Server vHBA pinned to an FC uplink in the same VSAN. Round Robin selection. Provides multiple FC end nodes to one F_Port off an FC Switch
NPIV
F_Port VSAN 1 N_Proxy
6100-A vFC 1 vFC 2
vFC 2
F_Proxy
F_Proxy N_Port vHBA vHBA 0 1
N_Port vHBA vHBA 0 1
Eliminates the FC domain on UCS Fabric Interconnect

One VSAN per F_port (multi-vendor) F_Port Cisco Public Trunking and Channeling with MDS, 5K
35
Server 1
VSAN 1
BRKCOM-2003
Server 2
VSAN 1
SAN End Host NPV Mode

N-Port Virtualisation Forwarding with MDS, Nexus 5000
SAN A
NPIV F_ Port Channel & F_Port VSAN Trunk
1,2
SAN B
NPIV
VSAN 1,2
F_Port Channeling and Trunking from MDS or Nexus 5000 to UCS
FC Port Channel behaves as one logical uplink

FC Port Channel can carry all VSANs (Trunk) UCS Fabric Interconnects remains in NPV end host mode Server vHBA pinned to an FC Port Channel
N_Proxy 6100-A vFC 1 vFC 2 6100-B vFC 1 vFC 2
F_Proxy N_Port vHBA vHBA 0 1
vHBA 0
vHBA 1
Server vHBA has access to bandwidth on any link member of the FC Port Channel
Load balancing based on FC Exchange_ID
Per Flow
Cisco Public
Server 1
VSAN 1
BRKCOM-2003
Server 2
VSAN 2
36
SAN FC Switch Mode

Direct Attach FC & FCoE Storage to UCS
UCS Fabric Interconnect behaves like an FC fabric switch
FC
FCoE
SAN
MDS MDS
Storage ports can be FC or FCoE

N_Port
Light subset of FC Switching features

Select Storage ports Set VSAN on Storage ports Default zoning per VSAN
VSAN 1 F_Port
VSAN 2 TE_Port
6100-A FC Switch vFC 1 vFC 2 F_Port N_Port vHBA vHBA 0 1
6100-B FC Switch vFC 1 vFC 2
No zoning configuration inputs in UCSM Connection to an external FC switch is reqd:

Zoning configured and pushed to UCS from MDS
vHBA 0
vHBA 1
Fabric Interconnect uses a FC Domain ID

Server 1
VSAN 1
Server 2
VSAN 2
37
Direct Attach IP Storage to UCS

NAS
Volume A Volume B
NAS direct attached to Appliance port Active/Standby

LAN
IP Storage attached to Appliance Port
NFS, iSCSI
C1
NAS appliance with single Controller head Each NAS controller port owns the I/O for a Uplink given Volume
U
vEth 2
Appliance Port A 6100-A

vEth 1
A
6100-B
vEth 1
U Port
vEth 2
Other port provides failover
Controller interfaces active/standby for a given volume when attached to separate 6100s Controller interfaces Active/Active when each handling their own volumes Sub-optimal forwarding possible if not careful
Insure vNICs are accessing Volumes local to its fabric
Cisco Public
vNIC 0
vNIC 1
vNIC 0
vNIC 1
Server 1
Accessing Volume A
BRKCOM-2003
Server 2
Accessing Volume A
38
Upstream Connectivity - Ethernet
End Host Mode Individual Uplinks

Dynamic Re-pinning of failed uplinks

All uplinks forwarding for all VLANs GARP aided upstream convergence No STP Sub-second re-pinning No server NIC disruption
BRKCOM-2003
Sub-second re-pinning
Pinning
Switching
vEth 1
VLAN 10
L2 Switching
VNIC stays up
VNIC 0
MAC A
vSwitch / N1K
ESX HOST 1
VM 1 MAC B VM 2 MAC C
40
VNIC 0
Server 2
Cisco Public
End Host Mode Port Channel Uplinks

Recommended: Port Channel Uplinks
No disruption No GARPs needed

More Bandwidth per Uplink Per flow uplink diversity No Server NIC disruption Fewer GARPs needed Faster bi-directional convergence Fewer moving parts RECOMMENDED
BRKCOM-2003
Sub-second convergence
Pinning Switching
vEth 1
VLAN 10
L2 Switching
NIC stays up
VNIC 0
MAC A
vSwitch / N1K
ESX HOST 1
VNIC 0
Server 2
Cisco Public
41
End Host Mode vPC Uplinks

vPC uplinks hide uplink & switch failures from Server VNICs
vPC Domain
No disruption No GARPs Needed!
More Bandwidth per Uplink Fabric A No Server NIC disruption Switch and Link resiliency Per flow uplink diversity No GARPs Faster Bi-directional convergence Fewer moving parts
6100 A vEth 3
VLAN 10
Pinning
vEth 1
Switching NIC stays up L2 Switching
VNIC 0 VNIC 0 Server 2

Cisco Public
vSwitch / N1K
ESX HOST 1
42
vPC RECOMMENDED
BRKCOM-2003
Inter-Fabric Traffic Example (1)

Cisco UCS VM-FEX
L2 Switching 6100 A
Dynamic
Dynamic VNIC1 Primary fabric A Backup fabric B

EHM
EHM
6100 B
Dynamic
VNIC 1
VNIC 2
Dynamic VNIC2 Primary fabric B Backup fabric A VM1 on VLAN 10 VM2 on VLAN 10 VM1 to VM2 traffic: 1) Leaves Fabric A 2) Gets L2 switched 3) Enters Fabric B
PTS VM 1 VM 2 ESX HOST

Cisco Public
BRKCOM-2003
43
Inter-Fabric Traffic Example (2)

VNIC 0 on Fabric A VNIC 1 on Fabric B VM1 Pinned to VNIC0 VM4 Pinned to VNIC1
L2 Switching 6100 A 6100 B
VM1 to VM4: 1) Leaves Fabric A 2) L2 switched upstream 3) Enters Fabric B
VM1 on VLAN 10 VM4 on VLAN 10

VNIC 0
EHM
EHM
VNIC 1
VNIC 0
VNIC 1
ESX HOST 1
ESX HOST 2
vSwitch / N1K Mac Pinning VM VM 1 2

vSwitch / N1K Mac Pinning VM VM 3 4

44
Singly Attached Uplinks

7K1 7K2
EHM
6100 A
EHM
6100 B
1. Traffic destined for a vNIC on the Red Uplink enters 7K1 2. Same scenario vice-versa for Green 3. All Inter-Fabric traffic traverses Nexus 7000 peer link
45
Recommended Topology
vPC uplinks to L3 aggregation switch
7K1
vPC Domain
keepalive
7K2
vPC peer-link
EHM
6100 A
EHM
6100 B
BRKCOM-2003
Cisco Public
46
Recommended Topology w/o vPC

With 4 x 10G (or more) uplinks per 6100 Port Channels Connect 6100s with End Host Mode to Aggregation L3 switch
EHM
6100 A
EHM
6100 B
All UCS uplinks forwarding No STP influence on the topology End Host Mode
47
Adapter Offerings
Gen1 Compatibility Adapters

NIC and HBA ASICs from: Qlogic, Emulex, Intel
Dual 10GbE/FCoE ports
10GbE/FCoE
Cisco Menlo ASIC

IEEE DCB VN-Tag Fabric Failover
Support for native drivers and utilities

Customer certified stacks
10GbE
FC
21w 4Gbps FC
PCIe Bus
vNIC Fabric Failover

49
Gen2 Compatibility Adapters

Emulex M72KR-E (CNA) QLogic M72KR-Q (CNA)
Single Emulex ASIC design Low power 13w 8 Gbps FC Emulex drivers, Eth & FC
Single QLogic ASIC design Lowest power 4.5w 8 Gbps FC QLogic drivers, Eth & FC
No vNIC Fabric Failover
BRKCOM-2003
Cisco Public
50
Gen2 Cost Adapters

Broadcom BCM57711 Intel M61KR-I
iSCSI acceleration iSCSI offload (HBA) iSCSI boot (future) TCP offload engine (TOE) Low cost 10GE
SR-IOV compatible iSCSI acceleration PXE Boot VMDq IEEE DCB FCoE software (future) No vNIC Fabric Failover
BRKCOM-2003
Cisco Public
51
Cisco Virtual Interface Card (VIC) (Palo)

Converged Network Adapter
FCoE in hardware
Eth
FC FC Eth
10GbE/FCoE
Single-OS and VM deployments
Virtualise in hardware PCIe compliant
Up to 58 distinct PCIe devices (H/W capable of 128)

Ethernet vNIC and FC vHBA
User Definable vNICs

0 1 2 3 58
2nd Tier Fabric Extender
18w
PCIe x16
52
Cisco Virtual Interface Card (VIC) (Palo)

10GbE/FCoE
For virtualisation environments

Bypass vSwitch to deliver VN-Link in hardware Tight integration with VMware vCenter vNIC as Hardware DVS-port
Eth
FC FC Eth
QoS
(8) COS based queues vNIC bandwidth guarantees
User Definable vNICs

0 1 2 3 58
18w
PCIe x16
53
Cisco VIC: VM-FEX Logical View

Fabric Interconnect A
vfc 1 vEth 1 vEth 2 vEth 3 vEth 4
Fabric Interconnect B
vEth 5 vEth 6 vEth 7 vEth 8 vfc 2
IOM A Cisco VIC
IOM B
vhba 0
BRKCOM-2003
VM 1
VM 2
VM 3
VM 4
VM 5
VM 6
VM 7
VM 50
vhba 1
54
Cisco Public
UCS Generation - 2 Hardware
Next Gen UCS Components and Capabilities

Additions to the UCS Fabric Portfolio
6248UP Fabric Interconnect
2x Fabric Capacity 40% Latency Reduction Unified Ports 2x Blade Chassis Bandwidth 160 Gb/s per Chassis
2208XP IO Module
VIC 1280
4x Blade Server Bandwidth Dual 40 Gb/s per Card 256 virtual Interfaces
UCS Version 2.0 Platform Features

Production Vlan 10,20 Public Vlan 31,32 Backup Vlan 40,41
End Host End Host
L2 Disjoint Networks More Flexible Designs Reduced Networking HW iSCSI Boot Support in UCSM Increased Customer Choice VM-FEX for RedHat KVM Additional Hypervisor Support
BRKCOM-2003
Cisco Public
56
UCS 6248UP Fabric Interconnect

Customer benefits
Highest density & performance for Unified Computing Fabric
Feature details
PID: UCS-FI-6248UP Double the port density in 1RU UCS 6248UP chassis comes with 32 fixed Unified Ports and 1 expansion module slot 16 Unified Ports Dual power supplies standard for both AC (at FCS) and DC -48V (July) Redundant front to back airflow
BRKCOM-2003
Cisco Public
57
UCS 6248: Unified Ports

Dynamic Port Allocation: Lossless Ethernet or Fibre Channel
FC Native Fibre Channel Benefits

Simplify switch purchase remove ports ratio guess work Increase design flexibility Remove specific protocol bandwidth bottlenecks
Eth Lossless Ethernet:
1/10GbE, FCoE, iSCSI, NAS

Use-cases
Flexible LAN & storage convergence based on business needs Service can be adjusted based on the demand for specific traffic
Cisco Public
58
UCS 6248: Unified Ports
Dynamic Port Allocation: Lossless Ethernet or Fibre Channel
Ports on the base card or the Unified Port GEM Module can either be Ethernet or FC Only a continuous set of ports can be configured as Ethernet or FC Ethernet Ports have to be the 1st set of ports Port type changes take effect after next reboot of switch for Base board ports or power-off/on of the GEM for GEM unified ports.
Base card 32 Unified Ports GEM 16 Unified Ports
Eth
FC
Cisco Public
Eth
FC
59
UCS 2208XP I/O Module

Feature details PID: UCS-IOM-2208XP
Double the uplinks 8x 10GE uplinks from each IOM/FEX Total 160 Gbps per chassis Quadruple the downlinks 32x 10GE or 4x 10GE from each IOM/FEX to each blade slot* Total: 80 Gbps per server slot* *Requires VIC 1280 for full server bandwidth Increased support for 8 egress CoS queues Lower latency Customer benefits Double the uplink bandwidth to the Fabric Quadruple the downlink bandwidth to the server slots Lower latency and better QoS
BRKCOM-2003
Cisco Public
60
IOM to Fabric Interconnect Port Pinning

Server-to-Fabric Port Pinning Configurations
FAN2 FAN1 FAN2 FAN1 FAN2 FAN1 STAT STAT STAT
FAIL FAIL FAIL FAIL OK OK OK OK
PS2
PS2
PS2
N10-PAC1-550W N10-PAC1-550W
N10-PAC1-550W
N10-PAC1-550W
PS2
OK
OK
OK
OK
N10-PAC1-550W
N10-PAC1-550W
N10-PAC1-550W
N10-PAC1-550W
UCS 5 1 0 8
UCS 5 1 0 8
SLOT
SLOT
SLOT
SLOT 7
Slot 1 Slot 3 Slot 5 Slot 7

OK FAIL OK FAIL

OK FAIL OK FAIL
!
SLOT SLOT
SLOT
SLOT
SLOT 6
SLOT
SLOT
SLOT 7

OK FAIL OK FAIL

OK FAIL OK FAIL
!
SLOT
SLOT
SLOT 6
SLOT
BRKCOM-2003
6100 to 2208 6200to 2208 its affiliates. All rights reserved. 2012 Cisco and/or
Cisco Public
6200 to 2208
61
STAT
FAN STAT
FAN STAT
FAN STAT
FAN STAT
FAIL
FAIL
FAIL
FAIL
FAN2 FAN1
PS1
PS1
PS1
PS1
160 Gb (Discrete Mode)
160 Gb (Port Channel Mode)

FAN STAT FAN STAT FAN STAT FAN STAT
UCS 1280 VIC

Customer benefits
Dual 4x 10 GE (80 Gb per host) UCS 2208 IOM UCS 2208 IOM
Feature details
Dual 4x 10 GE port-channels to a single server slot Host connectivity PCIe Gen2 x16 Side A Side B
HW Capable of 256 PCIe devices

OS restriction apply PCIe virtualisation OS independent (same as M81KR) Single OS driver image for both M81KR and 1280 VIC FabricFailover supported Eth hash inputs : Source MAC Address,Destination MAC
Address,Source Pprt, Destination Port,Source IP address,Destination IP address and VLAN
UCS 1280 VIC
256 PCIe devices
FC Hash inputs: Source MAC Address

Destination MAC Address,FC SID and FC DID and OXID
62
UCS Fabric Component Interoperability

Complete hardware inter-operability between Gen 1 and Gen 2
Fabric Interconnect 6100 6100 6100 6100 IOM 2104 2208 2104 2208 Adapter UCS M81KR UCS M81KR UCS1280 VIC UCS1280 VIC Supported Min Software version required UCSM 1.4(1) or earlier UCSM UCS 2.0 UCSM UCS 2.0 UCSM UCS 2.0
6200
6200 6200 6200
2104
2208 2104 2208
UCS M81KR
UCS M81KR UCS1280 VIC UCS1280 VIC
UCSM UCS 2.0

UCSM UCS 2.0 UCSM UCS 2.0 UCSM UCS 2.0
BRKCOM-2003
Cisco Public
63
End Host Mode Disjointed L2 Domains UCS ver 2.0 and beyond
Hardware/ FI independent The ability to selectively assign VLANs to uplinks (NO overlapping VLANs) Basing pinning decision on border port and vNIC VLAN membership Allocating a designated broadcast/multicast receiver on a per VLAN rather than global basis Max 31 disjoint Layer 2 domains supported
Production Vlan 10,20
Public Vlan 31,32
Backup Vlan 40,41
End Host
End Host
64
Q&A
Complete Your Online Session Evaluation

Complete your session evaluation:
Directly from your mobile device by visiting www.ciscoliveaustralia.com/mobile and login by entering your username and password Visit one of the Cisco Live internet stations located throughout the venue Open a browser on your own computer to access the Cisco Live onsite portal
Dont forget to activate your Cisco Live Virtual account for access to all session materials, communities, and on-demand and live activities throughout the year. Activate your account at any internet station or visit www.ciscolivevirtual.com.
Cisco Public
BRKCOM-2003
66
BRKCOM-2003
Cisco Public
67

Brkcom 2003

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Brkcom 2003

Uploaded by

Copyright:

Available Formats

UCS Networking Deep Dive

Overview / System Architecture

UCS Generation-2 Hardware

Unified Computing System (UCS)

Stateless Servers with Virtualised Adapters

UCS Building Blocks

Cisco UCS Networking: Physical Architecture

Uplink Ports OOB Mgmt Fabric Switch Server Ports

Compute Blades Half / Full width

2012 Cisco and/or its affiliates. All rights reserved.

Cisco UCS Networking: Physical Architecture

Uplink Ports OOB Mgmt Fabric Switch Server Ports

Compute Blades Half / Full width

2012 Cisco and/or its affiliates. All rights reserved.

Network Interface Virtualisation (NIV)

Cable Virtual Cable (VNTag)

Service Profile (Server)

Abstracting the Logical Architecture

What you get

vEth Dynamic, Rapid 1 Provisioning

Location Independence Blade or Rack Adapter

Service Profile (Server)

UCS 6100 Hardware Architecture

2012 Cisco and/or its affiliates. All rights reserved.

1-4 Fabric Ports to Interconnect

Chassis Management Controller

8 Backplane Ports to Blades

2012 Cisco and/or its affiliates. All rights reserved.

UCS IOM Slot to Uplink Pinning

All the HIF ports pinned to the active link

2012 Cisco and/or its affiliates. All rights reserved.

Static Pinning done by the system dependent on number of fabric ports

Blade Server Ports 1

2012 Cisco and/or its affiliates. All rights reserved.

Pinned HIFs are brought down

2012 Cisco and/or its affiliates. All rights reserved.

Addition of links requires reack of chassis.

Ethernet Switching Modes

Switching Modes: End Host

Server vNIC pinned to an Uplink port No Spanning Tree Protocol

6100 A vEth 3 Fabric A

Allows Multiple Active Uplinks per VLAN

2012 Cisco and/or its affiliates. All rights reserved.

End Host Mode Unicast Forwarding

End Host Mode Multicast Forwarding

Switching Modes: Switch

Fabric Interconnect behaves like a normal Layer 2 switch

6100 A vEth 3 Fabric A

Fabric Failover with Bare Metal OS

Fabric failures hidden from OS

Server w/ Cisco VIC or Menlo

6100 sends gratuitous ARP Everything to gain Nothing to lose

Fabric Failover with HYPER-V

Hyper-V soft switch only uses (1) NIC

Server w/ Cisco VIC or Menlo

HYPER V soft switch

MAC A Implicit MAC

MAC B Learned MAC

2012 Cisco and/or its affiliates. All rights reserved.

Fabric Failover with Hypervisor Pass-Through

MAC A Implicit MAC

MAC B Learned MAC