Professional Documents
Culture Documents
Centre Infrastructures
BRKDCT-2615
Carlos Pereira
Distinguished Systems Engineer II – WW Data Centre / Cloud
(with extensive credits and thanks to my fellow Cisco collegues: Victor Moreno, Patrice Bellagamba, Yves Louis, Mike Herbert)
#clmel
Active / Active Data Centres
Everybody wantsthat
Then try to figure thisout
… Then,
… and feeldivide & conquer
tired (or panic )
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Objectives Legend
• Understand the Active/Active Data Centre
requirements and considerations
Load SSL
• Provide considerations for Active/Active DC Balancer Offloader APIC
Design – inclusive for Metro areas - from
storage, DCI, LAN extension, ACI and
network services perspectives Application
SVI / HSRP Policy
IDS / IPS
• Brainstorm about ACI Fabric extension, Default Gw Infrastructure
stretch fabrics, application portability, VM Controller
mobility, policy synchronisation, etc.
• Share Experiences with State-full Devices WAN
placements and their impact within DCI Accelerator Firewall
environment
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Agenda
• Active-Active (A/A) Data Centre:
– Market & Business Drivers
– Terminology, Criticality levels and Solutions
Overview
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Some Important Trends Impacting the Data Centre Evolution
More Workloads are moving to The increasing density of Business Critical workloads hosted in the Cloud is driving new
1 Cloud Data Centres Multi-site designs to handle Business Continuity, Workload Mobility, and Disaster Recovery
Applications PaaS
Infrastructure HyperScale
Traditional Application Centric Data Centres
Data Centre Infrastructure (ACI)
Networking
Network
Network + Services
DC Abstraction & Automation
Switching Apps Policy
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
5 The App Market Transition – From Monolithic To Cloud-aware
IaaS PaaS
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Terminology
• The Terminology around Workload and Business Availability / Continuity is not
always consistent
• Some examples:
“Availability Zone”
• AWS - Availability Zones are distinct locations within a region that are engineered to be
isolated from failures in other Availability Zones
• OpenStack - An availability zone is commonly used to identify a set of servers that have
a common attribute. For instance, if some of the racks in your data centre are on a
separate power source, you can put servers in those racks in their own availability zone.
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Availability Zone and Regions – AWS Definitions
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Availability Zone and Regions - OpenStack Definitions
• Regions - Each Region has its own full
Openstack deployment, including its own
API endpoints, networks and compute
resources. Different Regions share one
set of Keystone and Horizon to provide
access control and Web portal. (Newer
deployments do not share Keystone)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
In-Region and Out-of-Region Data Centres
Business Continuity and Disaster Recovery
• Active/active — Traffic intended for the
failed node is either passed onto an existing
node or load balanced across the remaining
nodes.
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Industry Standard Measurements of Business Continuity
Time to Recover
Data Lost 7
p.m.
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Application Resiliency and Business Criticality Levels
Defining how a Service outage impacts Business will dictate a redundancy strategy (and cost)
Each Data Centre should accommodate all levels… Cost is important factor
Application Environment
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
The Application Environment Spans Many Cloud Resources
DCI Extends Cloud Resources to Support Multi-site Use Cases
Multi DC DC Fabric L4–L7 Hypervisors Compute Storage
WAN & Cloud Networking and Virtual
Services Networking
WAN Connectivity Data Centre Fabrics Physical and Virtual Services Hypervisors Unified Compute System (UCS) Storage
• IP Internet Access • Virtual Port Channel (vPC) • Firewalls • VMware vSphere • C-Series Rack Servers • NetApp
• MPLS VPN Access • VxLAN based (with our • Load Balancers • Microsoft Hyper-V • B-Series Blade Servers • EMC
• Physical or Virtual WAN router without SDN Controller) • IPSec VPN Termination • KVM (ex.: RedHat) • Physical and Virtual Interfaces • Direct Attached
• IP Path Optimisation • FabricPath • WAN Acceleration Service • Port and Security Profiles Storage
(LISP, DNS, Site Selector) • Application Centric • Network Analysis Hypervisor Services
Infrastructure (ACI) • Data Encryption • Live and Cold Integrated PoDs Storage Fabrics
L3 Routing and IGP Application Migrations • FlexPod • FC
• OSPF Fabric Services • Extended Clusters • vBlock • FCoE
• ISIS • Tenancy • High Availability and • Low Cost Compute PoDs • 10GE
• BGP • Secure Segmentation Recovery Services
(VRF, VLAN, VxLAN) • Site Affinity Services Data Replication
Data Centre Interconnect • Traffic QoS • Synchronous
• Overlay Transport • Bandwidth Reservation Virtual Switching • Asynchronous
Virtualisation (OTV) • Nexus 1000v • Hypervisor Based
• EoMPLS, VPLS • Virtual Interfaces • DWDM / IP / FCIP
• E-VPN
APPLICATION TEAMS CHOOSE FROM AVAILABLE DESIGN OPTIONS…
THESE FUNCTIONS ARE EXTENDED TO SUPPORT MULTI -SITE USE CASES
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
DCI Extensions Impact Each Tier of the Cloud Data Centre
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Virtualised Workload Mobility Inter-DCs Cisco-VMware With
EMC & NetApp
DC 1 DC 2
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Data Centre Interconnect
Path Optimisation Options
Egress
Addressed by FHRP Filtering
Ingress:
1. LISP Mobility
2. IGP Assist (A_la_RHI)
DC 1 3. GSLB DC 2
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Ex.1: vSphere Redundancy and Mobility Options Can Extend Across Geographies
VM High Availability VM Mobility Site / VM Recovery
VMware vSphere High Availability (HA) VMware vMotion and Storage vMotion VMware Site Recovery Manager (SRM)
Live Migration of VMs and VM storage Fully automated site recovery and migration
• Non-disruptive migration of VMs • Simple management of recovery and migration plans
• Non-disruptive migration of Virtual storage • Non-disruptive testing
VMware vSphere Fault Tolerance (FT) “Shared Nothing” VMware vMotion VMware vSphere Replication
Live Migration of VMs and storage WITHOUT shared storage Creates VM snapshot copies available for restoration through the vCenter
• Simple management of recovery and migration plans • Continuous replication to another location, within or between clusters
• Non-disruptive testing • Hypervisor based replication, VM granularity
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Ex.2: Hyper-V Redundancy and Workload Mobility Options Can Extend Across Geographies
Hyper-V Live Migration with Shared SMB Storage Hyper-V “Shared Nothing” Live Migration
DC-1 DC-2
DC-1 DC-2
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
VMware vSphere
and Data Centre 1
Microsoft Hyper-V
environments are
concurrently supported
in Cisco Cloud
solutions
Data Centre 2
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda
• Active-Active (A/A) Data Centre:
– Market & Business Drivers
– Terminology, Criticality levels and
Solutions Overview
• A/A Data Centre Design Considerations:
– Storage Extension
– Data Centre Interconnect (DCI) – L2 &
L3 scenarios
• A/A Metro Data Centres Designs
- Network Services and Applications (Path
optimisation)
• Cisco ACI and Active / Active Data Centre
• Q&A
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
Data Centre Interconnect
SAN Extension
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
SAN Extension
Synchronous vs. Asynchronous Data Replication
• Synchronous Data replication: The Application receives the acknowledgement for I/O
complete when both primary and remote disks are updated. This is also known as Zero
data loss data replication method (or Zero RPO)
– Metro Distances (depending on the Application can be 50-300kms max)
Synchronous Asynchronous
Data Replication Data Replication
4 1 2 1
2 3
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Synchronous Data Replication
Network Latency
Speed of Light is about 300000 Km/s
Speed is reduced to 200000 Km/s 5 μs per Km (8 μs per Mile)
That gives us an average of 1ms for the light to cross 200 Km of fibre
50 Kilometers 1ms
250 μs : Rec_Ready ?
1
250 μs : Wait for response?
2
250 μs : Send data
1
250 μs : Wait for Ack?
Core Network
DC 1 DC 2
Initiator
ESX-A source ESX-B target
Virtual Center
Volumes
Target
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Storage Deployment in DCI
Shared Storage Improvement Using Cisco IOA
Core Network
DC 1 DC 2
Virtual Center
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Storage Deployment in DCI
Option 2 - NetApp FlexCache (Active/Cache)
Core Network
DC 1 DC 2
NAS Read
Write ?
2 Temp
data
Read
Write
ESX-A source data 3 Cache ESX-B target
data
ACK 1
4
2 data
ACK
Virtual Center
FlexCache does NOT act as a write-back cache
FlexCache responds to the Host only if/when the original subsystem ack’ed to it
No imperative need to protect a Flexcache from a power Failure
BRKDCT-2615 http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/4.0/Netapp/dciNetapp.html
© 2015 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Storage Deployment in DCI
Option 3 - EMC VPLEX Metro (Active/Active)
Synchronous Latency
Distributed Virtual Volume
Fibre Channel
BRKDCT-2615
DC A
© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
DC B
Storage Deployment in DCI
Option 3 - EMC VPLEX Metro (Active/Active)
Core Network
DC 1 DC 2
Initiator
ESX-A source ESX-B target
Virtual Center
Target
From the Storage
LUNv LUNv
EMC
Initiator CLARiiON
EMC
VMAX
VPLEX Synchronous Latency requiments 5ms max VPLEX
Target Engine Engine
BRKDCT-2615 http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/4.0/
© 2015 Cisco and/or its affiliates. All rights reserved.
Cisco Public EMC/dciEmc.html
Storage Deployment in DCI
Option 4 - EMC VPLEX Geo (Active/Active)
Active/Active Storage Virtualisation Platform for the Private and
Hybrid Cloud
Enables workloads Mobility over Long distance at Asynchronous
distances using Microsoft Hyper-V.
DC A DC B
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
http://www.emc.com/collateral/hardware/white-papers/h8214-application-mobility-vplex-geo-wp.pdf
Agenda
• Active-Active (A/A) Data Centre:
– Market & Business Drivers
– Terminology, Criticality levels and
Solutions Overview
• A/A Data Centre Design Considerations:
– Storage Extension
– Data Centre Interconnect (DCI) – L2
& L3 scenarios
• A/A Metro Data Centres Designs
- Network Services and Applications
(Path optimisation)
• Cisco ACI and Active / Active Data
Centre
• Q&A
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Extending Virtual Tenant Space Outside the Fabric
Logical view of Multi-tier Applications
IP network IP network
Maintain the L2 segmentation End
to End toward the remote DC
Maintain the L3
segmentation of the
Layer 3 Edge public networks (VRF) Layer 3 Edge Gateway
Gateway to the outside world
Web Web
App App
DB DB
Type T0
Limited to a single access layer device
Type T1
Extended inside an aggregation block (POD)
Type T2
Extended between PODs part of the same DC
site
Type T3
Extended between PODs part of twin DC sites
connected via dedicated dark fibre links
T4 (DWDM)
Type T4
Extended between PODs part of twin DC sites T3 (Dark Fibre)
connected via xWDM links
Type T5 T1 T2
Extended between PODs part of distant remote
DC sites T0
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
LAN Extension for DCI
Technology Selection
Over dark fibre or protected D-WDM
VSS & vPC
Dual site interconnection Metro style
Ethernet
FabricPath
Multiple site interconnection
MPLS Transport
EoMPLS
Transparent point to point
VPLS
SP style
MPLS
Large scale & Multi-tenants, Point to Multipoint
E-VPN
Large scale & Multi-tenants, Point to Multipoint
IP Transport
OTV
Enterprise style Inter-site MAC Routing IP style
IP LISP
For Subnet extension and Path Optimisation
VXLAN (future for DCI)
Emerging limited A/A site interconnect (requires anycast gateway)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Dual Sites Interconnection
Leveraging MECs Between Sites
At DCI point:
STP Isolation (BPDU Filtering) Layer 2 only
And/ Or
Broadcast Storm Control Static L3
FHRP Isolation
BRKDCT-2615 vPC-vPC
© 2015 Cisco and/or <1.5 reserved. <1.6
its affiliates. All rights <2.8
Cisco Public <2.5 <1.2 <0.2 <0.2 <0.2
MACSec for Secure Data Centre Interconnect
Single Access dark Fibre Connectivity
VPLS
EoMPLS
Targeted-LDP
interfac PE1 NH: PE1
e VC: VCID NH: PE2
Label: 8 VC: VCID
Circuit type: Label: 27
xconnect <PE2> <VCID>
Ethernet type 4/5 Circuit type:
Ethernet type 4/5
LDP/RSVP
PE2
interfac
e
Active PW
Si
MPLS Core
Si
DCI DCI
Aggregation Layer
Active PW Aggregation Layer
DC1 DC2
interface port-channel70
description L2 PortChannel to DC 2
BPDU Filtering to maintain STP domains isolation spanning-tree port type edge trunk
spanning-tree bpdufilter enable
Storm-control for data-plane protection storm-control broadcast level 1*
Configuration applied at aggregation layer on the logical storm-control multicast level x *Value to be tuned, min is 0.3
port-channel interface
47
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Multi-Point Topologies
What Is VPLS?
PW
VFI
VLAN VLAN
MPLS
Core
SVI VFI SVI
PW
PW
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
VPLS Cluster Solutions
• Using clustering mechanism SUP720+ES
–Two devices in fusion as one
• VSS Sup720
• VSS Sup2T
• ASR9K nV virtual cluster SUP2T
One control-plane / two data-planes
• Dual node is acting as one only device
• Native redundancy (SSO cross chassis)
• Native load balancing
• Capability to use port-channel as attachment circuit ASR9K nV
PW
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cluster VPLS – Redundancy
Making Usage of Clustering
X Si Si
Si Si
224 412
Edge Ether-channel convergence in sub-second
Bridged
traffic 326 316
Traffic is directly going to working VSS node
Traffic exits directly from egress VSS node
Quad sup SSO for SUP2T since 1QCY13
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
VPLS
VPLS - Deployment
Cluster - Consideration
Deployment Consideration
ECMP Core
ECMP Core Requirements
Requirements
DC-1 DC-2
Build a symmetric core with two ECMP paths between each VSS
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Since 6.2(2)
Nexus 7000
Data Centre Interconnect with VPLS
VLAN tied to Active VFI
with neighbours to
Layer 2 switchport Trunk Portchannel
Primary N7K WAN Edge remote DC sites
VFI
VLAN X
Vlan X
VFI
Vlan X VLAN X
VFI VFI
Note: Virtual Port Channel (vPC) configuration not shown
PE2 PE4
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
10.0.0.2 10.0.0.4 54
E-VPN
Main Principles
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Solution Overview
(draft-ietf-l2vpn-pbb-evpn-)
DF Election with VLAN Carving PE PE
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
PBB-EVPN Principle (Provider Backbone Bridge)
BGP MAC Routing
BGP update: iBGP L2-NLRI
Virtual B-MAC: X Virtual B-MAC: Y
• next-hop: PE1
Shared by both PEs Shared by both PEs
• X, Label 100
Per each multi-home segment Per each multi-home segment
PE PE
PE1 PE3
M3
M1
S3 Host 3
Host 1 S1
• PE1 and PE2 share the same virtual B-MAC (: X), for the same multi-homing segment. In case of mc-
lag, PE can learn the virtual B-MAC according to Switch 1 LACP system MAC automatically
• Both PE1 and PE2 advertise virtual B-MAC to the remote PEs via BGP
• Remote PE3 and PE4 receive the BGP route, and install the virtual B-MAC in its L2FIB table. BGP
policy could be used for active/active path or primary/standby path
• PE3 and PE4 does the same thing, PE1 and PE2 learn the virtual B-MAC: Y
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
PBB-EVPN Principle (Provider Backbone Bridge)
Packet Forwarding
PE PE
PE1 PE3
M3
M1
S3 Host 3
Host 1 S1
PE2 PE4
VPN C-MAC NH B-MAC NH
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Overlay Transport Virtualisation (OTV) in a Nutshell
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Overlay Transport Virtualisation
OTV Control Plane
4
VLAN MAC IF
1 100 MAC A IP A
3 New MACs are
learned on VLAN 100
OTV updates exchanged via 100 MAC B IP A
the L3 core
Vlan 100 MAC A 3 300 MAC C IP A
300 MAC C IP A
South
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
OTV Overview
Inter-Sites Packet Flow
4
MAC TABLE Transport MAC TABLE
VLAN MAC IF Infrastructure VLAN MAC IF
Decap
100 MAC 1 Eth 2 IP A 3 5 IP B 100 MAC 1 IP A
MAC 1 MAC 3
MAC 1 MAC 3 West East
Server 1 Site Site Server 3
1 7
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
New Features
• F1/F2E used as Internal Interfaces
• Selective Unicast Flooding
Starting 6.2(2)
• Dedicated Data Broadcast Group
• OTV VLAN Translation
• OTV Fast Convergence
interface Overlay1
otv join-interface port-channel100
otv broadcast-group 239.1.1.5
otv control-group 239.1.1.1
otv data-group 232.1.1.0/24
otv extend-vlan 200-209
64
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Improving OTV Design
Physical Join interface
OTV OTV
∂
Internal Internal
Single physical Join interface Single physical Join interface
interface interface
hello
∂ hello
OTV OTV
hello hello
hello hello
hello hello
OTV OTV
Def Def
GWY GWY
OTV OTV
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
Improving OTV Design
Loopback Join interface
OTV OTV
• Secondary interface can be leveraged to better
load distribute the traffic over multiple layer 3
paths
Def Def
GWY GWY
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Placement of the OTV Edge Device
Option 1 - OTV in the DC Aggregation
Layer 2 Link
OTV VDC deployment replicated Layer 3 Link
OTV Virtual Link
in each POD
Inter-PoD & inter-DC LAN Data Centre
extension with a pure L3 core
Isolated STP domain in each POD
STP filtered across the OTV overlay by
L3
default
STP Root L2 STP Root
Independent STP root bridge per POD
vPC facing the access layer
devices
Loop free topology inside each POD
Loop free topology per POD can be also
be through FabricPath.
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Placement of the OTV Edge Device
Option 2 - OTV at the Aggregation with L2-L3 Boundary on External Firewalls
L2
Aggregation
Firewall Firewall
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
DCI Convergence Summary
Robust HA is the guiding principle
Common Failures:
1. Core failures
Multipath routing (or TE FRR) sub-sec OSFP
2. Join interface failures Core
Link Aggregates across line-cards sub-sec 1
4. ED component failures 1x
HW/SW resiliency sub-sec
4 1x 2
Aggregation 4x
Extreme failures (unlikely): OTV
VDC
OTV
VDC
1x. Core partition 3
3x. Site partition 4
LISP Behaviour
Loc/ID “Split”
IP core
10.1.0.1 When the Device Moves, Keeps
Device IPv4 or IPv6 1.1.1.1 Its IPv4 or IPv6 Address.
Address Represents 2.2.2.2 It Has the Same Identity
10.1.0.1
Identity Only.
Its Location Is Here!
Only the Location Changes
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
A LISP Packet Walk
How does LISP operate? 3
EID-prefix: 10.2.0.0/24
Mapping Locator-set:
Entry Non-LISP site
This Policy Controlled
1 2.1.1.1,
Non-LISPpriority:
site 1, weight: 50 (D1)
DNS Entry: by Destination Site
2.1.2.1, priority: 1, weight: 50 (D2)
D.abc.com A 10.2.0.1
10.1.0.0/24
LISP Site
S ITR PITR
2 1.1.1.1 5.4.4.4
10.1.0.1 -> 10.2.0.1 IP Network 5.3.3.3
EID-to-RLOC
4 mapping
1.1.1.1 -> 2.1.1.1 5.1.1.1 5.2.2.2
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
A LISP Packet Walk
After moving the host 3
EID-prefix: 10.2.0.0/24
Mapping Locator-set:
Entry Non-LISP site
1 3.1.1.1,
Non-LISPpriority:
site 1, weight: 50 (D1) Detect the move and
DNS Entry: 3.1.2.1, priority: 1, weight: 50 (D2) update mappings
D.abc.com A 10.2.0.1
10.1.0.0/24
LISP Site
S ITR PITR
2 1.1.1.1 5.4.4.4
10.1.0.1 -> 10.2.0.1 IP Network 5.3.3.3
4 EID-to-RLOC
mapping
1.1.1.1 -> 3.1.1.1 5.2.2.2
5.1.1.1
10.1.0.1 -> 10.2.0.1
2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1
ETR
5
West-DC East-DC
10.1.0.1 -> 10.2.0.1
D
10.2.0.0/24 10.3.0.0/24
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
A LISP Packet Walk 3
EID-Prefix: 10.2.0.0/24
How about Non-LISP Sites? Mapping Locator-Set:
1 Entry 2.1.1.1, priority: 1, weight: 50 (D1)
DNS Entry:
D.abc.com A 10.2.0.1 2.1.2.1, priority: 1, weight: 50 (D2)
Non-LISP Site
Non-LISP Site
S
2
192.3.0.1 -> 10.2.0.1 PITR
4.4.4.4
4
5.3.3.3
4.4.4.4- > 2.1.2.1 EID-to-RLOC
192.3.0.1 -> 10.2.0.1 mapping
5.1.1.1 5.2.2.2
IP Network
2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1
ETR
5
192.3.0.1 -> 10.2.0.1West-DC East-DC
D 10.2.0.0/24 10.3.0.0/24
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
LISP Roles and Address Spaces MappingEID RLOC
(ITR/ETR)
PxTR RLOC Space
• Proxy Tunnel Routers - PxTR ETR
• Coexistence between LISP and
non-LISP sites EID Space
• Ingress/Egress: PITR, PETR
• EID to RLOC Mapping DB Address Spaces
• EID = End-point Identifier
• RLOC to EID mappings
• Host IP or prefix
• Distributed across multiple Map • RLOC = Routing Locator
Servers (MS) • IP address of routers in the backbone
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
LISP Host-Mobility
Needs:
Non-LISP Sites
• Global IP-Mobility across subnets
• Optimised routing across extended subnet sites LISP Site PXTR
XTR
LISP Solution: Mapping DB
IP Network
• Automated move detection on XTRs
• Dynamically update EID-to-RLOC mappings
• Traffic Redirection on ITRs or PITRs
LAN Extensions
Benefits:
LISP-VM (XTR)
• Direct Path (no triangulation) West-DC East-DC
• Connections maintained across move
• No routing re-convergence
• No DNS updates required
• Transparent to the hosts
RLOC EID LISP Encap/Decap
• Global Scalability (cloud bursting)
• IPv4/IPv6 Support
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Host-Mobility Scenarios
Moves Without LAN Extension Moves With LAN Extension
DR Location or Mapping DB
Mapping DB
Cloud Provider
IP Network
DC
Internet or
Shared WAN
LAN Extension
DC-Access
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Agenda
• Active-Active (A/A) Data Centre:
– Market & Business Drivers
– Terminology, Criticality levels and
Solutions Overview
• A/A Data Centre Design
Considerations:
– Storage Extension
– Data Centre Interconnect (DCI) –
L2 & L3 scenarios
• A/A Metro Data Centres Designs
- Network Services and Applications
(Path optimisation)
• Cisco ACI and Active / Active Data
Centre
• Q&A
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
Example: 3-Site Data Centre Interconnect Model
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
Metro Virtual Data Centre
High-availability application and data solution architecture which leverages
a dual data centre physical infrastructure
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Active / Active Metro Design Cool ! What if they are both
ACI on the Metro sites ?
To come on the 2nd half of
this session
Cisco Public
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Example Active-Active Metro Topology
Cisco Public
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Extended Storage Example: Multi-Hop FCoE using NetApp Fabric MetroCluster
Optical
Network
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Nexus 1000v Extensions Across Geographies
VSMs and VEMs can span Metro distances for enhanced availability
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Live Workload Migration Baseline Configuration for a 3-Tier Application
The Application, Data, and Services are Operating in Data Centre 1 (Microsoft SharePoint, SQL example)
3-Tier Application
(Web, App, Database) in a
Palladium Container in DC-1
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Step 1 of a Live Workload Migration:
Live Migrate the Application, Data, and Virtual Services from Data Centre 1 to Data Centre 2
Live vMotion all 3-Tiers of the
Application to new hosts in DC-2
(Web, App, Database)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Step 2 of a Live Workload Migration:
Cutover to a new Network Container in DC-2… the Application, Data, and all Services are Moved to Data Centre 2
Synchronous Storage
APP DATABASE
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda
• Active-Active (A/A) Data Centre:
– Market & Business Drivers
– Terminology, Criticality levels and
Solutions Overview
• A/A Data Centre Design
Considerations:
– Storage Extension
– Data Centre Interconnect (DCI) –
L2 & L3 scenarios
• A/A Metro Data Centres Designs
- Network Services and Applications
(Path optimisation)
• Cisco ACI and Active / Active Data
Centre
• Q&A
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
Network Service Placement for Metro Distances
A/S stateful devices stretched across 2 locations – nominal workflow
• Historically this has been
well accepted for most of
Metro Virtual DC (Twin-DC)
L3 Core
• Network Services are usually active on
primary DC
• Distributed pair of Act/Sby FW & SLB on
Outside VLAN
each location
• Additional VLAN Extended for state
FW FT and session synch
synchronisation between peers
Inside VLAN
• Source NAT for SLB VIP
Src-NAT
scenario is limited to 2 sites
SLB session synch
Front-end VLAN
Subnet A
Subnet A Back-end VLAN
L3 Core
• FW failover to remote site
• Front-end server farm moves to
remote site
Outside VLAN
• Source NAT for SLB VIP maintains
the return path thru the Active SLB
• Partial move of a server farm is not
optimised
Inside VLAN
• Understand and identify the multi-
tier frameworks
VIP VIP VLAN
Src-NAT
Front-end VLAN
Subnet A
Subnet A Back-end VLAN
Outside VLAN
• It is preferred to migrate the whole multi-
tier framework and enable FHRP filtering
to reduce the trombone effect
Inside VLAN
• FHRP filtering is ON on the Front-
end & Back-end side gateways
VIP VLAN • Source NAT for SLB VIP maintains the
Src-NAT return path thru the Active SLB
HSRP • Understand and identify the multi-tier
Front-end VLAN Filter frameworks
Subnet A
Subnet A Back-end VLAN
Subnet A Subnet A
Back-End VLAN
BRKDCT-2615 DC-1 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public DC-2
Can Cisco ASA Firewall help here ?
Current Statement:
Non currently supported: Spanned Interface mode with ASA in Firewall Routed mode
97
ASA Clustering (9.0)
TCP SYN cookies with Asymmetrical Traffic workflows
Cluster Control Link (CCL)
Director
Owner
SYN
SYN
SYN/ACK
SYN/ACK
SYN/ACK
Client Server
3) SYN/ACK arrives at non-owner unit
4) decodes the owner information from the SYN cookie
5) forward packet to the owner unit over CCL
Outside Inside
Network Network
It is possible that the SYN/ACK from the server arrives at a non-owner unit before the connection is built at the director.
• As the owner unit processes the TCP SYN, it encodes within the Sequence # which unit in the cluster is the owner
•BRKDCT-2615
Other units can decode
© 2015 that
Cisco and/or information
its affiliates. and forward
All rights reserved. Ciscothe
PublicSYN/ACK directly to the owner without having to query the director
Single ASA Clustering Stretched Across Multiple DC
Case 1: LISP Extended Subnet Mode with ASA Clustering (Stateful Live migration with LAN extension)
M-DB • One Way Symmetric Establishment is achieved via
the CCL
• Current active sessions are maintained stateful
• New Sessions are optimised dynamically
• Up to 10ms max one-trip latency, extend the ASA
ITR cluster for config and state synch
L3 Core
Update your Table
BRKDCT-2615
DC-1
© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
DC-2
Recommendations
1. Layer 2 extensions represent a challenge for optimal routing
2. Consider the implications of stretching the network and security
services over multiple DCs
3. For migration over long distances, when possible enable
network path optimisation for traffic :
Client to server communication (Ingress Optimisation)
Server to Client communication for symmetrical return traffic
(Egress optimisation)
Server to Server communication (bandwidth and Latency
optimisation)
4. Otherwise provision enough bandwidth (2 times the needs) and
compute the total latency due to ping-pong workflows
5. When moving a VM /Tier, move all the framework
6. Network and security policies must be maintained
7. Consider FW clustering stretched across DC’s to reduce the
hair-pining workflow
http://yves-louis.com/DCI/?p=785 (Post 27.n)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
Agenda
• Active-Active (A/A) Data Centre:
– Market & Business Drivers
– Terminology, Criticality levels and
Solutions Overview
• A/A Data Centre Design
Considerations:
– Storage Extension
– Data Centre Interconnect (DCI) – L2
& L3 scenarios
• A/A Metro Data Centres Designs
- Network Services and Applications
(Path optimisation)
• Cisco ACI and Active / Active Data
Centre
• Q&A
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 101
ACI Introduces Logical Network Provisioning of Stateless
Hardware with Application Network Profile (ANP)
Web App DB
ACI Fabric
Application Policy
Infrastructure
Controller
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 102
Fabric Infrastructure
Important Concepts – Inside and Outside APIC
‘Outside’ EPG associated Forwarding Policy for ‘inside’ EPG’s defined by associated
with external network Bridge Domain network policies
policies (OSPF, BGP, …
peering)
Web App DB
EP EP EP EP EP EP EP EP EP EP EP EP
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Connection to Outside Network
Relationship to rest of components (Connectivity view)
L2 Network MPLS/IP Network
Private Network Sophia (VRF 10) Private Network Antipolis (VRF 20)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Extend L2 Domain Out of ACI Fabric
3 options (2 available)
• Three ways of extend L2 domain beyond ACI fabric
1. Manually assign a port to a VLAN which in turn
mapped to an EPG. This extend EPG beyond ACI Fabric
ACI fabric
2. Create a L2 connection to outside network.
Extend bridge domain beyond ACI fabric. VLAN 10
Allow contract between EPG inside ACI and trunk
EPG outside of ACI
3. Remote VTEP (future)
Layer-2 DCI
(OTV, VPLS, xWDM, Dark fibre)
WEB EPG
(VLAN10)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI L2 Connection to Outside Network
Integration with brownfield DC and Migration purpose
BGP Route Reflector 1. Extends the Bridge Domain
ACI Fabric 1 DC-1 beyond ACI fabric mapping
AS#300 the Leaf/Interface/VLAN to
the External EPG
Bind a physical
2. Apply contract (filter, QoS…)
Leaf/Interface/vlan_ID
3. Interface-VLAN trunk to
1 ACI Fabric to an Ext. EPG X
outside
4. L2 adjacency establishment
EPG A between the BD and outside
EPG B 3 the fabric
Bridge Domain “DC1-green”
Bridge Domain “DC1-yellow L2
5. Traffic is policed in one or
EPG EPG
Bridge Domain “DC1-blue”
EPG APP EPG WEB
L2 external both ways according to the
L2 external EPG Z
EPG APP EPG WEB external EPG Y contract
A B EPG X
2
Via the Extended Bridge Network configuration Standard Dot1Q Use cases
VLAN • Extend to legacy Access PoD
• Map the interface VLAN to an External EPG
• Migration purposes
• One External EPG per Bridge Domain VLAN 100 4 • UCS/F.I. attachment
• Ethernet: Interface, VLAN, VxLAN (future)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Single Fabric Scenarios
Single Site (Single Pod)
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Single Fabric Scenarios
Multi-Site (Stretched) Fabric
Site/Room ‘A’ Site/Room ‘B’
Interconnect Leaf
Nodes
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Single Fabric Scenarios
Multi-Site (Stretched) Fabric
Site/Room ‘A’ Site/Room ‘B’
Interconnect Leaf
Nodes
Interconnect Leaf
Nodes
Standby Group
VTEP IP VNID Tenant Packet
APIC Policy
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Multi-Fabric Scenarios
APIC Controller Domain APIC Controller Domain
Fabric ‘A’ Fabric ‘B’
Web1 Web2
App1 App2
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Multi-Fabrics – Current Options
External Synchronisation of Fabric Policy
Site ‘A’ Site ‘B’
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Multi-Fabrics – Current Options
Multiple Domains
Site ‘A’ Site ‘B’
• Multiple APIC Clusters (N+1 Redundancy • Replication of VRF, Bridge Domain and
for each Fabric) EPG name spaces (multiple networks)
• Multiple Operational Domains (Changes • Layer 2 and/or Layer 3 interconnect
are immediately propagated to all nodes between fabrics
‘within’ each fabric, external tools correlate • VMM Clusters can be extended if ‘not’
across fabrics) integrated with APIC
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Multi-site Abstraction and Portability of Network Metadata
and Docker-based Applications
TechWise TV
http://blogs.cisco.com/datacenter/busines Business Continuity and Workload Mobility
http://www.cisco.com/c/en/us/td/docs/solutions/Enter
s-continuity-and-workload-mobility-for- http://cs.co/9000my6i
prise/Data_Center/VMDC/DCI/1-0-1/DG/DCI.html
the-private-cloud-cisco-validated-design-
part-1/
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public
Q&A
Complete Your Online Session Evaluation
Give us your feedback and receive a
Cisco Live 2015 T-Shirt!
Complete your Overall Event Survey and 5 Session
Evaluations.
BRKDCT-2615 © 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public