You are on page 1of 7

Cisco FabricPath is a proprietary protocol that uses ISIS to populate a routing table that is used for

layer 2 forwarding.
Whether we like or not, there is often a need for layer 2 in the Datacenter for the following reasons:

Some applications or protocols require to be layer 2 adjacent


It allows for virtual machine/workload mobility
Systems administrators are more familiar with switching than routing
A traditional network with layer 2 and Spanning Tree (STP) has a lot of limitations that makes it less
than optimal for a Datacenter:

Local problems have a network-wide impact


The tree topology provides limited bandwidth
The tree topology also introduces suboptimal paths
MAC address tables dont scale
In the traditional network, because STP is running, a tree topology is built. This works better for for
flows that are North to South, meaning that traffic passes from the Access layer, up to Distribution, to
the Core and then down to Distribution and to the Access layer again. This puts a lot of strain on Core
interconnects and is not well suited for East-West traffic which is the name for server to server traffic.
A traditional Datacenter design will look something like this:

If we want end-to-end L2, we could build a network like this:

What would be the implications of building such a network though?

Large failure domain


Unknown unicast and broadcast flooding through large parts of the network
A large number of STP instances needed unless using MST
Topology change will have a large impact on the network and may cause flooding
Large MAC address tables
Difficult to troubleshoot
A very brittle network
So lets agree that we dont want to build a network like this. What other options do we have if we still
need layer 2? One of the options is Cisco FabricPath.
FabricPath provides the following benefits:

Reduction/elimination of STP
Better stability and convergence characteristics
Simplified configuration
Leverage parallell paths
Deterministic throughput and latency using typical designs
VLAN anywhere flexibility
The FabricPath control plane consists of the following elements:

Routing table Uses ISIS to learn Switch IDS (SIDs) and build a routing table
Multidestination trees Elects roots and builds multidestination trees
Mroute table IGMP snooping learns group membership at the edge, Group Member LSPs
(GM-LSPs) are flooded by ISIS into the fabric
Observe that LSPs has nothing to do with MPLS in this case and that this is not MAC based routing,
routing is based on SIDs.
FabricPath ISIS learns the shortest path to each SID based on link metrics/path cost. Up to 16 equal
(ECMP) routes can be installed. Choosing a path is based on a hashing function using Src IP/Dst
IP/L4/VLAN which should be good for avoiding polarization.
FabricPath supports multidestination trees with the following capabilities:

Multidestination traffic is contained to a tree topology, a network-wide identifier (Ftag) is


assigned to each tree
A root switch is elected for each multidestination tree
Multipathing is supported through multiple trees
Note that root here has nothing to do with STP, think of it in terms of multicast routing.

Multidestination trees do not dictate forwarding for unicast, only for multidestination packets.
The FabricPath data plane behaves according to the following forwarding rules:

MAC table Hardware performs MAC lookup at CE/FabricPath edge only


Switch table Hardware performs destination SID lookups to forward unicast frames to other
switches
Multidestination table A hashing function selects the tree, multidestination table identifies on
which interfaces to flood based on selected tree
The Ftag used in FabricPath identifies which ISIS topology to use for unicast packets and for
multidestination packets, which tree to use.
If a FabricPath switch belongs to a topology, all VLANs of that topology should be configured on that
switch to avoid blackholing issues.
FabricPath supports 802.1p but can also match/set DSCP and match on other L2/L3/L4 information.
With FabricPath, edge switches only need to learn:

Locally connected host MACs


MACs with which those hosts are bidirectionally communicating
This reduces the MAC address table capacity requirements on Edge switches.
FabricPath Designs
There are different designs that can be used together with FabricPath. The first one is routing at the
Aggregation layer.

The first design is the most classic one where STP has been replaced by FP in the Access layer and
routing is used above the Aggregation layer.
This design has the following characteristics:

Evolution of current design practices


The Aggregation layer functions as FabricPath spine and L2/L3 boundary
FabricPath switching for East West intra VLAN traffic
SVIs for East West inter VLAN traffic
Routed uplinks for North South routed flows
Access layer provides pure L2 functions
FabricPath core ports facing Aggregation layer
CE edge ports facing hosts
Optionally vPC+ can be used for active/active host connections

This design is the simplest option and is an extension of regular Access/Aggregation designs. It
provides the following benefits:

Simplified configuration
Removal of STP
Traffic distribution over all uplinks without the use of vPC
Active/active gateways
VLAN anywhere at the Access layer
Topological flexibility
Direct-path forwarding option
Easily provision additional AccessAggregation bandwidth
Easily deploy L4-L7 services
Can use vPC+ towards legacy Access switches
There is also the centralized routing design which looks like the following:

Centralized routing has the following characteristics:

Traditional Aggregation layer becomes pure FabricPath spine


Provides uniform any-to-any connectivity between leaf switches
In simplest case, only FabricPath switching occurs in spine
Optionally, some CE edge ports exist to provide external router connections
FabricPath leaf switches, connecting to spine, have specific personality
Most of the leaf switches will provide server connectivity, like traditional access switches in Routing
at Aggregation designs
Two or more leaf switches provide L2/L3 boundary, inter-VLAN routing and North-South routing
Other or same leaf switches may provide L4-L7 services

Decouples L2/L3 boundary and L4-L7 services provisioning from Spine


Simplifies Spine design
The different traffic flows in this design looks like the following:

Another design is the multi-pod design which can look like the following:

The multi-pod design has the following characteristics:

Allows for more elegant DC-wide versus pod-local VLAN definition/isolation


No need for pod-local VLANs to exist in core
Can support VLAN id reuse in multiple pods

Define FabricPath VLANs -> map VLANs to topology -> map topology to FabricPath core
port(s)

Default topology always includes all FabricPath core ports


Map DC-wide VLANs to default topology
Pod-local core ports also mapped to pod-local topology
Map pod-local VLANs to pod-local topology
This post briefly describes Cisco FabricPath which is a technology for building scalable L2 topologies,
allowing for more bisectional bandwidth to support East-West flows which are common in Datacenters.
To dive deeper into FabricPath, visit the Cisco Live website.

Thanks to Daniels.

You might also like