Understanding VxRail Network Integration Requirements

The VxRail appliance product requires specific switching infrastructure, configuration and physical
connectivity to enable appliance operations and usage. The switching infrastructure and required
configuration effort is not included with the purchase of the VxRail appliance. This guide is targeted
for the network administrator at a VxRail customer site, and provides an explanation on the specifics
of VxRail network integration.

VXRAIL NETWORK INTEGRATION INTRODUCTION 2

VxRail Physical and Virtual Network Integration 2
VxRail Virtual Networks 2
VxRail Virtual Network Segmentation 2
CONFIGURATION RULES FOR THE ‘TOP OF RACK’ ETHERNET SWITCHES 4
Configuration rules for multicasting on the ‘top of rack’ switches 4
Configuration rules for Virtual SAN VLAN on the ‘top of rack’ switches 4
Configuration rules for Management VLAN on the ‘top of rack’ switches 4
Configuration rules for switch port connected to VxRail nodes 4
Configuration rules for switch ports supporting upstream traffic to core network 4
Configuration rules for inter-switch links (ISLs) 4
TOP OF RACK SWITCH CONFIGURATION EXAMPLES 5
VXRAIL PHYSICAL AND LOGICAL NETWORKING TOPOLOGY 7
VxRail Network Integration Introduction
The EMC VxRail hyper-converged appliance integrates a pool of ESXi hosts and network infrastructure to
work as a single unified platform to enable the delivery of virtualized applications. The application
delivery platform requires the customer’s network infrastructure and configuration settings to be in
compliance with VxRail’s specifications and requirements to enable unified management, support for
virtual machine traffic and mobility, and support for a shared virtual datastore. The customer’s network
infrastructure and settings must be in compliance with the VxRail specifications and requirements prior
to the on-site implementation of the appliance to ensure a successful deployment.

VxRail Physical and Virtual Network Integration

Each VxRail node will have redundant physical connectivity to ports on one or more Ethernet switches. It
is a best practice that the Ethernet switches be dedicated for VxRail, and that the switch be installed in
close proximity to the VxRail appliance. Ideally, the Ethernet switching should be on the top of the same
rack as the VxRail appliance. For that reason, EMC refers to these switches as ‘top of rack’ Ethernet
switches.

The physical switching infrastructure will enable the flow of traffic for the virtual networks required for
the VxRail appliance, and also provide connectivity to the core network infrastructure for the customer’s
end users and IT administrators. The connections between the VxRail nodes and the ‘top of rack’
Ethernet switches link the virtual distributed switch serving the virtual network traffic on the appliance
with the physical network infrastructure. Network traffic can freely pass between the all of the nodes
that comprise the VxRail appliance when the configurations of the physical and logical switches are in
sync.

VxRail Virtual Networks

The VxRail appliance requires a minimum of 4 logical networks to be configured in order to be operable:

NAME DESCRIPTION ISOLATED?

Management Enables administrative access to VxRail No
Virtual SAN Enables virtual datastore for virtual machine deployment Yes
vMotion Enables virtual machine mobility between VxRail nodes Yes
Guest VM Network Enables network connectivity for guest virtual machines No

 Traffic on an ‘isolated’ network is restricted to the ‘top of rack’ Ethernet switch and does not pass
through uplinks on the ‘top of rack’ switching to the core network.
 Traffic on a network that is not ‘isolated’ is not restricted, and is allowed to pass through the uplinks
on the ‘top of rack’ Ethernet switch, and up into the core network for end user access.
 More than one ‘Guest VM Network’ can be defined, but at least one ‘Guest VM Network’ must be
defined.

VxRail Virtual Network Segmentation

Network segmentation is used to restrict network traffic flow to clearly defined paths in the virtual and
physical network infrastructure. Virtual LANs (VLANs) are the supported method to enforce virtual
network isolation and enable network segmentation on VxRail. Conversely, a ‘flat’ network can also be
deployed, whereby no Virtual LANs are configured. A ‘flat’ network is supported, but it will undermine
the benefits of network segmentation.

To enable network segmentation on VxRail, a unique VLAN ID and VLAN name is assigned to each of the
defined VxRail virtual networks. During the initial configuration of the VxRail appliance, the VLAN ID and
Name properties are configured into to the VxRail virtual networks. During the normal course of
operations of the VxRail appliance, the VLAN ID will be attached, or ‘tagged’, to the packets carrying
traffic on the VxRail virtual networks. These packets will be restricted to paths in the integrated physical
and virtual network where they have been granted access.

When a packet attempts to pass through either a physical or virtual switch port, the VLAN ID is
compared with the list of allowed VLANs for that port. If a match is found, network traffic is allowed to
pass through the port. If a match for the VLAN ID is not found on the port, the network traffic cannot
pass through that port.

If VLANs are defined for the VxRail virtual networks, then the top of rack Ethernet switches must be also
be configured so that the virtual and physical networks are in sync. The example below illustrates the
basic configuration guidelines for both isolated and non-isolated virtual networks:

VLAN Virtual Network Name Allow passage Allow passage Allow passage
ID through all VxRail through uplinks? through ISLs?
node ports?
100 Virtual SAN Yes No Yes
200 Marketing VM Network Yes Yes Yes

All VxRail traffic must be able to pass through every switch port that physically connected to a VxRail
node. If network segmentation is used, the ‘top of rack’ Ethernet switch must be configured to allow this
‘tagged’ traffic to pass through each port connected to a VxRail node. This is true for both’ isolated’ and
‘non-isolated’ virtual networks.

For virtual traffic defined as ‘isolated’, such as the ‘Virtual SAN’ network shown in the example, the top
of rack Ethernet switch must be configured to allow this ‘tagged’ traffic to pass through only the ports
connected to the VxRail nodes. This specific network enables the virtual SAN datastore used by the
virtual machines on the VxRail appliance, and all nodes must be able to access the storage devices on
every node in the appliance. However, the traffic on this specific virtual network does not need pass
through the uplinks and into the core network.

For virtual traffic defined as ‘non-isolated’, such as ‘Marketing VM Network’ shown in the example, the
top of rack Ethernet switch must be configured to allow the ‘tagged’ traffic to pass through the uplinks
and into the core network. This virtual network is targeted to support end user applications running on
virtual machines, and this will enable end user connectivity.
Configuration rules for the ‘top of rack’ Ethernet switches
The configuration rules for the ‘top of rack’ Ethernet switch focus on switch settings and these three
specific items:

 Switch ports connected to the VxRail nodes

 Switch ports configured as ‘uplinks’ to the core network
 Inter-switch links (ISLs) used to create a passageway for network traffic between two or more
switches

Configuration rules for multicasting on the ‘top of rack’ switches

VxRail appliances advertise themselves on the network using the VMware VxRail ‘loudmouth’ service.
This service requires multicasting to be enabled. The ‘top of rack’ switches must be configured to carry
IPv4 multicast and IPv6 multicast traffic to all ports connected to the VxRail nodes.

NOTE: Multicast is not required on the entire network. It is only required on the switch ports connected
to the VxRail nodes.

Configuration rules for Virtual SAN VLAN on the ‘top of rack’ switches
For Virtual SAN VLANs, you need to enable IPv4 IGMP snooping and the snooping querier.

Configuration rules for Management VLAN on the ‘top of rack’ switches

You need to enable IPv4 and IPv6 IGMP snooping and the snooping querier on the Management
network. By default, the Management virtual network is untagged. It is the best practice to configure
the Management virtual network to use the native VLAN on the ‘top of rack’ switch.

Configuration rules for switch port connected to VxRail nodes

 Configure each interface to be a trunk port
 If VLANs are planned for the VxRail virtual networks, ensure that all of the VxRail virtual networks
(Management, vMotion, Virtual SAN and all guest VM networks) are permitted through these ports.
 Do not configure ‘port-channels’ for VxRail node ports. Ports on VxRail nodes do not support LACP,
and do not utilize channel groups.

Configuration rules for switch ports supporting upstream traffic to core network
 Configure uplinks as ‘port channel’ (LAG)
 Set ‘port channel’ to trunk mode
 Add Interfaces that are connected to upstream switches to this ‘port channel’
 Ensure all VLANs that require passage up to the core network are added to the ‘port-channel’

Configuration rules for inter-switch links (ISLs)

 If VLANs are planned for the VxRail virtual networks, ensure that all of the VxRail virtual networks
(Management, vMotion, Virtual SAN and all guest VM networks) are permitted across these links
Top of Rack Switch Configuration Examples
The code samples shown in this section should be used to provide guidance on the command syntax
needed to configure specific functions in a switch. The examples shown are from a Brocade VDX 6740
switch. The code samples shown here for knowledge sharing purposes only, and do not constitute the
full set of configuration steps required to configure a switch for VxRail.

Commands for switch configurations are vendor specific. For detailed configuration syntax, please
reference the applicable vendor documentation. EMC offers the ‘Brocade VDX 6740 Deployment Guide
for VxRail’ for reference purposes: https://www.vmware.com/files/pdf/products/evorail/vmware-
evorail-brocade-vdx6740-deployment-guide.pdf

Example 1: The three required VxRail virtual networks, and one guest network

interface Vlan 10
name management
ip igmp snooping enable
ip igmp snooping querier enable
ipv6 mld snooping enable
ipv6 mld snooping querier enable
!
interface Vlan 20
name vsan
ip igmp snooping enable
ip igmp snooping querier enable
!
interface Vlan 30
name vmotion
!
interface Vlan 100
name GuestNetwork1
!
no vlan dot1q tag native
Figure 1: Sample switch code: Define VxRail VLANs

 IPv4 IGMP snooping, IPv6 MLD snooping, and the snooping queriers are enabled on the
‘management’ network
 IPv4 IGMP snooping and the snooping querier is enabled on the ‘vsan’ network
 Tagging of untagged traffic on the native VLAN is disabled

Example 2: A switch port connected to a 10Gb port on a VxRail node

 interface TenGigabitEthernet 1/0/8
switchport
switchport mode trunk
switchport trunk allowed vlan add 10,20,30,100
switchport trunk tag native-vlan
switchport trunk native-vlan 10
spanning-tree shutdown
no shutdown
!
Figure 2: Sample switch code: VxRail node port

 Port 8 on Slot 0 is configured for 10Gb connectivity

 Port is configured as a trunk port
 Port is configured to allow VLANs for all defined VxRail virtual networks to pass through
 VLAN 10 is for the ‘management’ network, and is configured for the Native VLAN

Example 3: Switch ports configured as uplinks

interface TenGigabitEthernet 1/0/47

channel-group 6142 mode active type standard
no shutdown
!
interface TenGigabitEthernet 1/0/48
channel-group 6142 mode active type standard
no shutdown
!
interface Port-channel 6142
switchport
switchport mode trunk
switchport trunk allowed vlan add 10,100
switchport trunk tag native-vlan
spanning-tree shutdown
no shutdown
!
Figure 3: Sample switch code: Uplinks

 Ports 47 and 48 on Slot 0 are configured as uplink ports

 Ports 47 and 48 are configured to channel group 6142
 Channel group 6142 configures these ports as trunk ports
 Channel group 6142 allows traffic from VLAN 10 (management) and VLAN 100 (GuestNetwork1)
passage upstream to the core network

Example 4: Switch ports configured as inter-switch links

 interface TenGigabitEthernet 1/0/46
fabric isl enable
fabric trunk enable
switchport
switchport mode trunk
switchport trunk allowed vlan add 10,20,30,100
switchport trunk tag native-vlan
spanning-tree shutdown
no shutdown
!

Figure 4: Sample switch code: Inter-Switch Links

 Port 46 is configured as an inter-switch link

 The ISL is configured to allow VLANs for all defined VxRail virtual networks to pass through

VxRail Physical and Logical Networking Topology

 All VxRail nodes except the model 60 require two 10Gb connections to the ‘top of rack’ switch
 The VxRail model 60 nodes requires four 1Gb connections to the ‘top of rack’ switch

Figure 5: VxRail to 10Gb ToR Switch Connectivity

Each ‘top of rack’ switch will need uplink ports to connect to upstream switches in the core network. If
you have two ‘top of rack’ switches, you will also need to interconnect them using inter-switch links
(ISLs).
Figure 6: Dual-switch topology with 2 VxRail appliances