Cisco

EMC ViPR Cisco IVR Cross-Connect

Zoning (VPLEX)
October 9, 2015 Cisco, Vblock CISCO IVR, CROSS-CONNECT, EMC VPLEX, METRO,
VIPR, Zoning
Known ViPR&VPLEX Storage Provisioning Issue:
The following error may be encountered while provision a shared VPLEX distributed volume to
an ESXi Cluster using ViPR v2.x 2.3:

The reason why this issue occurs during a ViPR storage provisioning task with VPLEX is due to
the fact that ViPR incorrectly attempts to apply two simultaneous updates to the Cisco MDS IVR
database, correctly the MDS database is locked by the first task and the second task times out
resulting in a failed ViPR provisioning process. The tasks should be executed in a sequential
fashion allowing each task to complete and then commit changes to the IVR database thus
removing the lock it held once the commit is successful. Once the database lock is removed then
the subsequent task may execute on the database.
Workaround:
Executing an exclusive storage provisioning order from ViPR catalog for a single ESXi host
works perfectly, including automatically creating the required Cross-Connect Zoning, this is due
to the fact the single workflow performs MDS IVR database updates sequentially. During the
single ESXi host exclusive storage provisioning task ViPR creates the necessary initiators,
storage views and IVR Zones (both local and cross-connect zoning) for a single host. BUT
performing a shared storage provisioning task to an ESXi Cluster fails in a single catalog order, it
will also fail if two exclusive storage provision orders are executed at the same time. In summary
the workaround is to execute an exclusive storage provisioning order for each host in the cluster
individually one at a time. Once this is complete and each host has a volume presented and
VPLEX has the correct initiators and storage views created by ViPR, you may then create a new

distributed LUN for the whole ESXi cluster. ViPR simply adds the new distributed volumes to
existing storage views in VPLEX (there is no zoning going on when you run the ddev creation,
thus no locking). Once you have a working distributed volume for all of the hosts, you may then
remove the exclusive volumes and everything should function accordingly. Ensure to verify that
all the required zoning (including IVR Zones) is configured correctly on all switches and the
ESXi hosts can see all associated paths.
NOTE: ViPR engineering plan to enhance the Zoning workflow with an additional step to
obtain/monitor any IVR database locks before proceeding with the IVR zoning operations.
This will be targeted for the next ViPR release. I will provide updates to this post in due
course.
Solution Example:
The below diagram depicts the connectivity requirements in order to implement a ViPR storage
provisioning solution with a VPLEX Metro configuration using Cross-Connect Zoning:

From the above digram you can see that an ISL is in place for Site-to-Site connectivity, in this
example configuration the ISL carries VPLEX-FC-WAN-Replication traffic over
VSAN30(Fabric-A) and VSAN31(Fabric-B) -(VPEX FC WAN COM). VSAN30 is stretched
between Fabric-A switches on both sites and VSAN31 is stretched between both switches on
Fabric-B for Site1&2. VSAN30&31 can be used as transit VSANs for this example IVR
configuration.
In order for ViPR v2.x to successfully execute the task of automatically creating the required
cross-connect zoning the following configuration needs to be in place (as per example diagram
above):
Site1:
Fabric-A, VSAN10: associated interfaces|PC (even ESX hba of site1, VPLEX FE&BE and
PC30) added as members to vsan10.

Fabric-B, VSAN11: associated interfaces|PC (odd ESX hba of site1, VPLEX FE&BE and PC30)
added as members to vsan11.
Site 2:
Fabric-A, VSAN20: associated interfaces|PC (even ESX hba of site2, VPLEX FE&BE and
PC31) added as members to vsan20.
Fabric-B, VSAN21: associated interfaces|PC (odd ESX hba of site2, VPLEX FE&BE and
PC31) added as members to vsan21.
Site1 Site2:
Fabric-A: VSAN30 used as a transit vsan over Port-channel 30.
Fabric-B: VSAN31 used as a transit vsan over Port-channel 31.
A prereq is required in order for ViPR to successfully create the cross-connect zoning
automatically as part of the provisioning workflow, the prereq is to manually create an IVR zone
on fabric A, connecting vsan 10 and vsan 20 and an IVR zone on Fabric B connecting vsan11
and vsan 21 (example IVR Zones provided below).
In the case of ViPR v2.2 an additional prereq task is required and that is to stretch the VSANs
between sites, as per this example VSAN20 gets added to switch-A on Site 1 and vice-versa
VSAN10 added to switch-A on Site2, repeat same for Fabric-B switches but no local interfaces
are assigned to these dummy VSANs, essentially a VSAN20 is created without any member on
Switch-A Site1 etc. This is done for all respective VSANs as can be seen in the example
configuration provided below. As part of the VSAN stretch ensure to add the allowed VSANs to
the respective port-channels:
Port-Channel 30 Allowed VSAN 10,20,30
Port-Channel 31 Allowed VSAN 11,21,31
Once the VSAN is stretched across the sites as per the prereq for ViPR v2.2, ViPR will then
automatically create the required IVR zones as part of the provisioning workflow.
Note: The vArray should be set for Automatic Zoning for all this to occur.
Example MDS Configuration
These are example configuration steps to be completed on both sites MDS switches in order to
enable Cisco Inter-VSAN Routing (IVR is the standard for cross-connect zoning with VPLEX
Metro) and to enable automatic cross-connect zoning with ViPR:
FABRIC A Switches
feature ivr
ivr nat
ivr distribute
ivr commit
system default zone distribute full
system default zone mode enhanced
ivr vsan-topology auto
zone mode enhanced vsan 10
zone mode enhanced vsan 20
zone mode enhanced vsan 30
vsan database
vsan 10 name VSAN10
vsan 20 name VSAN20
vsan 30 name vplex1_wan_repl_vsan30

interface port-channel 30
channel mode active
switchport mode E
switchport trunk allowed vsan 10
switchport trunk allowed vsan add 20
switchport trunk allowed vsan add 30
switchport description CROSS-SITE-LINK
switchport speed 8000
switchport rate-mode dedicated
Configuring FABRIC A switches Fcdoamin priorities:
Site1:
fcdomain priority 2 vsan 10
fcdomain domain 10 static vsan 10
fcdomain priority 100 vsan 20
fcdomain domain 22 static vsan 20
fcdomain priority 2 vsan 30
fcdomain domain 30 static vsan 30
Site2:
fcdomain priority 100 vsan 10
fcdomain domain 12 static vsan 10
fcdomain priority 2 vsan 20
fcdomain domain 20 static vsan 20
fcdomain priority 100 vsan 30
fcdomain domain 32 static vsan 30
Example: configuring Inter-VSAN routing (IVR) Zones connecting an ESXi host HBA0
over VSANs 10 and 20 from site1->site2 and vice versa site2->site1 utilising the transit
VSAN30:
device-alias database
device-alias name VPLEXSITE1-E1_A0_FC02 pwwn 50:00:14:42:A0:xx:xx:02
device-alias name VPLEXSITE1-E1_B0_FC02 pwwn 50:00:14:42:B0:xx:xx:02
device-alias name VPLEXSITE2-E1_A0_FC02 pwwn 50:00:14:42:A0:xx:xx:02
device-alias name VPLEXSITE2-E1_B0_FC02 pwwn 50:00:14:42:B0:xx:xx:02
device-alias name ESXi1SITE1-VHBA0 pwwn xx:xx:xx:xx:xx:xx:xx:xx
device-alias name ESXi1SITE2-VHBA0 pwwn xx:xx:xx:xx:xx:xx:xx:xx
device-alias commit
device-alias distribute
ivr zone name ESXi1SITE1-VHBA0_VPLEXSITE2-E1_A0_FC02
member device-alias ESXi1SITE1-VHBA0 vsan 10
member device-alias VPLEXSITE2-E1_A0_FC02 vsan 20
ivr zone name ESXi1SITE1-VHBA0_VPLEXSITE2-E1_B0_FC02
member device-alias ESXi1SITE1-VHBA0 vsan 10
member device-alias VPLEXSITE2-E1_B0_FC02 vsan 20
ivr zone name ESXi1SITE2-VHBA0_VPLEXSITE1-E1_A0_FC02
member device-alias ESXi1SITE2-VHBA0 vsan 20
member device-alias VPLEXSITE1-E1_A0_FC02 vsan 10

ivr zone name ESXi1SITE2-VHBA0_VPLEXSITE1-E1_B0_FC02

member device-alias ESXi1SITE2-VHBA0 vsan 20
member device-alias VPLEXSITE1-E1_B0_FC02 vsan 10
ivr zoneset name IVR_vplex_hosts_XC_A
member ESXi1SITE1-VHBA0_VPLEXSITE2-E1_A0_FC02
member ESXi1SITE1-VHBA0_VPLEXSITE2-E1_B0_FC02
member ESXi1SITE2-VHBA0_VPLEXSITE1-E1_A0_FC02
member ESXi1SITE2-VHBA0_VPLEXSITE1-E1_B0_FC02
ivr zoneset activate name IVR_vplex_hosts_XC_A
ivr commit
FABRIC B Switches
feature ivr
ivr nat
ivr distribute
ivr commit
system default zone distribute full
system default zone mode enhanced
ivr vsan-topology auto
zone mode enhanced vsan 11
zone mode enhanced vsan 21
zone mode enhanced vsan 31
vsan database
vsan 11 name VSAN11
vsan 21 name VSAN21
vsan 31 name vplex1_wan_repl_vsan31
interface port-channel 31
channel mode active
switchport mode E
switchport trunk allowed vsan 11
switchport trunk allowed vsan add 21
switchport trunk allowed vsan add 31
switchport description CROSS-SITE-LINK
switchport speed 8000
switchport rate-mode dedicated
Configuring FABRIC B switches Fcdoamin priorities:
Site1:
fcdomain priority 2 vsan 11
fcdomain domain 11 static vsan 11
fcdomain priority 100 vsan 21
fcdomain domain 23 static vsan 21
fcdomain priority 2 vsan 31
fcdomain domain 31 static vsan 31
Site2:
fcdomain priority 100 vsan 11
fcdomain domain 13 static vsan 11

fcdomain priority 2 vsan 21

fcdomain domain 21 static vsan 21
fcdomain priority 100 vsan 31
fcdomain domain 33 static vsan 31
Example configuring Inter-VSAN routing (IVR) zones connecting an ESXi host HBA1 over
VSANs 11 and 21 from site1->site2 and vice versa site2->site1 utilising the transit VSAN31:
device-alias database
device-alias name VPLEXSITE1-E1_A0_FC02 pwwn 50:00:14:42:A0:xx:xx:03
device-alias name VPLEXSITE1-E1_B0_FC02 pwwn 50:00:14:42:B0:xx:xx:03
device-alias name VPLEXSITE2-E1_A0_FC02 pwwn 50:00:14:42:A0:xx:xx:03
device-alias name VPLEXSITE2-E1_B0_FC02 pwwn 50:00:14:42:B0:xx:xx:03
device-alias name ESXi1SITE1-VHBA1 pwwn xx:xx:xx:xx:xx:xx:xx:xx
device-alias name ESXi1SITE2-VHBA1 pwwn xx:xx:xx:xx:xx:xx:xx:xx
device-alias commit
device-alias distribute
ivr zone name ESXi1SITE1-VHBA1_VPLEXSITE2-E1_A0_FC03
member device-alias ESXi1SITE1-VHBA1 vsan 11
member device-alias VPLEXSITE2-E1_A0_FC03 vsan 21
ivr zone name ESXi1SITE1-VHBA1_VPLEXSITE2-E1_B0_FC03
member device-alias ESXi1SITE1-VHBA1 vsan 11
member device-alias VPLEXSITE2-E1_B0_FC02 vsan 21
ivr zone name ESXi1SITE2-VHBA1_VPLEXSITE1-E1_A0_FC03
member device-alias ESXi1SITE2-VHBA0 vsan 20
member device-alias VPLEXSITE1-E1_A0_FC02 vsan 10
ivr zone name ESXi1SITE2-VHBA1_VPLEXSITE1-E1_B0_FC03
member device-alias ESXi1SITE2-VHBA1 vsan 21
member device-alias VPLEXSITE1-E1_B0_FC03 vsan 11
ivr zoneset name IVR_vplex_hosts_XC_B
member ESXi1SITE1-VHBA1_VPLEXSITE2-E1_A0_FC03
member ESXi1SITE1-VHBA1_VPLEXSITE2-E1_B0_FC03
member ESXi1SITE2-VHBA1_VPLEXSITE1-E1_A0_FC03
member ESXi1SITE2-VHBA1_VPLEXSITE1-E1_B0_FC03
ivr zoneset activate name IVR_vplex_hosts_XC_B
ivr commit
Verification commands to check status of configuration:
show fcdomain domain-list
Verifies unique domain ID assignment. If a domain overlap exists, edit and verify the alloweddomains list or manually configure static, non-overlapping domains for each participating switch
and VSAN.
show interface brief
Verifies if the ports are operational, VSAN membership, and other configuration settings covered
previously.
show fcns database
Verifies the name server registration for all devices participating in the IVR.

show zoneset active

Displays zones in the active zone set. This should include configured IVR zones.
show ivr fcdomain
Displays the IVR persistent fcdomain database.
show ivr internal
Shows the IVR internal troubleshooting information.
show ivr pending-diff
Shows the IVR pending configuration.
show ivr service-group
Shows the difference between the IVR pending and configured databases.
show ivr tech-support
shows information that is used by your customer support representative to troubleshoot IVR
issues.
show ivr virtual-domains
Shows IVR virtual domains for all local VSANs.
show ivr virtual-fcdomain-add-status
Shows IVR virtual fcdomain status.
show ivr vsan-topology
Verifies the configured IVR topology.
show ivr zoneset
Verifies the IVR zone set configuration.
show ivr zone
Verifies the IVR zone configuration.
clear ivr zone database
Clears all configured IVR zone information.
Note: Clearing a zone set erases only the configured zone database, not the active zone database.
Useful CISCO Docs:
Cisco IVR Troubleshooting
IVR Zones and Zonesets
Inter-VSAN Routing (IVR) definition: An IVR zone is a set of end devices that are allowed to
communicate across VSANs within their interconnected SAN fabric. An IVR path is a set of
switches and Inter-Switch Links (ISLs) through which a frame from an end device in one VSAN
can reach another end device in some other VSAN. Multiple paths can exist between two such
end devices. A Transit VSAN is a VSAN that exists along an IVR path from the source edge VSAN
of that path to the destination edge VSAN of that path, in the example solution diagram below
you will see that VSAN 30 and VSAN 31 are transit VSANs. Distributing the IVR Configuration
Using CFS: The IVR feature uses the Cisco Fabric Services (CFS) infrastructure to enable
efficient configuration management and to provide a single point of configuration for the entire
fabric in the VSAN.
Thanks to @HeagaSteve,Joni,Hans,@dclauvel & Sarav for providing valuable input.
Leave a comment

CISCO MDS Useful Show Commands

September 30, 2015 Cisco CISCO MDS, COMMANDS, show, Useful

CONFIG:
show startup-config
show running-config
show running-config diff
show run |include host|gateway|ntp|fcdomain|http|telnet|zoneset|vsan
show start |include host|gateway|ntp|fcdomain|http|telnet|zoneset|vsan
show install all status
show switchname
show wwn switch
show switch summary
show version
show cdp neighbors
show boot
show system internal flash
show snmp host
show ntp peers
show ssh server
show telnet server
Switch Serial Number:
The switch serial number can be retrieved by using either of the two following commands:
show license host-id
show sprom backplane 1 | grep Serial
LICENSING:
show feature
show license brief
show license usage
show port-license
show port-license | grep -i acquire
HARDWARE:
show hardware
show hardware internal errors all
show environment
show environment temperature
show environment temperature module 1
show environment power
show inventory
show inventory module
show inventory chassis
show inventory xbar
show clock
INTERFACES:
show interface mgmt0
show ip interface
show ip interface mgmt 0
show interface brief

show interface fc1/1 brief

show run interface fc1/1
show interface description
show interface description | grep VMAX
show interface fc1/1 | include description
show interface transceiver
show interface transceiver details
show interface fc1/1 transceiver
show interface fc1/1 transceiver details
show int | include CRC
show int | include errors
Show int fc1/1 | include errors
Show int fc1/1-48 | include errors
Show int fc1/1-32 | include errors
Show int fc2/1-32 | include errors
show interface fc1/1-32,fc2/1-32 | include errors
show interface fc1/1-32,fc2/1-32,fc3/1-32,fc4/1-32 | include errors
show interface fc1/15,fc2/15,fc3/21,fc4/22 | include errors
show int fc1/1,fc1/12 |include fc|vsan|description|CRC|errors
show port-channel summary
show port-channel usage
show port-channel database
show port-channel consistency
show interface port-channel X
show run interface port-channel X
show port internal info
show port internal info interface fc1/1
show port internal info interface port-channel X
show port-resources module 1
show port-resources module 2
VSANs (Virtual SANs):
show vsan
show vsan membership
show vsan membership usage
show vsan X membership
show vsan membership interface fc1/1
show vsan membership interface fc1/1-32
show vsan membership interface port-channel X
show topology vsan X
FLOGI (FABRIC LOGIN):
*Displays devices connected locally over physical ports.
show flogi database
show flogi database details
show flogi database interface fc1/1
show flogi database interface port-channel X
show flogi database vsan X

show flogi database fcid 0x000xxx

show flogi internal info flogi-table vsan X
show flogi internal event-history debugs
show flogi internal vsan X
show flogi internal errors
show flogi internal vsan-state X
FCDOMAIN (Fibre Channel domain):
*Displays global information about fcdomain configurations.
show fcdomain
show fcdomain vsan X
show fcdomain domain-list
show fcdomain domain-list vsan X
show fcdomain address-allocation vsan X
FCNS (Fibre Channel Name Server):
*Displays wwn&fcid including devices connected over IVR/PC/ISL.
show fcns database
show fcns database detail vsan X
show fcns database fcid 0x010440 detail vsan X
show rscn statistics vsan X
show fspf database vsan X
FC & DEVICE ALIAS:
show fcalias
show fcalias vsan X
show fcalias pending vsan X
show device-alias database
ZONING:
show zone (Displays all VSANs)
show zone status
show zone status vsan X (Display zoning mode status enhanced/basic)
show zone active
show zone active vsan X
show zone vsan X
show zone statistics
show zone statistics vsan X
show zone active vsan X
show zone active vsan X | grep ZoneHostName
show zone analysis vsan X
show zone analysis active vsan X
show zone policy vsan X
show zone pending-diff vsan X
show zone member pwwn 21:00:00:XX:XX:XX:XX:XX
show zone name ZoneX
show zoneset
show zoneset vsan X-Y
show zoneset active (An asterix (*) next to the device indicates it is logged into the name server)
show zoneset vsan X-Y

show zoneset active vsan X

show zoneset brief
show zoneset brief vsan X
show zone-attribute-group vsan X
show zoneset pending active vsan X
show zoneset pending vsan X
IVR (Inter VSAN Routing):
show ivr
show ivr vsan-topology
show ivr zone
show ivr zone | grep HostZoneName
show ivr zoneset status
show ivr zoneset active
show ivr internal fcns database
CFS (Cisco Fabric Services):
show cfs merge status
show cfs peers
show cfs status
SUPPORT:
show system uptime
show tech-support brief
show tech-support detail
Show tech-support flogi
Show tech-support fcdomain
show tech-support vsan x
show tech-support zone
show tech-support zone vsan X
show tech-support ivr
show logging server
Show logging log file
show logging last lines
Show ntp peer-status
show system health
show accounting log
show processes cpu
show processes cpu history
show processes cpu module 1
show processes cpu module 2
show processes log
show processes log details
show processes memory
show flash
Leave a comment

Cisco Nexus 3064 Configuring

Jumbo Frames
September 2, 2015 Cisco 3064, Cisco, jumbo, nexus, qos
The default MTU size on a Nexus 3064 switch is 1500 bytes, the following details how to
reconfigure the switch for a system wide MTU value of 9216(jumbo).

Note: The Cisco Nexus 3000 Series switch does not fragment frames. As a result, the switch
cannot have two ports in the same Layer 2 domain with different maximum transmission units
(MTUs). A per-physical Ethernet interface MTU is not supported. Instead, the MTU is set
according to the QoS classes. You modify the MTU by setting Class and Policy maps.
As per Cisco Documentation

Configuring Jumbo Frames:

Begin by creating a policy-map of type network-qos named aptly in this example JumboFrames
and assign as the system-qos. (As stated above you cannot configure on a per interface level).
n3k-sw# config t
n3k-sw(config)# policy-map type network-qos ?
WORD Policy-map name (Max Size 40)
n3k-sw(config)# policy-map type network-qos JumboFrames
n3k-sw(config-pmap-nq)# class type network-qos class-default
n3k-sw(config-pmap-nq-c)# mtu 9216
n3k-sw(config-pmap-nq-c)# system qos
n3k-sw(config-sys-qos)# service-policy type network-qos ?
WORD Policy-map name (Max Size 40)
n3k-sw(config-sys-qos)# service-policy type network-qos class-default
n3k-sw(config-sys-qos)# show policy-map type network-qos
Type network-qos policy-maps
===============================
policy-map type network-qos JumboFrames
class type network-qos class-default
mtu 9216

Verification of the change can be validated by running a show queuing interface command on
one of the 3k interfaces (interface Ethernet 1/2 in this example):
n3k-sw# show queuing interface ethernet 1/2

Ethernet1/2 queuing information:

qos-group sched-type oper-bandwidth
0 WRR 100
qos-group 0
HW MTU: 9216 (9216 configured)
drop-type: drop, xon: 0, xoff: 0
If you run the show interface command then a value of 1500 will be displayed despite the
system wide change that was made:
n3k-sw# show queuing interface ethernet 1/2

Ethernet1/2 is up
Dedicated Interface

Hardware: 100/1000/10000 Ethernet

Description: 6296 2A Mgmt0
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
Leave a comment

CISCO UCS Rebooting

Fabric Interconnect(s)
April 10, 2015 Cisco cisco ucs, Fabric Interconnect, Reboot, State
Begin by connecting to the cluster IP over SSH and checking which FI is Primary/Subordinate:
FI-A# show cluster state
A: UP, PRIMARY
B: UP, SUBORDINATE

Note: show cluster extended-state will provide more detailed information.

Having confirmed the B fabric switch is the subordinate connect to FI-B mgmt cli interface:
FI-A# connect local-mgmt B

From FI-B local-mgmt interface issue the reboot command:

FI-B(local-mgmt)# reboot
Before rebooting, please take a configuration backup.
Do you still want to reboot? (yes/no):yes

Run the cluster state command again to check on the status of FI-B:
FI-A(local-mgmt)# show cluster state
A: UP, PRIMARY
B: DOWN, INAPPLICABLE
HA NOT READY
Peer Fabric Interconnect is down

Once the cluster enters a HA READY status, make FI-B the Primary switch in order to reboot FIA:
FI-A(local-mgmt)# cluster lead b

Note: After initiating a fail over the SSH session will disconnect, re-connect to the cluster and
confirm cluster state.
Connect to local mgmt a and reboot FI-A:
FI-B# connect local-mgmt a
FI-A(local-mgmt)# reboot
Before rebooting, please take a configuration backup.
Do you still want to reboot? (yes/no):yes

Confirm HA READY status before setting FI-A back to PRIMARY:

FI-B# show cluster state
B: UP, PRIMARY
A: UP, SUBORDINATE
HA READY

Set FI-A as PRIMARY, this will need to be set from current PRIMARY FI-B mgmt interface:
FI-B# connect local-mgmt b
FI-B(local-mgmt)# cluster lead a

Leave a comment

Cisco Nexus 5K Unified Port Conversion

March 11, 2015 Cisco Cisco, conversion, nexus, PORT, unified
Nexus 5000 series UP switches have the option of either enabling individual ports as Ethernet
or Fibre Channel ports. One of the strict guidelines is to begin enabling ethernet ports at the first
port 1/1 and in the case of Fiber Channel begin at the last port of the module and work
backwards. If this rule is not followed you will receive the following error(s):
ERROR: Ethernet range starts from first port of the module
ERROR: FC range should end on last port of the module

For this example we will configure Slot-4 on a 5596-UP switch; enabling all 16 ports on the
modules for fibre channel connectivity. Here is the view of the switch prior to converting the
ports to FC:

As you can see the first Slot has 48 unified ports and we have added an additional unified
module to Slot-4. Following the unified port guidelines we would start assigning FC ports at 1/48
for Slot-1 and 4/16 for Slot-4 (As per this example). With a 5596-UP switch the first Slot has 32
unified ports built in, with the option of adding an additional module to Slot-2, again following
the guidelines for a 5548-UP switch the Ethernet ports begin at 1/1, FC at 1/32 and for Slot-2
Ethernet allocation begins at 2/1 and FC at 2/16.
Converting the entire Slot-4 module to FC:
switch# Show interface brief

Eth4/1 1 eth access down SFP validation failed 10G(D)

Eth4/2 1 eth access down SFP validation failed 10G(D)
Eth4/3 1 eth access down SFP not inserted 10G(D)
Eth4/4 1 eth access down SFP not inserted 10G(D)
Eth4/5 1 eth access down SFP not inserted 10G(D)
Eth4/6 1 eth access down SFP not inserted 10G(D)
Eth4/7 1 eth access down SFP not inserted 10G(D)
Eth4/8 1 eth access down SFP not inserted 10G(D)
Eth4/9 1 eth access down SFP not inserted 10G(D)
Eth4/10 1 eth access down SFP not inserted 10G(D)
Eth4/11 1 eth access down SFP not inserted 10G(D)
Eth4/12 1 eth access down SFP not inserted 10G(D)
Eth4/13 1 eth access down SFP not inserted 10G(D)
Eth4/14 1 eth access down SFP not inserted 10G(D)
Eth4/15 1 eth access down SFP validation failed 10G(D)
Eth4/16 1 eth access down SFP validation failed 10G(D)
Enter global config mode:
switch# configure terminal

Jump on Slot-4:
switch(config)# slot 4

Configure all 16 Ports on the unified module as a native Fibre Channel port:
switch(config-slot)# port 1-16 type fc

Save config:
switch(config-slot)# copy running-config startup-config

Important point to note here is that a full reload is not required when converting a port on an
expansion module(GEM), you just need to simply power cycle the GEM card (In the case of
converting a port on the main slot-1 then a full reload of the switch is required.):
switch(config-slot)# poweroff module 4
switch(config-slot)# no poweroff module 4

View the port configuration after power cycle of Slot-4:

switch# show interface brief

fc4/1 1 auto on up swl F 8

fc4/2 1 auto on up swl F 8
fc4/3 1 auto on sfpAbsent
fc4/4 1 auto on sfpAbsent
fc4/5 1 auto on sfpAbsent
fc4/6 1 auto on sfpAbsent
fc4/7 1 auto on sfpAbsent
fc4/8 1 auto on sfpAbsent
fc4/9 1 auto on sfpAbsent
fc4/10 1 auto on sfpAbsent
fc4/11 1 auto on sfpAbsent
fc4/12 1 auto on sfpAbsent
fc4/13 1 auto on sfpAbsent
fc4/14 1 auto on sfpAbsent
fc4/15 1 auto on down swl
fc4/16 1 auto on down swl

Note: after issuing a conversion to FC on the switch you have 120 days to acquire a permanent
license:
switch# show license usage

Feature Ins Lic Status Expiry Date Comments

Count

FCOE_NPV_PKG No Unused
FM_SERVER_PKG No Unused
ENTERPRISE_PKG No In use Grace 119D 20H
FC_FEATURES_PKG No In use Grace 119D 20H

VMFEX_FEATURE_PKG No Unused
ENHANCED_LAYER2_PKG No Unused
LAN_BASE_SERVICES_PKG No Unused
LAN_ENTERPRISE_SERVICES_PKG No Unused

3 Comments

Cisco MDS Upgrading Firmware

October 3, 2014 Cisco 9000, Cisco, code, Firmware, MDS, Upgrade
MDS 9000 Series Firmware Upgrade
It is always good practice to read the software release notes before proceeding with an upgrade.
The Cisco MDS 9000 NX-OS and SAN-OS Software release notes can be found here.
These are the steps I follow when upgrading:
1. Open an SSH session to the switch.
2. Take a backup of your configuration. Firstly ensure that your running configuration has been
applied to the startup-config:
copy running-config startup-config
Then backup your startup configuration. In this example we are backing up to an ftp server:
copy system:startup-config ftp://FTPserver_IP/startup-config.cfg
3. Ensure the switch is in a healthy state before proceeding:

If the MDS is a Director Level switch, then check the redundancy and module status:
show system redundancy status
show module

During the upgrade the standby supervisor is upgraded first. Once the upgrade has completed on
the standby then an automatic switchover occurs and the upgraded standby becomes primary
while the other supervisor is upgraded. The odd time I have experienced with a Director switch
that the supervisor does not switch back to the original primary after the code upgrade has
completed. In this scenario simply use these cmds to switch manually:
attach module #
system switchover (if running on standby)

4. Check the current firmware version:

5. Change directory to bootflash and upload the new bin files to the switch. This example uses an
ftp transfer:
copy ftp://user@IP_Address/m9100-s3ek9-kickstart-mz.5.2.8c.bin m9100-s3ek9-kickstartmz.5.2.8c.bin
copy ftp://user@IP_Address/m9100-s3ek9-mz.5.2.8c.bin m9100-s3ek9-mz.5.2.8c.bin
6. Ensure the code was uploaded to the bootflash directory successfully:

7. Determine if the upgrade will be non-disruptive:

show install all impact system bootflash:///m9100-s3ek9-mz.5.2.8c.bin
For a non-disruptive upgrade the switches are designed in such a way that the ports will remain
connected and traffic flow will continue non-disrupted throughout the upgrade. This is so
because the switch has been engineered in such a way that the control plane is separate to the
data plane. In the case of a director level switch the modules are upgraded and rebooted in a
rolling fashion one at a time non-disruptively(non-disruptively due to the separation of the
control and data planes).
8. Check the MD5 and confirm the files validity:
show version image bootflash:///m9100-s3ek9-mz.5.2.8c.bin
show version image bootflash:///m9100-s3ek9-kickstart-mz.5.2.8c.bin
9. Check for any incompatibilities between the running and the upgrade code versions. This will
check for features that are currently running on the switch but are not supported on the upgrade
code level:
show incompatibility system bootflash:///m9500-sf2ek9-mz.5.2.8c.bin
10. Enter the following command to install the new firmware on the switch:
install all system m9100-s3ek9-mz.5.2.8c.bin kickstart m9100-s3ek9-kickstart-mz.5.2.8c.bin

11. Display the status of the upgrade:

show install all status

12. Run show version to ensure the switch is now running at the upgraded code level. As you
can see from the below example the system version is still on the previous firmware level a
reload is required to apply the upgrade. NOTE: A reload is disruptive to traffic flow (As pointed
out by @dynamoxxx below). This is not common and I have only identified this with the 9148
Switch when upgrading from 5.0.x to 5.8.x.

After reload the system version should be at the upgraded version:

It is good practice to delete the old install files from the bootflash directory:
cd bootflash:
delete m9100-s3ek9-kickstart-mz.5.0.1a.bin

delete m9100-s3ek9-mz.5.0.1a.bin

2 Comments

Cisco MDS Clear All Zoning Configuration

August 13, 2014 Cisco CISCO MDS, clear, DELETE, remove, zoneset, Zoning
The steps below detail the process of clearing all zoning configurations on an MDS switch.
Including: fcaliases, zonesets and zones. If you wish to remove individual zones from a zoneset
then please see post here: How To Remove Zones from an Active Zoneset
Note: Ensure this is a standalone switch; if it is connected to other switches in the fabric then
you may potentially affect the entire fabric.
1. Firstly determine the names of the active zonesets for each vsan. In this example we will clear
down all the zoning associated with the zonset vsan10_zs on vsan 10:
MDS-9148# show zoneset active | inc zoneset
zoneset name vsan10_zs vsan 10
2. Next you will need to deactivate the zoneset vsan10_zs on vsan 10:
MDS-9148# conf t
MDS-9148(config)# no zoneset activate name vsan10_zs vsan 10
Enhanced zone session has been created. Please commit the changes when done.
MDS-9148(config)# zone commit vsan 10
Commit operation initiated. Check zone status
3. Remove the zonset vsan10_zs from the configuration:
MDS-9148(config)# no zoneset name vsan10_zs vsan 10
MDS-9148(config)# zone commit vsan 10
MDS-9148(config)# show zoneset active
Zoneset not present
4. Clear all the zones from the database associated with vsan 10:
MDS-9148# clear zone database vsan 10
Enhanced zone session has been created. Please commit the changes when done.
MDS-9148(config)# zone commit vsan 10
MDS-9148(config)# show zone
Zone not present
MDS-9148# show fcalias
Alias not present
5. Clear any device alias entries:
MDS-9148(config)# clear device-alias database

MDS-9148(config)# device-alias commit

MDS-9148(config)# show device-alias database
6. Exit config mode, save your running config to startup and reload the switch:
MDS-9148(config)# exit
MDS-9148# copy run startup-config
[########################################] 100%
Copy complete, now saving to disk (please wait)
MDS-9148# reload
This command will reboot the system. (y/n)? [n] y
Then apply your new zoning configuration:
copy ftp://FTP-Server-IP/Zoning.cfg system:running-config
EXAMPLE
#### FABRIC A ####
show zoneset active | inc zoneset
conf t
no zoneset activate name vsan10_zs vsan 10
zone commit vsan 10
no zoneset name vsan10_zs vsan 10
zone commit vsan 10
show zoneset active
clear zone database vsan 10
zone commit vsan 10
show zone
show fcalias
clear device-alias database
device-alias commit
show device-alias database
exit
copy run startup-config
#### FABRIC B ####
show zoneset active | inc zoneset
conf t
no zoneset activate name vsan11_zs vsan 11
zone commit vsan 11
no zoneset name vsan10_zs vsan 11
zone commit vsan 11
show zoneset active
clear zone database vsan 11
zone commit vsan 11
show zone
show fcalias
clear device-alias database
device-alias commit
show device-alias database

exit
copy run startup-config
## Then apply your new zoning configuration: ##
copy ftp://10.10.10.1/ZoningA.cfg system:running-config
show zoneset active
copy run start
copy ftp://10.10.10.1/ZoningB.cfg system:running-config
show zoneset active
copy run start
If prompted for vrf the default entry is management:
Enter vrf (If no input, current vrf default is considered): management
1 Comment

CISCO UCS Blades

Memory Troubleshooting
July 11, 2014 Cisco Blade, Cisco, Error, MEMORY, TROUBLESHOOTING
This is a guest blog post kindly contributed by Eric Daly @daly_eric.
In an effort to pin point specific DIMMs within UCS that are throwing an error, please follow
these simple steps. Be aware that an entire memory channel with up to 3 DIMMs may show as
disabled in a channel, all because of one DIMM with uncorrectable errors.
1) Check the Inventory > Memory tab to see which DIMMs are not registering. Make note of the
DIMM Location (F0,F1,F2) as per Below:

2) Review the SEL Log and search for the specific DIMM throwing uncorrectable memory
error. In this case you will see from the image below that the F2 DIMM was causing the issue.
If nothing shows in SEL log perform steps 3-5.

3) Reset CIMC controller of blade (Recover Server > Reset CIMC (Server controller). Wait a
minute or 2.
4) Re-acknowledge blade. Takes 2-3 mins
5) Review SEL Log again as per step 2 in order to identify the faulting DIMM.
DEEPER ANALYSIS
1) Download techsupport for the specific chassis where the suspect blade is located.

2) Extract the tar and then extract the relvant zip file for suspect blade. There are 2 files which
will give you a clear picture of memory DIMM failures. MrcOut.txt and DimmBl.log
3) Locate the DimmBl.log file and open this with Word (not notepad).

4) You will get a summary of first page telling you if blade has any DIMMs with uncorrectable
errors
====================== SUMMARY OF DIMM ERRORS ======================
NO DIMM ECC ERRORS ON THIS BLADE
====================== DIMM BL RAM DATABASE DUMP ========================
====== RAM DB DUMP =====
--- Control Header :
DataBaseFormatVersion : 2
FaultSensorInitDone : 0x00
SyncTaskInitDone : 0x01
DimmBLEnabledBySAM : FALSE
MostRecentHostBootTime : Sat Jun 28 19:07:28 2014
PreviousHostBootTime : Sat Jun 28 02:47:42 2014
MostRecentHostShutdownTime : Sat Jun 28 02:58:12 2014
ErrorSamplingIntervalLength : 1209600
DBSyncPeriod : 3600
CurrentIntervalIndex : 0
---------------------- PER DIMM ERROR COUNTS ----------CORRECTABLE ERRORS UNCORRECTABLE ERRORS
DIMM ID Total This Boot Total This Boot
----------------------------------------------------------A0 0 0 0 0
A1 0 0 0 0
A2 0 0 0 0
B0 0 0 0 0
B1 0 0 0 0
B2 0 0 0 0
C0 0 0 0 0
C1 0 0 0 0
C2 0 0 0 0
D0 0 0 0 0
D1 0 0 0 0
D2 0 0 0 0
E0 0 0 0 0

E1
E2
F0
F1
F2
G0
G1
G2
H0
H1
H2

0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0

1 Comment

Cisco UCS Reinitialize FC Port Channel

June 24, 2014 Cisco cisco ucs, Disable, Enable, Fabric Interconnect, FC Port Channel,
Reinitialize
Fabric Interconnect CLI:
By connecting directly to the fabric interconnect you can quickly issue the following commands
to Reinitialize (Disable&Enable) the specified FC Port Channel. Often this is required when
initally configuring an FC Port Channel with an MDS. The example below will show how to
Reinitialize Port Channel 10 on Fabric A and Port Channel 11 on Fabric B:
Note: for assistance on cmds tail command with a ?
Fabric A
Commands issued in sequence:
scope fc-uplink
scope fabric A
scope port-channel 10
show
disable
commit-buffer
enable
commit-buffer
show
show detail expand
FI-A# scope fc-uplink
FI-A /fc-uplink # scope fabric A
FI-A /fc-uplink/fabric # scope port-channel 10
FI-A /fc-uplink/fabric/port-channel # show
Port Channel:
Port Channel Id Name Oper State Oper Speed (Gbps)
--------------- ---------- ---------------- ----------------10 FC-PC10-Fabric-A
Up 32
FI-A /fc-uplink/fabric/port-channel # disable
FI-A /fc-uplink/fabric/port-channel* # commit-buffer
FI-A /fc-uplink/fabric/port-channel # enable
FI-A /fc-uplink/fabric/port-channel* # commit-buffer
FI-A /fc-uplink/fabric/port-channel # show

Port Channel:
Port Channel Id Name Oper State Oper Speed (Gbps)
--------------- ---------- ---------------- ----------------10 FC-PC10-Fabric-A
Up 32
FI-A /fc-uplink/fabric/port-channel # show detail expand
Port Channel:
Port Channel Id: 10
Name: FC-PC10-Fabric-A
Admin State: Enabled
Oper State: Up
Admin Speed: Auto
Oper Speed (Gbps): 32
Member Port:
Fabric ID: A
Slot Id: 1
Port Id: 29
Membership: Up
Admin State: Enabled
Current Task:
Fabric ID: A
Slot Id: 1
Port Id: 30
Membership: Up
Admin State: Enabled
Current Task:
Fabric ID: A
Slot Id: 1
Port Id: 31
Membership: Up
Admin State: Enabled
Current Task:
Fabric ID: A
Slot Id: 1
Port Id: 32
Membership: Up
Admin State: Enabled
Current Task:

Fabric B
scope fc-uplink
scope fabric B
scope port-channel 11
show
disable
commit-buffer
enable
commit-buffer
show
show detail expand
In order to troubleshoot the FC Uplink ports:
scope fc-uplink
scope fabric A A|B

 show interface this will display all FC Uplink Ports

show interface detail
scope interface 1 29
show interface detail
show fault
Connect Via Nexus OS:
connect nxos a | b
show san-port-channel database
show san-port-channel summary
show san-port-channel internal info all
show interface
show interface brief
show interface fc1/29
show interface transceiver
Check for FLOGI:
show npv flogi-table
UCS Manager:
Click on the SAN tab in UCS Manager, click to expand SAN Cloud Fabric A FC Port
Channels. From here you have the option to Disable|Enable the highlighted FC Port Channel:

1 Comment

Cisco UCS How to Query UCS for WWNs

Associated with Blades
April 3, 2014 Cisco cisco ucs, powertool, wwn
These are very useful tricks in order to obtain a list of WWNs associated with the blades in a
UCS environment. Instead of trawling through the GUI you can easily connect to UCS via the

Cisco UCS PowerTool or SSH to the Fabric Interconnect and quickly retrieve the WWN
information for each/All Host/blades in UCS.
UCS PowerTool
You can download the latest version of Cisco UCS PowerTool from:
UCS PowerTool Download
Firstly launch the UCS PowerTool and Connect to the UCS system by issuing the cmd:
PS C:\> Connect-Ucs
Enter your fabric interconnect IP address hitting the return key and then inputting your creds.
Once you have connected enter the following cmd to bring up a list of all the blades (service
profiles) and their associated vHBA WWNs:
PS C:\> Get-UcsServiceProfile -type instance | Get-UcsVhba | Select Dn,Addr,NodeAddr

If you want to reduce the list to only WWPNs, then use the following:
PS C:\> Get-UcsServiceProfile -type instance | Get-UcsVhba | Select Dn,Addr

Filter by vHBA-0:
Get-UcsServiceProfile -type instance |Get-UcsVhba | select DN,Name,Addr| where {$_.Name
-eq vHBA-0} | sort DN
Filter by vHBA-1:
Get-UcsServiceProfile -type instance |Get-UcsVhba | select DN,Name,Addr| where {$_.Name
-eq vHBA-1} | sort DN

In order to display the associated VSAN details:

PS C:\> Get-UcsServiceProfile -type instance | Get-UcsVhba | Get-UcsVhbaInterface | select
Dn,Initiator,Vnet

Get all service profile instances in UCS:

PS C:\> Get-UcsServiceProfile Type instance
List Boot Policy assigned to each blade:
C:\> Get-UcsServiceProfile -type instance | select Name,BootPolicyName
Fabric Interconnect CLI
Another option is to connect directly to the fabric interconnect. Using the cmd fi01-A# show
server adapter identity will return a detailed list all of the UCS blades WWN details:

For a brief list of all WWNs assigned to each host use this simple cmd:
sh identity wwn
To list an individual server WWPN details:
fiA-31-A# scope chassis 1
fiA-31-A /chassis # scope server 1
fiA-31-A /chassis/server # scope adapter 1
fiA-31-A /chassis/server/adapter # show host-fc-if
FC Interface:
Id Wwn Model Name Operability
- - -
1 20:00:00:25:B5:25:A0:6F UCSB-MLOM-40G-01
vHBA-0 Operable
2 20:00:00:25:B5:25:B1:6F UCSB-MLOM-40G-01
vHBA-1 Operable
From the scope server is is also possible to learn the DN(Distinguished Name):
fiA-31-A /chassis/server # show server adapter vnics
FC Interface:
Adapter Interface Vnic Dn Dynamic WWPN Type
- - -

1 1 org-root/ls-xap-esx001/fc-vHBA-0 20:00:00:25:B5:25:A0:6F Fc
1 2 org-root/ls-xap-esx001/fc-vHBA-1 20:00:00:25:B5:25:B1:6F Fc
Check which vHBA is assigned to fabric A/B:
fiA-31-A# scope service-profile server 1/1
fiA-31-A /org/service-profile # show vhba
vHBA:
Name Fabric ID Dynamic WWPN
-
vHBA-0 A 20:00:00:25:B5:25:A0:6F
vHBA-1 B 20:00:00:25:B5:25:B1:6F
Get a full List of all WWPNs based on Fabric A/B:
fiA-31-A# scope org
fiA-31-A /org # show wwn-pool
WWN Pool:
Name Purpose Size Assigned
- -
Global-WWNN-Pool Node WWN Assignment 128 9
vHBA-0-Fabric-A Port WWN Assignment 128 9
vHBA-1-Fabric-B Port WWN Assignment 128 9
List all Fabric-A WWPN and DN information:
fiA-31-A# scope org
fiA-31-A /org # scope wwn-pool vHBA-0-Fabric-A
fiA-31-A /org/wwn-pool # show initiator
WWN Initiator:
Id Name Assigned Assigned To Dn
-
20:00:00:25:B5:25:A0:6F Yes org-root/ls-xap-esx001/fc-vHBA-0
List all Fabric-B WWPN and DN information:
fiA-31-A# scope org
fiA-31-A /org # scope wwn-pool vHBA-1-Fabric-B
fiA-31-A /org/wwn-pool # show initiator
WWN Initiator:
Id Name Assigned Assigned To Dn
-
20:00:00:25:B5:25:B1:6F Yes org-root/ls-xap-esx001/fc-vHBA-1
Thank you Brendan Lucey for your assistance.
3 Comments

Cisco MDS How To Remove Zones from an

Active Zoneset
March 22, 2014 Cisco ACTIVE ZONESET, CISCO MDS, REMOVE ZONE
1. Firstly we need to know the specific names of the Zones that we intend to delete. To gather the
full list of zone members within a Zoneset run show zoneset vsan xx. The output will return all of

the member names for the Zoneset, the output can be reduced if you know the naming
conventions associated with the hosts; for example if the Zone names begin with V21212Oracle1 then issuing the command show zoneset brief | include V21212Oracle-1 will return in this case
all the Zones associated with Oracle-1:

2. To View the active Zones for Oracle-1 within the Zonseset: show zoneset active | include
V21212Oracle-1

3. Example of Removing half the Zones (Paths) associated with host Oracle-1 from the active
Zoneset name vsan10_zs:
config t
zoneset name vsan10_zs vsan 10
no member V21212Oracle-1_hba1-VMAX40K_9e0
no member V21212Oracle-1_hba1-VMAX40K_11e0
no member V21212Oracle-1_hba2-VMAX40K_7e0
no member V21212Oracle-1_hba2-VMAX40K_5e0
4. Re-activating the Zoneset vsan10_zs after the config changes of removing the specified
Zoneset members:
zoneset activate name vsan10_zs vsan 10
zone commit vsan 10
5. Finally removing the Zones from the configuration:
no zone name V21212Oracle-1_hba1-VMAX40K_9e0 vsan 10
no zone name V21212Oracle-1_hba1-VMAX40K_11e0 vsan 10
no zone name V21212Oracle-1_hba2-VMAX40K_7e0 vsan 10
no zone name V21212Oracle-1_hba2-VMAX40K_5e0 vsan 10
zone commit vsan 10
end
copy run start
Confirm configuration contains the correct Active Zoning:
show zoneset brief | include V21212Oracle-1

show zoneset active | include V21212Oracle-1

3 Comments

CISCO MDS Verifying VNX &

VMAX Connectivity
February 11, 2014 Cisco, VMAX, VNX CISCO MDS, EMC VNX, FCID, FCNS, PORT, VMAX
Somtimes you may have encountered an issue where a VNX/VMAX front-end port gets cabled to
the wrong MDS Switch Port that you had designed for and in fact the port description that was
applied to the MDS Port is incorrect. In this case it is invaluable to have a command to verify
the VNX/VMAX port connected to the MDS. There are many situations where this command
would be useful; for example you may not have the ability to do a physical check as the DC is
remote and you need to ensure the interface description that you are assigning the MDS FC Port
is correct and resulting zoning configurations are accurate. Confirming these connections
remotely through a command on the MDS is very benneficial during these types of situations.
Else you may end up zoning to ports you did not design for.
VMAX Example
So lets take an example: the design and expectation here is to have the VMAX Port 9G:1
connected to MDS FC Interface 2/37 and the interface was given the relevant description:

From this result we can see that the port was labeled as per design as VMAX 9G:1. Now we
need to confirm this is the actual port connected to FC2/37.
To analyse the connectivity of a specific interface we first need to retrieve the FCID for this port:
show interface fc2/37

Now that we know the FCID is 0x010440, we can run our magic Cisco cmd to verify which
VMAX FA Port is actually connected to the MDS Port FC2/37 :
show fcns database fcid 0x010440 detail vsan 10
Note: FCNS = Fibre Channel Name Server.

From the output we can confirm that there is a problem; the expected VMAX Port was 9G1 but
in fact 7G1 is the VMAX port patched to FC2/37 (SYMMETRIX::000195701570::SAF7gB::FC::5876_229). Thus either we update the description of the interface or have the correct
VMAX Port patched.
To modify the description:
interface fc2/37
Switchport description VMAX20K-7g1
no shutdown
VNX Example

Interface FC1/25 as per design is connected to Service Processor A2 front-end port 2:

Running show interface fc1/25 in order to confirm port description and retrieve the FCID:

Now that we know the FCID is 0x010500, we can query the FCNS database for details of what is
connected at the other end of FC1/25:
show fcns database fcid 0x010500 detail vsan 10

From the output we can confirm the correct port is connected from the VNX.
Another method of confirming the correct port is connected, is by gathering the WWPN from the
VNX/VMAX port and then running the show flogi database interface fc 1/25 command on the
MDS:

Reverse Lookup
From the VNX we can run a naviseccli -h SP_IP port -list:

From the output we can see that SPA_6(Logical Port) is connected to the MDS interface WWN
20:19:54:7f:ee:e2:9e:f8.
Given this information we can lookup the Interface Port number by issuing: show fcs database |
include 20:19:54:7f:ee:e2:9e:f8
Thus we can conclude from this output that the VNX Physcial Port SPA:2_2 is connected to MDS
Port FC1/25.
Note: If we want to look up the details of all the switch ports on the MDS this is the command:
show fcns database detail
1 Comment

EMC VMAX 10K Zoning with Cisco

MDS Switches
October 6, 2013 Cisco, Vblock, VMAX 10K, Cisco, EMC, fabric, MDS, Symmetrix, VMAX,
zoneset, Zoning
In this example I will show how to complete the zoning for a two Host ESX Cluster (ESX01 &
ESX02), using a dual Fabric connecting to a EMC VMAX 10K Dual Engine SAN. As can be seen
from the image below this configuration will provide redundancy for the ESX host on both HBAs
as well as on the Switch and VMAX levels.

This example will detail Zoning the ESX Cluster Hosts to front-end ports on a VMAX10K
(2xEngines/4xDirectors) using FA ports 1E0,2E0,3E0,4E0. The Best Practice for the VMAX10K,
if beginning with 2 or more engines, is to assign a cluster across 2 VMAX Engines, one port per
director for a total of 4 ports.
Each VMAX Engine will have connectivity to each SAN Fabric.
Odd directors are connected to Fabric A (1E0,3E0).
Even directors are connected to Fabric B (2E0,4E0).
In this configuration, the zones are created with one HBA and one FA port (2 Zones per HBA):
ESX HBA-0 is zoned to one port on each Engine. Using Director 1 Engine 1 and Director 3
of Engine 2. (Fabric A)
ESX HBA-1 is zoned to one port on each Engine. Using Director 2 Engine 1 and Director 4
of Engine 2. (Fabric B)
A good rule of thumb is to use all the zero ports on directors first before utilizing the one
ports Go wide before you go deep.
MDS-SERIES Zoning Commands
The configuration steps below will detail creating:
Aliass for ESX Hosts

Aliass for VMAX-10K Targets

Creating Zones
Creating Zonesets
FABRIC A SCRIPT
##### Aliass for ESX Hosts (Initiators)#####
fcalias name esx-01_hba0 vsan 10
member pwwn 20:00:00:25:B5:01:A0:01
fcalias name esx-02_hba0 vsan 10
member pwwn 20:00:00:25:B5:01:A0:02
##### Aliass for VMAX-10K (Targets)#####
fcalias name VMAX10K_1e0 vsan 10
member pwwn 50:00:09:75:F0:xx:xx:00
fcalias name VMAX10K_3e0 vsan 10
member pwwn 50:00:09:75:F0:xx:xx:08
##### Create ZONES #####
Single-initiator-single-target is the preferred zoning practice.
Note: EMC always recommends using one initiator and one target in each zone.
ESX-01 HBA-0:
zone name esx-01_hba0-VMAX10K_1E0 vsan 10
member fcalias esx-01_hba0
member fcalias VMAX10K_1E0
zone name esx-01_hba0-VMAX10K_3E0 vsan 10
member fcalias esx-01_hba0
member fcalias VMAX10K_3E0
ESX-02 HBA-0:
zone name esx-02_hba0-VMAX10K_1E0 vsan 10
member fcalias esx-02_hba0
member fcalias VMAX10K_1E0
zone name esx-02_hba0-VMAX10K_3E0 vsan 10
member fcalias esx-02_hba0
member fcalias VMAX10K_3E0
##### ZONESET #####
zoneset name vsan10_zs vsan 10
member esx-01_hba0-VMAX10K_1E0
member esx-01_hba0-VMAX10K_3E0
member esx-02_hba0-VMAX10K_1E0
member esx-02_hba0-VMAX10K_3E0
zoneset activate name vsan10_zs vsan 10
zone commit vsan 10
end
##### CONFIRM NEW ADDITIONS #####
show zoneset brief
show zoneset name vsan10_zs
show zoneset active

show zone active vsan 10 | grep esx-01_hba0

show zone active vsan 10 | grep esx-02_hba0
show zone active vsan 10 | grep 20:00:00:25:B5:01:A0:01
show zone active vsan 10 | grep 20:00:00:25:B5:01:A0:02
FABRIC B SCRIPT
##### Aliass for ESX Hosts (Initiators)#####
fcalias name esx-01_hba1 vsan 11
member pwwn 20:00:00:25:B5:01:B1:01
fcalias name esx-02_hba1 vsan 11
member pwwn 20:00:00:25:B5:01:B1:02
##### Aliass for VMAX-10K (Targets)#####
fcalias name VMAX10K_2E0 vsan 11
member pwwn 50:00:09:75:F0:xx:xx:0C
fcalias name VMAX10K_4E0 vsan 11
member pwwn 50:00:09:75:F0:xx:xx:04
##### Create ZONES #####
zone name esx-01_hba1-VMAX10K_2E0 vsan 11
member fcalias esx-01_hba1
member fcalias VMAX10K_2E0
zone name esx-01_hba1-VMAX10K_4E0 vsan 11
member fcalias esx-01_hba1
member fcalias VMAX10K_4E0
zone name esx-02_hba1-VMAX10K_2E0 vsan 11
member fcalias esx-02_hba1
member fcalias VMAX10K_2E0
zone name esx-02_hba1-VMAX10K_4E0 vsan 11
member fcalias esx-02_hba1
member fcalias VMAX10K_4E0
##### Create ZONESET #####
zoneset name vsan11_zs vsan 11
member esx-01_hba1-VMAX10K_2E0
member esx-01_hba1-VMAX10K_4E0
member esx-02_hba1-VMAX10K_2E0
member esx-02_hba1-VMAX10K_4E0
zoneset activate name vsan11_zs vsan 11
zone commit vsan 11
end
##### CONFIRM NEW ADDITIONS #####
show zoneset brief
show zoneset name vsan11_zs
show zoneset active
show zone active vsan 11 | grep esx-01_hba1
show zone active vsan 11 | grep esx-02_hba1

show zone active vsan 11 | grep 20:00:00:25:B5:01:B1:01

show zone active vsan 11 | grep 20:00:00:25:B5:01:B1:02
By using VMware ESX Servers with multiple physical HBAs and connecting to multiple
directors in different engines this will benefit I/O intensive workloads and will increase
redundancy. Balancing accross resources is always the best approach. This is an example
configuration and it is always advisible to utilize the EMC tools available to help determine your
exact required configuration.
EXAMPLE SINGLE ENGINE DESIGN:
This is the proposed design with the ports cabled as follows for the initial Cluster:
Cluster 1 FABRIC_A 2E0, 1G0
FABRIC_B 1E0, 2G0
Essentially splitting ports from each director across different fabrics as per the following
diagram:

Then Incrementing Port usage per cluster for example:

Cluster 2 FABRIC_A 2F0, 1H0
FABRIC_B 1F0, 2H0

From such a configuration if a switch failure occurs then we lose half the ports on each director
but at least both directors can cater for the workload as opposed to one director.
8 Comments

Cisco MDS Zoning RENAME

June 28, 2013 Cisco Cisco, MDS, Rename, Zoning
In this post, Ill detail some useful commands that can be leveraged to rename Cisco zoning
components:
FCALIAS
ZONE
ZONESET
RENAME ALIAS FOR HOSTS (vsan id of 10):
switch(config)# fcalias rename old-AliasName new-AliasName vsan 10
RENAME ZONES:
switch(config)# zone rename old-ZoneName new-ZoneName vsan 10
RENAME ZONESETS:
switch(config)# zoneset rename old-ZoneSetName new-ZoneSetName vsan 10
Once complete Activate and Commit these changes:
zoneset activate name vsan10_zs vsan 10
zone commit vsan 10
To view the changes made:
switch# show fcalias
Displays alias mappings.
switch# show zone
Displays zone mappings.
switch# show zoneset
Displays all zone sets.
switch# show zoneset active
Displays the active zone set for each VSAN, including
which devices have logged in and are communicating.
switch# show flogi database
Displays the devices that have logged into the fabric.
Retrieve pWWNs for aliases and zoning from this
database.
Example of changing 4 Hosts on both Fabric A&B:
Fabric A
fcalias rename VMwareESX4_v0 VMwareESX5_v0 vsan 10
fcalias rename VMwareESX3_v0 VMwareESX4_v0 vsan 10
fcalias rename VMwareESX2_v0 VMwareESX3_v0 vsan 10
fcalias rename VMwareESX1_v0 VMwareESX2_v0 vsan 10
zone rename VMwareESX4_v0-VNX5400-b4 VMwareESX5_v0-VNX5400-b4 vsan 10
zone rename VMwareESX4_v0-VNX5400-a4 VMwareESX5_v0-VNX5400-a4 vsan 10
zone rename VMwareESX3_v0-VNX5400-b4 VMwareESX4_v0-VNX5400-b4 vsan 10
zone rename VMwareESX3_v0-VNX5400-a4 VMwareESX4_v0-VNX5400-a4 vsan 10

zone rename VMwareESX2_v0-VNX5400-b4 VMwareESX3_v0-VNX5400-b4 vsan 10

zone rename VMwareESX2_v0-VNX5400-a4 VMwareESX3_v0-VNX5400-a4 vsan 10
zone rename VMwareESX1_v0-VNX5400-b4 VMwareESX2_v0-VNX5400-b4 vsan 10
zone rename VMwareESX1_v0-VNX5400-a4 VMwareESX2_v0-VNX5400-a4 vsan 10
zoneset activate name vsan10_zs vsan 10
zone commit vsan 10
Fabric B
fcalias rename VMwareESX4_v1 VMwareESX5_v1 vsan 11
fcalias rename VMwareESX3_v1 VMwareESX4_v1 vsan 11
fcalias rename VMwareESX2_v1 VMwareESX3_v1 vsan 11
fcalias rename VMwareESX1_v1 VMwareESX2_v1 vsan 11
zone rename VMwareESX4_v1-VNX5400-b1 VMwareESX5_v1-VNX5400-b1 vsan 11
zone rename VMwareESX4_v1-VNX5400-a1 VMwareESX5_v1-VNX5400-a1 vsan 11
zone rename VMwareESX3_v1-VNX5400-b1 VMwareESX4_v1-VNX5400-b1 vsan 11
zone rename VMwareESX3_v1-VNX5400-a1 VMwareESX4_v1-VNX5400-a1 vsan 11
zone rename VMwareESX2_v1-VNX5400-b1 VMwareESX3_v1-VNX5400-b1 vsan 11
zone rename VMwareESX2_v1-VNX5400-a1 VMwareESX3_v1-VNX5400-a1 vsan 11
zone rename VMwareESX1_v1-VNX5400-b1 VMwareESX2_v1-VNX5400-b1 vsan 11
zone rename VMwareESX1_v1-VNX5400-a1 VMwareESX2_v1-VNX5400-a1 vsan 11
zoneset activate name vsan11_zs vsan 11
zone commit vsan 11
For additional information about all Cisco SAN-OS commands, refer to the Cisco MDS9000
Family Command Reference.
1 Comment
Search for:

@DavidCRing
Top Posts & Pages

EMC RecoverPoint Architecture and Basic Concepts

EMC VNX - New Shutdown Options

EMC VMAX3 - CLI Cheat Sheet

Categories
Categories

Archives
Archives

David Ring

EMC VNXe 3200 Configuration Steps Via UEMCLI (Part1) October 22, 2015 David
Ring
EMC ViPR Cisco IVR Cross-Connect Zoning (VPLEX) October 9, 2015 David Ring

 EMC UIM/P Editing The Database October 4, 2015 David Ring

Blog at WordPress.com. | The Big Brother Theme.
Follow

Follow David Ring

Get every new post delivered to your Inbox.
Join 1,346 other followers
Build a website with WordPress.com