Sun Blade 6000 Ethernet Switched Software Configuration Guide

Sun Blade 6000 Ethernet Switched
NEM 24p 10GbE

Software Configuration Guide
Part Number E26660-01

October 2010
Copyright 2010, 2011, Oracle and/or its affiliates. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by
intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce,
translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse
engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report
them to us in writing.
If this is software or related software documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S.
Government, the following notice is applicable:
U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government
customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and
agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the
restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government
contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500
Oracle Parkway, Redwood City, CA 94065.
This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for
use in any inherently dangerous applications, including applications which may create a risk of personal injury. If you use this software or
hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to
ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in
dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel and Intel
Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or
registered trademarks of SPARC International, Inc.
This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle
Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products,
and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of
third-party content, products, or services.
Copyright 2010, 2011, Oracle et/ou ses affilis. Tous droits rservs.
Ce logiciel et la documentation qui l'accompagne sont protgs par les lois sur la proprit intellectuelle. Ils sont concds sous licence et soumis
des restrictions d'utilisation et de divulgation. Sauf disposition de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire,
traduire, diffuser, modifier, breveter, transmettre, distribuer, exposer, excuter, publier ou afficher le logiciel, mme partiellement, sous quelque
forme et par quelque procd que ce soit. Par ailleurs, il est interdit de procder toute ingnierie inverse du logiciel, de le dsassembler ou de le
dcompiler, except des fins d'interoprabilit avec des logiciels tiers ou tel que prescrit par la loi.
Les informations fournies dans ce document sont susceptibles de modification sans pravis. Par ailleurs, Oracle Corporation ne garantit pas
qu'elles soient exemptes d'erreurs et vous invite, le cas chant, lui en faire part par crit.
Si ce logiciel, ou la documentation qui l'accompagne, est concd sous licence au Gouvernement des Etats-Unis, ou toute entit qui dlivre la
licence de ce logiciel ou l'utilise pour le compte du Gouvernement des Etats-Unis, la notice suivante s'applique :
U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government
customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and
agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the
restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government
contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500
Oracle Parkway, Redwood City, CA 94065.
Ce logiciel ou matriel a t dvelopp pour un usage gnral dans le cadre d'applications de gestion des informations. Ce logiciel ou matriel
n'est pas conu ni n'est destin tre utilis dans des applications risque, notamment dans des applications pouvant causer des dommages
corporels. Si vous utilisez ce logiciel ou matriel dans le cadre d'applications dangereuses, il est de votre responsabilit de prendre toutes les
mesures de secours, de sauvegarde, de redondance et autres mesures ncessaires son utilisation dans des conditions optimales de scurit. Oracle
Corporation et ses affilis dclinent toute responsabilit quant aux dommages causs par l'utilisation de ce logiciel ou matriel pour ce type
d'applications.
Oracle et Java sont des marques dposes d'Oracle Corporation et/ou de ses affilis. Tout autre nom mentionn peut correspondre des marques
appartenant d'autres propritaires qu'Oracle.
AMD, Opteron, le logo AMD et le logo AMD Opteron sont des marques ou des marques dposes d'Advanced Micro Devices. Intel et Intel Xeon
sont des marques ou des marques dposes d'Intel Corporation. Toutes les marques SPARC sont utilises sous licence et sont des marques ou des
marques dposes de SPARC International, Inc.
Ce logiciel ou matriel et la documentation qui l'accompagne peuvent fournir des informations ou des liens donnant accs des contenus, des
produits et des services manant de tiers. Oracle Corporation et ses affilis dclinent toute responsabilit ou garantie expresse quant aux contenus,
produits ou services manant de tiers. En aucun cas, Oracle Corporation et ses affilis ne sauraient tre tenus pour responsables des pertes subies,
des cots occasionns ou des dommages causs par l'accs des contenus, produits ou services tiers, ou leur utilisation.
Contents
Using This Documentation
Related Documentation
Acronyms and Abbreviations
CLI Command Modes
Feedback
Support and Accessibility
Configuring SEFOS
5
5
6
7
7
8
Connecting to SEFOS
Configuring SEFOS System Features
Default Configurations
Configuring the Default IP Address
Configuring the IP Address for an Interface
Configuring the Restoration File Name
Saving the Current Configurations for Restoration
Erasing a Saved Configuration File
Copying a Configuration File To A Remote Location
Copying a Configuration From A Remote Location to Flash
Copying System Logs To A Remote Location
Copying a File From One Remote Site or Flash to Another Remote Site or Flash
Enabling or Disabling Incremental Save Flag
Enabling or Disabling the Auto Save Flag
Configuring the Default VLAN Identifier
Enabling or Disabling Trap Generation on an Interface
Configuring Debug Logging
Configuring ACL Filters
Configuring Quality of Services
Configuring Port Mirroring
Configuring Rate Limiting
Sample Configurations
Topology Switching Feature
Getting Started
Configuring VLAN Forwarding
Rapid Spanning Tree Protocol
Configuring Link Aggregation
Saving Configurations
Topology - Routing Feature
Configuring Static Unicast Route Entries
Configuring Dynamic Routing - RIP
Configuring Dynamic Routing - OSPF
Topology SFP+ LAG and Backplane Portface Connection
Configuring Basic SFP+ LAG and Backplane
Topology Layer 2 Switch LAG and QSFP Splitter SAN Connection
Configuring a Layer 2 Switch LAG and QSFP Splitter SAN Connection
Topology Switched NEM High Availability Connection
9
10
10
12
13
14
15
17
18
18
18
18
19
19
19
21
22
22
25
28
30
32
33
33
36
38
41
44
46
46
53
55
57
57
61
61
66
3
Setting Up a Four NEM/Two Chassis High Availibility Configuration

66
Topology Four Switched NEMs, High Availability Connection
85
Setting Up a Four NEM/Four Chassis High Availibility Configuration
85
Setting Up QSFP Ports Into a Link Aggregation......................................................................... 86
Checking the Connectivity Status on the NEMs......................................................................... 87
4 Sun Blade 6000 Ethernet Switched NEM 24p 10GbE Software Configuration Guide October 2011
Chapter
1
Using This Documentation
The SEFOS offers a rich set of system features, such as login services, copying and writing facilities, duplex and
negotiation support, and many other features. This document explains how to configure the system features in a
switch running SEFOS. You should have a basic knowledge of the protocol as a prerequisite.
CLI Command Modes
Feedback
Description
Links
All Oracle products
http://oracle.com/documentation
Sun Blade 6000 Ethernet Switched

NEM 24p 10GbE
http://www.oracle.com/pls/topic/lookup?ctx=SB6K-24p-10GbE
Sun Network 10GbE Switch 72p
http://www.oracle.com/pls/topic/lookup?ctx=SN-10GbE-72p
Sun Blade 6000 Modular System
http://www.oracle.com/pls/topic/lookup?ctx=sb6000
Oracle Integrated Lights Out

Manager (Oracle ILOM) 3.0
http://www.oracle.com/pls/topic/lookup?ctx=ilom30

The following acronyms and abbreviations are used in this document:
ACL
Access Control List
BOOTP
Bootstrap Protocol
CLI
Command Line Interface
DHCP
Dynamic Host Configuration Protocol
DSCP
Differentiated Services Code Point
HTTP
Hyper Text Transfer Protocol
ICMP
Internet Control Message Protocol
IP
Internet Protocol
MAC
Media Access Control
MTU
Maximum Transfer Unit
NVRAM
Non-Volatile Random Access Memory
PNAC
Port based Network Access Control
QoS
Quality-of-Service
RADIUS
Remote Authentication Dial-In User Service
RARP
Reverse Address Resolution Protocol
SEFOS
Sun Ethernet Fabric Operating System
SNMP
Simple Network Management Protocol
TCP
Transmission Control Protocol
CLI Command Modes

The following table lists the configuration modes used in this document with their access and exit methods.
Command Mode
Access Method
Prompt
Exit Method
User EXEC
Access SEFOS from the

Oracle ILOM prompt with
read-only rights (privilege
level 1).
SEFOS>
Use the logout or exit

command to return to the
Oracle ILOM prompt.
Privileged EXEC
Access SEFOS from

Oracle ILOM prompt with
full administrative rights
(privilege level 15).
SEFOS#
Use the logout or exit

command to return to the
Oracle ILOM prompt.
Global configuration
SEFOS(config)#
From User EXEC mode,
use the enable command.
Use the end command to

return to Privileged EXEC
mode.
Interface configuration
From Global Configuration SEFOS(config-if)#

mode, use the
interface
interface-type
Use the exit command to

return to Global
Configuration mode, or use
the end command to return
Command Mode
Access Method
Prompt
interface-id
command.
Exit Method
to Privileged EXEC mode.
Feedback
Provide feedback on this documentation at:
http://www.oracle.com/goto/docfeedback

Description
Links
Access electronic support through

My Oracle Support
http://support.oracle.com
For hearing impaired:
http://www.oracle.com/accessibility/support.html
Learn about Oracle's commitment

to accessibility
http://www.oracle.com/us/corporate/accessibility/index.html
Chapter
2
Configuring SEFOS
This chapter describes how to configure SEFOS software.
Connecting to SEFOS
Configuring an Interface to Acquire Dynamic IP
Configuring the Base MAC Address
Configuring the Login Authentication Method
Configuring Authorized Managers for Configuring the Switch
Copying a File From One Remote Site or Flash to Another Remote Site or Flash
Configuring Switch Clock

Enabling/Disabling Console CLI through Serial Port
Enabling/Disabling HTTP
Configuring HTTP Port Number
Configuring Management VLAN List
Configuring MTU Frame Size for an Interface
Configuring Timers for Redundancy Manager
Configuring a Manual Switch Over from Active to Standby
Configuring Sizing Parameters
Connecting to SEFOS
To configure the SEFOS software, you must first establish a connection to your system. The connection must be
through the NEM's serial port or through ssh to the NEM's IP address.
Refer to the Oracle ILOM documentation for instructions on how to establish these connections.
You can connect to the NEM from the CMM. As an added security measure, such a connection is
read-only in SEFOS.
1. To access SEFOS from the ILOM prompt:
-> cd /NEM/fs_cli/
The SEFOS# prompt appears. For example:
-> cd /NEM/fs_cli/
cd: Connecting to Fabric Switch CLI as user root
SEFOS#
You are now connected to SEFOS and you can begin to configure the SEFOS features.
2. When you are finished using SEFOS, type exit to return to the ILOM prompt:
SEFOS# exit
Connection closed by foreign host.
cd: The session with /NEM/fs_cli has ended.

->
3. When you are finished with ILOM, type exit:
exit
Figure 1. Configuration of Basic System Features
Figure 2. Configuration of Advanced System Features
Feature
IP Address and Subnet Mask
Default Setting
10.0.0.1 and 255.0.0.0
IP Allocation Mode
Manual
Dynamic IP Allocation Protocol
DHCP
Feature
Base MAC Address
Default Setting
The switch's base MAC address is derived from the
system's standard unique MAC address, shown on
the yellow sticker. This MAC address should be
sufficient for normal functionality of the switch, so
you do not have to change it.
Login Authentication Mode
Local
Authorized Managers
All users are allowed to configure the switch
Restoration File Name
switch.conf
CLI Console
Enabled
HTTP Status
Disabled
HTTP Port Number
80
Management VLAN
VLAN 1
MTU
9162
Trap Generation on an Interface
Enabled
Checksum on Tunnel Interface
Disabled
MTU Discovery on Tunnel Interface
Disabled
Debug logging
Logs in console
Max number of ISID
131072
Max number of ISID per context
65536
Max number of ports per ISID
128
Max number of ports per ISID_per_context
128
Max number of SVLANs per ISID
4094
Max number of CVLANs per ISID
4094
11

When you configure the default IP address, it will be written to the NVRAM. This default IP address will be used as
the IP address of the default interface when the switch is restarted.
1. Configure the default IP address.
Enter the Global Configuration mode:
SEFOS# configure terminal
Configure the default IP address and subnet mask as 10.0.0.100 and 255.255.0.0, respectively:
SEFOS(config)# default ip address 10.0.0.100 subnet-mask 255.255.0.0
Exit from the Global Configuration mode:
SEFOS(config)#end
2. View the default IP address and subnet mask:
SEFOS# show nvram
Default IP Address
: 10.0.0.1
Default Subnet Mask
: 255.0.0.0
Default IP Address Config Mode
: Manual
Default IP Address Allocation Protocol
: DHCP
Switch Base MAC Address
: 00:14:4f:6c:63:0f
Default Interface Name
: Ex0/1
Default RM Interface Name
: NONE
Config Restore Option
: No restore
Config Save Option
: No save
Auto Save
: Disable
Incremental Save
: Enable
Roll Back
: Enable
Config Save IP Address
: 0.0.0.0
Config Save Filename
: switch.conf
Config Restore Filename
: switch.conf
PIM Mode
: Sparse Mode
IGS Forwarding Mode
: MAC based
Cli Serial Console
: No
SNMP EngineID
: 80.00.08.1c.04.46.53
SNMP Engine Boots
: 16
Default VLAN Identifier
: 1
Stack PortCount
: 0
ColdStandby
: Disable
NPAPI Mode
: Synchronous
Dynamic Port Count
: 24
VLAN 1 (the default VLAN) will have this IP address and subnet mask after the switch is
restarted only if the allocation method is set to manual.

This procedure describes how to configure the IP address that will be used for sending and receiving the packets.
1. Configure an IP address for a VLAN interface.
Enter the Interface Configuration mode:
SEFOS(config)# interface vlan 1
Shut down the VLAN interface:
SEFOS(config-if)# shut down
Configure the IP address and subnet mask:
SEFOS(config-if)# ip address 10.0.0.100
255.0.0.0
Bring up the VLAN interface:

SEFOS(config-if)# no shut down
Exit from the Interface Configuration mode:
SEFOS(config)# end
You must shut down the interface before you can configure the IP address for that interface.
2. View the configured interface IP address:

SEFOS# show ip interface
vlan1 is up, line protocol is up
Internet Address is 10.0.0.100/8
Broadcast Address
10.255.255.255
13

This procedure describes how to write the restoration file to the NVRAM. This file will be used as the configuration
restoration file.
1. Configure the restoration file name.
Configure the configuration restoration file name for the switch:
SEFOS(config)# default restore-file myconfig.conf
SEFOS(config)# end
2. View the default restoration file name:
SEFOS# show nvram
Default IP Address
: 10.0.0.1
Default Subnet Mask
: 255.0.0.0
: Manual
: DHCP
: 00:14:4f:6c:63:0f
: Ex0/1
: NONE
: No restore
Config Save Option
: No save
Auto Save
: Disable
Incremental Save
: Enable
Roll Back
: Enable
: 0.0.0.0
: switch.conf
: myconfig.conf
PIM Mode
: Sparse Mode
IGS Forwarding Mode
: MAC based
Cli Serial Console
: No
SNMP EngineID
: 80.00.08.1c.04.46.53
SNMP Engine Boots
: 16
: 1
Stack PortCount
: 0
ColdStandby
: Disable
NPAPI Mode
: Synchronous
Dynamic Port Count
: 24

This procedure describes how to write the running configuration to a flash file, a startup configuration file or to a
remote site.
1. Save the configuration that is currently running to a file:
SEFOS# write startup-config
2. View the current restoration settings:
SEFOS# show nvram
Default IP Address
: 10.0.0.1
Default Subnet Mask
: 255.0.0.0
: Manual
: DHCP
: 00:14:4f:6c:63:0f
: Ex0/1
: NONE
: Restore
Config Save Option
: Startup save
Auto Save
: Enable
Incremental Save
: Enable
Roll Back
: Enable
: 0.0.0.0
: switch.conf
: myconfig.conf
PIM Mode
: Sparse Mode
IGS Forwarding Mode
: MAC based
Cli Serial Console
: No
SNMP EngineID
: 80.00.08.1c.04.46.53
SNMP Engine Boots
: 16
: 1
Stack PortCount
: 0
ColdStandby
: Disable
NPAPI Mode
: Synchronous
Dynamic Port Count
: 24
15
3. View the restoration status:

SEFOS# show system information
Hardware Version
: 2.5.4_00161614
Firmware Version
: 1.0.2.0
Switch Name
: NEM 24P
System Contact
: Sun Microsystems, Inc.
System Location
Logging Option
: Console Logging
: Local
Config Save Status
: Successful
Remote Save Status
: Not Initiated
Config Restore Status
: Not Initiated
4. Reboot the switch.

Refer to the documentation that came with your switch for those instructions.
5. View the restoration status after rebooting the switch:
SEFOS# show
system information
Hardware Version
: 2.5.4_00161614
Firmware Version
: 1.0.2.0
Switch Name
: NEM 24P
System Contact
System Location
Logging Option
: Console Logging
: Local
Config Save Status
: Not Initiated
Remote Save Status
: Not Initiated
: Successful
The current configurations will be saved to a file that you specified when you ran the
restore-file command in the previous section.
The default name for the restoration file is switch.conf.
The switch will start with the saved configuration when you reboot the switch.

This procedure describes how to clear the contents of the startup configuration or set the parameters in NVRAM to
their default values.
1. Erase the saved configuration file:
SEFOS# erase startup-config
2. View the erase status settings:
SEFOS# show nvram
Default IP Address
: 10.0.0.1
Default Subnet Mask
: 255.0.0.0
: Manual
: DHCP
: 00:14:4f:6c:63:0f
: Ex0/1
: NONE
: No restore
Config Save Option
: No save
Auto Save
: Enable
Incremental Save
: Enable
Roll Back
: Enable
: 0.0.0.0
: switch.conf
: myconfig.conf
PIM Mode
: Sparse Mode
IGS Forwarding Mode
: MAC based
Cli Serial Console
: No
SNMP EngineID
: 80.00.08.1c.04.46.53
SNMP Engine Boots
: 16
: 1
Stack PortCount
: 0
ColdStandby
: Disable
NPAPI Mode
: Synchronous
Dynamic Port Count
: 24
The switch will start with the default configurations when you reboot the switch.
17

This procedure describes how to back up the initial configuration file to flash or to a remote location.
Copy the configuration file - switch.conf - to the location 12.0.0.100.
SEFOS#
copy startup-config tftp://12.0.0.100/switch.conf

This procedure describes how to copy the backup configuration file from a remote location to the current location of
exe for restoration.
Copy the startup configuration file switch.conf stored in the host 12.0.0.100 to the current location of exe:
SEFOS#
copy tftp://12.0.0.100/switch.conf startup-config
This action will copy the file switch.conf from the host 12.0.0.100 to the current exe location. This file will be used
as the startup configuration file when you reboot the switch.

This procedure describes how to upload the log file to a remote location.
Upload the log file to the remote location 12.0.0.100:
SEFOS# copy logs tftp://12.0.0.100/logfile
Copying a File From One Remote Site or Flash to Another Remote Site
or Flash
This procedure describes how to copy a file from one remote site or flash to another remote site or flash.
Copy the file script.txt from the remote location 12.0.0.100 to flash:
SEFOS# copy tftp://12.0.0.100/script.txt
flash:script.txt

Enabling the incremental save flag updates the in-memory database for every configuration at runtime.
Disabling the incremental save flag does not update the in-memory database for any configuration at runtime.
However, you can save the memory database by running the write startup-config command. See the
section Saving the Current Configurations for Restoration for those instructions.
Enable the incremental save flag on the switch.
Enable the incremental save flag on the switch:
SEFOS(config)# incremental-save enable
Disable the incremental save flag on the switch:
SEFOS(config)# incremental-save disable
SEFOS(config)# end

This procedure describes how to enable or disable updates of the runtime configuration in the configuration file.
Enable or disable the auto save flag on the switch.
Enable the auto save flag on the switch:
SEFOS(config)# auto-save trigger enable
Disable the auto save flag on the switch:
SEFOS(config)# auto-save trigger disable
SEFOS(config)# end

This procedure describes how to write the VLAN ID to the NVRAM. This ID will be used as the default VLAN ID
when the switch is restarted.
1. Configure the default VLAN identifier.
19
Configure the default VLAN ID as 10:

SEFOS(config)# default vlan id 10
SEFOS(config)# end
2. View the default VLAN ID:
SEFOS# show nvram
Default IP Address
: 10.0.0.1
Default Subnet Mask
: 255.0.0.0
: Manual
: DHCP
: 00:14:4f:6c:63:0f
: Ex0/1
: NONE
: No restore
Config Save Option
: No save
Auto Save
: Enable
Incremental Save
: Enable
Roll Back
: Enable
: 0.0.0.0
: switch.conf
: myconfig.conf
PIM Mode
: Sparse Mode
IGS Forwarding Mode
: MAC based
Cli Serial Console
: No
SNMP EngineID
: 80.00.08.1c.04.46.53
SNMP Engine Boots
: 16
: 10
Stack PortCount
: 0
ColdStandby
: Disable
NPAPI Mode
: Synchronous
Dynamic Port Count
: 24
You should not change the default VLAN ID when some configurations are already
saved.
After you have configured the default VLAN ID, you must restart the switch before
saving any configuration.

This procedure describes how to enable or disable trap generation either on the physical interface or on the portchannel interface.
1. Disable the SNMP trap on the interface extreme-ethernet 0/1.
Enter the Interface Configuration mode for extreme-ethernet 0/1:
SEFOS(config)# interface extreme-ethernet 0/1
Disable the trap on the interface:
SEFOS(config-if)# no snmp trap link-status
Exit from configuration mode:
SEFOS(config-if)# end
2. View the trap state for the interface extreme-ethernet 0/1:
SEFOS# show interface extreme-ethernet 0/1
Ex0/1 up, line protocol is up (connected)
Bridge Port Type: Customer Bridge Port
Hardware Address is 00:08:02:03:04:01
MTU
9216 bytes, Full duplex, 10 Gbps,
No-Negotiation
HOL Block Prevention enabled.

Input flow-control is on, output flow-control is on
Link Up/Down Trap is disabled
Reception Counters
Octets
: 26635006
Unicast Packets
: 3064
Discarded Packets
: 0
Error Packets
: 0
Unknown Protocol
: 0
Transmission Counters
Octets
: 3725169
Unicast Packets
: 0
Discarded Packets
: 0
Error Packets
: 0
21
If the trap is enabled, then the switch sends trap messages to the SNMP Manager on
specific events such as link up, link down, and so on.

This procedure describes how to configure where the debug logs are to be displayed (on the console or to a file).
1. Modify the logging option of debug traces.
Log the debug traces to a file:
SEFOS(config)# debug-logging file
SEFOS(config)# end
2. View the logging option:
SEFOS# show system information
Hardware Version
: 5.2.3
Firmware Version
: 4.0.0.0
Switch Name
: SEFOS
System Contact
: info@sun.com
System Location
: Sun Microsystems Inc.
Logging Option
: File Logging
: Local
Config Save Status
: Not Initiated
Remote Save Status
: Not Initiated
: Not Initiated

ACL filters are used to filter packets at the hardware based on certain filtering criteria configured or programmed in
the switch. The switch examines each packet to determine if it should be blocked or if it should be forwarded based
on the configured access lists.
The following example shows how to block the IP traffic from a host with an IP address of 12.0.0.100. See Figure 2
for the topology for this procedure. Port 1 of the switch is connected to the host. Run the following commands on
switch 1. The IP address of Host 1 is assumed to be 12.0.0.100 in this procedure.
1. Configure the IP address of the switch to 12.0.0.1:
SEFOS(config-if)# shutdown
255.0.0.0
SEFOS(config-if)# no shutdown
SEFOS(config-if)# exit
2. Create an IP filter with ID 11:
SEFOS(config)# ip access-list extended 11
The filter type can be extended or standard. Standard filters are used to filter the traffic based on the source IP
address and the destination IP address. Extended filters can also be used to specify the protocol ID, TCP/UDP port
numbers, DSCP values, and flow label. In this example, the IP packets with 12.0.0.100 as the source address will be
filtered.
3. Deny the IP traffic from host 12.0.0.100 to any network or host:
SEFOS(config-ext-nacl)# deny ip host 12.0.0.100 any
SEFOS(config-ext-nacl)# exit
4. Apply the IP filter 11 to port1:
SEFOS(config-if)# ip access-group 11 in
5. View the configuration details:
SEFOS# show access-lists
IP ACCESS LISTS
----------------Extended IP Access List 11
-----------------------------
23
Filter Priority
: 1
Filter Protocol Type
: ANY
IP address Type
: IPV4
Source IP address
: 12.0.0.100
Source IP address mask
: 255.255.255.255
Source IP Prefix Length
: 32
Destination IP address
: 0.0.0.0
Destination IP address mask
: 0.0.0.0
Destination IP Prefix Length
: 0
Flow Identifier
: 0
In Port List
: Ex0/1
Out Port List
: NIL
Filter TOS
: NIL
Filter DSCP
: NIL
Filter Action
: Deny
Status
: Active
6. Send the forwarding traffic from a smart bit machine to switch Port 1 with a source IPv4 address of
12.0.0.100.
The packet will not be forwarded to Port 2.
7. Remove the IP filter from port 1:
SEFOS(config-if)# no ip access-group 11 in
IP ACCESS LISTS
----------------------------Filter Priority
: 1
: ANY
IP address Type
: IPV4
Source IP address
: 12.0.0.1
: 255.255.255.255
: 32
: 0.0.0.0
: 0.0.0.0
: 0
Flow Identifier
: 0
In Port List
: NIL
Out Port List
: NIL
Filter TOS
: NIL
Filter DSCP
: NIL
Filter Action
: Deny
Status
: InActive
8. Send the forwarding traffic from a smart bit machine to switch Port 1 with a source IPv4 address of
12.0.0.100.
The packets will be forwarded to Port 2.

Packets received at the switch can be classified to a particular class of service based on the filters configured.
Certain policies can be applied to them before forwarding the packet. The following example illustrates the
classification of the TCP packets received in the switch and changing the DSCP value in the IP header of the TCP
packets to 46.
See Figure 2 for the topology for this procedure. Connect Port 1 and Port 2 to the IXIA/SMART BITS.
1. Enable Port 1 and Port 2:
2. Enable QoS:
SEFOS(config)# qos enable
SEFOS(config)# end
3. Create an access control list filter for TCP packets:
25
SEFOS(config)# ip access-list extended 11

SEFOS(config-ext-nacl)# permit tcp any any
SEFOS(config-ext-nacl)# exit
SEFOS(config-if)# ip access-group 11 in
SEFOS(config-if)#
end
4. Specify the class mapping for the incoming packet and policy mapping for the classified packet.
Create a class map and enter into the Class Map Configuration mode:
SEFOS(config)# class-map 10
Configure the match criteria for the class map with the criteria specified by the access list 11 (already
configured earlier as all tcp packets):
SEFOS(config-cls-map)# match access-group ip-access-list 11
SEFOS(config-cls-map)# set class 100
Exit from the Class Map Configuration mode:
SEFOS(config-cls-map)# exit
Create a policy map to apply on the packet before forwarding it:
SEFOS(config)# policy-map 10
Create a policy map for class 100 packets and set the DSCP value in the IP header for the outgoing
packets matched by this class as 46:
SEFOS(config-ply-map)# set policy class 100 default-priority-type ipdscp 46
SEFOS(config-ply-map)# exit
IP ACCESS LISTS
----------------------------Filter Priority
: 1
: TCP
IP address Type
: IPV4
Source IP address
: 0.0.0.0
: 0.0.0.0
: 0
: 0.0.0.0
: 0.0.0.0
: 0
Flow Identifier
: 0
In Port List
: Ex0/1
Out Port List
: NIL
Filter TOS
: NIL
Filter DSCP
: NIL
Filter Source Ports From
: 1
Filter Source Ports Till
: 65535
Filter Destination Ports From
: 1
Filter Destination Ports Till
: 65535
Filter Action
: Permit
Status
: Active
MAC ACCESS LISTS

----------------No MAC Access Lists have been configured
SEFOS# show class-map 10
QoS Class Map Entries
--------------------ClassMapId
: 10
L2FilterId
: None
L3FilterId
: 11
PriorityMapId
: None
CLASS
: 100
PolicyMapId
: 10
PreColor
: None
Status
: Active
SEFOS# show policy-map 10

QoS Policy Map Entries
---------------------PolicyMapId
: 10
IfIndex
: 0
Class
: 100
DefaultPHB
: IP DSCP 46
27
MeterId
: 0
ConNClass
: 0
ExcNClass
: 0
VioNClass
: 0
ConfAct
: None.
ExcAct
: None.
VioAct
: None.
6. Verify the functionality of the policy configuration by generating 100 TCP packets from IXIA to Port 1.
You will notice that these packets are received at Port 2 and have a DSCP value 46.

Port mirroring is used to monitor the packets of a particular port on another port. The following example shows how
to mirror all incoming packets on Port 1 to Port 2.
See Figure 1 for the topology for this procedure. Port 1 of the switch is connected to the host and Port 2 is connected
to a packet-capturing device to verify that packet is mirrored at Port 2. Run the following commands on switch 1.
The IP address of Host 1 is assumed to be 12.0.0.100 in this procedure.
1. Configure the IP address of the switch as 12.0.0.1:
255.0.0.0
2. View the ping between the host and the switch:
SEFOS# ping 12.0.0.100
Reply Received From :12.0.0.100, TimeTaken : 30 msecs
--- 12.0.0.100 Ping Statistics --3 Packets Transmitted, 3 Packets Received, 0% Packets Loss
The ping reply from Host 1 to the switch is not captured at Port 2, which is connected to the packet capturing device
prior to the port-mirroring configuration.
3. Run the following commands on the switch to enable mirroring for incoming packets at Port 1 to Port 2.
Configure the destination interface for mirroring as Port 2:
SEFOS(config)# monitor session destination interface extreme-ethernet 0/2
Configure the source interface for mirroring as Port 1 and apply mirroring for incoming packets:
SEFOS(config)# monitor session source interface extreme-ethernet 0/1 rx
View the configuration details:
SEFOS# show port-monitoring
Port Monitoring
is enabled
Monitor Port : Ex0/2

Port
Ingress-Monitoring
Egress-Monitoring
----
------------------
-----------------
Ex0/1
Enabled
Disabled
Ex0/2
Disabled
Disabled
Ex0/3
Disabled
Disabled
Ex0/4
Disabled
Disabled
Ex0/5
Disabled
Disabled
Ex0/6
Disabled
Disabled
Ex0/7
Disabled
Disabled
Ex0/8
Disabled
Disabled
Ex0/9
Disabled
Disabled
Ex0/10
Disabled
Disabled
Ex0/11
Disabled
Disabled
Ex0/12
Disabled
Disabled
Ex0/13
Disabled
Disabled
Ex0/14
Disabled
Disabled
Ex0/15
Disabled
Disabled
Ex0/16
Disabled
Disabled
Ex0/17
Disabled
Disabled
Ex0/18
Disabled
Disabled
29
Ex0/19
Disabled
Disabled
Ex0/20
Disabled
Disabled
Ex0/21
Disabled
Disabled
Ex0/22
Disabled
Disabled
Ex0/23
Disabled
Disabled
Ex0/24
Disabled
Disabled
4. Ping 12.0.0.100 and verify that the ICMP reply was received at Port 2:
5. Verify that the ping reply from Host 1 to the switch is captured at Port 2 using the packet-capturing
device.

SEFOS can be configured to limit the rate of traffic received on a particular interface. If the traffic is above the
configured threshold level, then the packet will get dropped. The following example illustrates the configuration for
limiting a multicast traffic at Port 1 to a rate of 50 packets per second.
See Figure 2 for the topology for this procedure. Port 1 and Port 2 are connected to SMART BITS/IXIA to monitor
the rate of packet forwarding. Configure the rate limiting for multicast packets on Port 1 as 50 packets per second
and then generate a multicast traffic from IXIA at the rate of 1 Mbps. Packets received at Port 2 must be at the rate
of 50 packets per second.
1. Run the following commands in the switch to enable Port 1 and Port 2:
2. Set the rate limit to 50 packets per second.

Enter the Interface Configuration mode:
Set the rate limit to 50 packets per second:
SEFOS(config-if)# storm-control multicast level 50
Exit from Global Configuration mode:
SEFOS# show interface extreme-ethernet 0/1 storm-control
Ex0/1
DLF Storm Control
: Disabled
Broadcast Storm Control
: Disabled
Multicast Storm Control
: Enabled
Multicast Storm Control
: 50
4. View the functionality by generating multicast packet from IXIA at the rate of 1 Mbps to Port 1.
You will notice that the packets received at the second port of IXIA are at the rate of 50 packets per second only.
31
Chapter
3
Sample Configurations
This chapter provides sample configurations for the Layer 2 switching features used in the Sun Blade 6000 Ethernet
Switched NEM 24p 10GbE from Oracle using the Sun Ethernet Fabric Operating System (SEFOS).
Getting Started
Setting Up QSFP Ports Into a Link Aggregation

Checking the Connectivity Status on the NEMs

The following figure shows a sample topology for the switching feature.
Figure 3. Sample Topology For the Switching Feature
Getting Started
1. Connect to SEFOS.
If you are not already connected to SEFOS, see the instructions in Chapter 1, then return here. You should see the
SEFOS prompt when you are connected to SEFOS:
SEFOS#
2. View the status of the switch interfaces:
SEFOS# show interface status
Port
Status
Duplex
Speed
Negotiation
----
------
------
-----
-----------
Ex0/1
connected
Full
10 Gbps
No-Negotiation
Ex0/2
not connected
Full
10 Gbps
No-Negotiation
Ex0/3
not connected
Full
10 Gbps
No-Negotiation
Ex0/4
not connected
Full
10 Gbps
No-Negotiation
33
Ex0/5
not connected
Full
10 Gbps
No-Negotiation
Ex0/6
not connected
Full
10 Gbps
No-Negotiation
Ex0/7
not connected
Full
10 Gbps
No-Negotiation
Ex0/8
not connected
Full
10 Gbps
No-Negotiation
Ex0/9
not connected
Full
10 Gbps
No-Negotiation
Ex0/10
not connected
Full
10 Gbps
No-Negotiation
Ex0/11
not connected
Full
10 Gbps
No-Negotiation
Ex0/12
not connected
Full
10 Gbps
No-Negotiation
Ex0/13
not connected
Full
10 Gbps
No-Negotiation
Ex0/14
not connected
Full
10 Gbps
No-Negotiation
Ex0/15
connected
Full
10 Gbps
No-Negotiation
Ex0/16
connected
Full
10 Gbps
No-Negotiation
Ex0/17
not connected
Full
10 Gbps
No-Negotiation
Ex0/18
not connected
Full
10 Gbps
No-Negotiation
Ex0/19
not connected
Full
10 Gbps
No-Negotiation
Ex0/20
not connected
Full
10 Gbps
No-Negotiation
Ex0/21
not connected
Full
10 Gbps
No-Negotiation
Ex0/22
not connected
Full
10 Gbps
No-Negotiation
Ex0/23
not connected
Full
10 Gbps
No-Negotiation
Ex0/24
connected
Full
10 Gbps
No-Negotiation
3. View the default IP address:

Broadcast Address
10.255.255.255
When the system is booted up, it will come up with a default IP address of 10.0.0.1 for the default VLAN 1.
4. Enter the Global Configuration mode:
SEFOS(config)#
5. Change the default IP address of VLAN 1:
SEFOS(config-if)# ip address 20.0.0.1 255.0.0.0
6. View the default IP address of VLAN 1:
Broadcast Address
20.255.255.255
SEFOS#
7. Ping Switch 1 from Blade Server A.
The ping should be successful.
8. Check the MAC address entry:
SEFOS# show mac-address-table
Vlan
Mac Address
Type
Ports
----
-----------
----
-----
00:1b:21:4a:ab:81
Learnt
Ex0/15
Total Mac Addresses displayed: 1
The MAC address shown should correspond to the MAC address of Blade Server A.
9. Repeat Step 5 on Switch 2 to change the default IP address of VLAN 1 IP to 20.0.0.2.
10. Ping from Blade Server C to Blade Server A or Blade Server B.
The ping should be successful.
11. Configure the topology.
To configure the topology, all the interfaces should be up.
In Switch 1, make the interface extreme-ethernet 0/4, 0/5, 0/6, 0/15, and 0/16 up as shown above. In Switch 2, make
the interface extreme-ethernet 0/3, 0/4, 0/5, 0/6, 0/15, and 0/16 up as shown above.
Following are the expected results from the configurations made to load the Sun Ethernet Fabric Operating
System on to the Sun Blade 6000 Ethernet Switched NEM 24p 10GbE target.
1. Ping from Blade Server A and Blade Server B should be successful to switch 1 (20.0.0.1).
2. Ping from Blade Server C should be successful to switch 2 (20.0.0.2).
35
3. Use a capture packet utility to capture packet examples on Blade Server A, Blade Server B and Blade Server
C for the port connected to the switch. All of these hosts should receive STP, GMRP, GVRP packets continuously.
By default, all the ports on a target will be members of default VLAN 1.

This procedure describes how to configure and test VLAN forwarding feature of SEFOS running on a Sun Blade
6000 Ethernet Switched NEM 24p 10GbE target. This procedure tests whether the incoming packets are properly
switched on the member ports of the VLAN.
See Figure 3 for the topology for this procedure.
1. Follow the instructions in the Getting Started section, if you have not already done so.
2. Disable GVRP.
VLAN 1 is the default VLAN. All the ports will be members of the default VLAN.
3. Verify the VLAN membership:
SEFOS# show vlan
Vlan database
------------Vlan ID
: 1
Member Ports
: Ex0/1, Ex0/2, Ex0/3, Ex0/4, Ex0/5, Ex0/6

Ex0/7, Ex0/8, ...
Untagged Ports
: Ex0/1, Ex0/2, Ex0/3, Ex0/4, Ex0/5, Ex0/6

Ex0/7, Ex0/8, ...
Forbidden Ports
: None
Name
Status
: Permanent
4. Ping from Blade Server A to Blade Server B:

# ping 20.0.0.20
PING 20.0.0.20 (20.0.0.20) 56(84) bytes of data.
64 bytes from 20.0.0.20: icmp_seq=1 ttl=255 time=0.384 ms
--- 20.0.0.20 ping statistics --5 packets transmitted, 5 received, 0% packet loss, time 4002ms
rtt min/avg/max/mdev = 0.224/0.294/0.384/0.056 ms
5. Ping from Blade Server B to Blade Server A:

# ping 20.0.0.10
6. Change the member ports of VLAN 1 on Switch 2:

7. Ping from Blade Server C to Blade Server B:
# ping 20.0.0.20
37

This procedure shows the traffic flow in the Rapid Spanning Tree protocol configuration on SEFOS running on the
Sun Blade 6000 Ethernet Switched NEM 24p 10GbE targets. This procedure verifies whether a loop is present in the
topology, if it is detected and if the traffic is blocked for the alternate port.
2. Configure VLAN forwarding in Switch 1 and Switch 2 using the instructions in the section Configuring
VLAN Forwarding.
3. Run the following command on Switch 1 to verify the port status in RSTP mode:
SEFOS# show spanning-tree
Root Id
Priority
32768
Address
00:14:4f:6c:63:0f
Cost
Port
0 [0]
This bridge is the root

Max age 20 Sec, forward delay 15 Sec
MST00
Spanning tree Protocol has been enabled
MST00 is executing the rstp compatible Multiple Spanning Tree Protocol
Bridge Id
Priority
Address
32768
00:14:4f:6c:63:0f
Max age is 20 sec, forward delay is 15 sec

Dynamic Path Cost is Disabled
Name
Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/3
Designated
Forwarding
2000
128
P2P
Ex0/4
Designated
Forwarding
2000
128
P2P
Ex0/5
Designated
Forwarding
2000
128
P2P
Ex0/6
Designated
Forwarding
2000
128
P2P
Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
4. Run the following command on Switch 2 to verify the port status in RSTP mode:
Root Id
Priority
32768
Address
00:14:4f:6c:63:0f
Cost
2000
Port
3 [Ex0/3]

MST00
Bridge Id
Priority
Address
32768
00:15:ed:09:09:10

Name
Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/3
Root
Forwarding
2000
128
P2P
Ex0/4
Alternate
Discarding
2000
128
P2P
Ex0/5
Alternate
Discarding
2000
128
P2P
Ex0/6
Alternate
Discarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
5. Ping from Blade Server A to Blade Server C:

# ping 20.0.0.30
Traffic is forwarded through the extreme-ethernet 0/3 port on Switch 1.

6. Shut down the extreme-ethernet 0/3 port from Switch 2 and verify the port status:
39

SEFOS(config)# int ext 0/3; shut; exit
SEFOS(config)# exit
Root Id
Priority
32768
Address
00:14:4f:6c:63:0f
Cost
2000
Port
4 [Ex0/4]

MST00
Bridge Id
Priority
Address
32768
00:15:ed:09:09:10

Name
Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/4
Root
Forwarding
2000
128
P2P
Ex0/5
Alternate
Discarding
2000
128
P2P
Ex0/6
Alternate
Discarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P

# ping 20.0.0.30
Traffic should be forwarded through the extreme-ethernet 0/4 port on Switch 1. If the Rapid Spanning Tree protocol
is working as expected, there will be a change in the port state. This procedure shows that if the root port in the
RSTP mode goes down for any reason, an alternate port in the RSTP will forward the traffic.

This procedure describes how to configure and test the Link Aggregation feature of SEFOS running on a Sun Blade
6000 Ethernet Switched NEM 24p 10GbE target.
2. Create a port-channel group 100 on Switch 1 and link the ports in the group:
SEFOS(config)# set port-channel enable
SEFOS(config)# interface port-channel 100
SEFOS(config-if)# channel-group 100 mode active
3. Repeat the previous step on Switch 2.
4. Verify the VLAN membership:
SEFOS# show vlan
Vlan ID
: 1
Member Ports
: Ex0/1, Ex0/2, Ex0/7, Ex0/8, Ex0/9, Ex0/10

Ex0/11, Ex0/12, Ex0/13, Ex0/14, Ex0/15, Ex0/16
Ex0/17, Ex0/18, Ex0/19, Ex0/20, Ex0/21, Ex0/22
41
Ex0/23, Ex0/24, po100

Untagged Ports
: Ex0/1, Ex0/2, Ex0/7, Ex0/8, Ex0/9, Ex0/10

Ex0/11, Ex0/12, Ex0/13, Ex0/14, Ex0/15, Ex0/16
Ex0/17, Ex0/18, Ex0/19, Ex0/20, Ex0/21, Ex0/22
Ex0/23, Ex0/24, po100
Forbidden Ports
: None
Name
Status
: Permanent
5. Verify the STP port status on Switch 1:

Root Id
Priority
32768
Address
00:14:4f:6c:63:0f
Cost
Port
0 [0]

MST00
Bridge Id
Priority
Address
32768
00:14:4f:6c:63:0f

Name
Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
po100
Designated
Discarding
1900
128
P2P
6. Ping continuously from Blade Server A to Blade Server C.

There should not be any data loss during traffic forwarding.
7. Verify the port channel summary:
SEFOS# show etherchannel summary
Port-channel Module Admin Status is enabled
Port-channel Module Oper Status is enabled

Port-channel System Identifier is
00:14:4f:6c:63:0f
Flags:
D - down
P - in port-channel
I - stand-alone
H - Hot-standby (LACP only)
Number of channel-groups in use: 1

Number of aggregators: 1
Group
Port-channel
Protocol
Ports
------------------------------------------------------------------------100
Po100(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
8. Shut down the port extreme-ethernet 0/3 and verify the port channel summary:
SEFOS(config)# int ext 0/3
SEFOS(config-if)# shut
00:14:4f:6c:63:0f
Flags:
D - down
P - in port-channel
I - stand-alone

Group
Port-channel
Protocol
Ports
------------------------------------------------------------------------100
Po100(P)
LACP
Ex0/3(D),Ex0/4(P),Ex0/5(P),Ex0/6(P)
If traffic flows continuously through the port channel group, then Link Aggregation was configured properly.
43
This procedure describes how to save configurations even after restarting the switch.
2. Save the configurations for Switch 1 and Switch 2:
This may take a few seconds. You will see the message [OK] when the configurations have been successfully
saved.
3. Exit the SEFOS session:
SEFOS# exit
Connection closed by foreign host.
cd: The session with /NEM/fs_cli has ended.
4. At the ILOM prompt, change directories to the /SP directory:

-> cd /SP
5. Reset the service processor:
-> reset
Are you sure you want to reset /SP (y/n)? y
Performing reset on /SP
Broadcast message from root (Wed Feb 11 23:13:07 1970):
The system is going down for reboot NOW!
6. Log in when the system comes back online.

The default password is changeme.
[root]# ssh my-nem -l root
root@my-nem's password:
Sun(TM) Integrated Lights Out Manager (Fabric Component Edition)
Version 3.0.0.0
Copyright 2009 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Warning: password is set to factory default.
7. Connect to the Sun Ethernet Fabric Operating System again:

-> cd /NEM/fs_cli
cd: Connecting to Fabric Switch CLI as user fs_cli_usr
8. Verify the port channel summary:

00:14:4f:6c:63:0f
Flags:
D - down
P - in port-channel
I - stand-alone

Group
Port-channel
Protocol
Ports
------------------------------------------------------------------------100
Po100(P)
LACP
Ex0/3(D),Ex0/4(P),Ex0/5(P),Ex0/6(P)
Once Switch 1 and Switch 2 are both restarted, the Sun Ethernet Fabric Operating System will come up with all the
previous configurations.
45

The following figure shows a sample topology for the routing feature.
Figure 4. Sample Topology For the Routing Feature

This procedure describes how to configure the static route entries in SEFOS running on switched NEMs. This
procedure will verify that the traffic between hosts, which are on two different networks, is reachable by configuring
the required static route in SEFOS.
2. Configure Switch 1 based on the topology shown in Figure 4:
SEFOS(config)# set gvrp disable
SEFOS(config)# vlan 10
SEFOS(config-vlan)# exit
SEFOS(config)# interface range extreme-ethernet 0/1-2
SEFOS(config-if-range)# switchport access vlan 10
SEFOS(config-if-range)# no shutdown
SEFOS(config-if-range)# exit

3. Ping from Blade Server C to Blade Server A:
Reply
Reply
Reply
Received
Received
Received
From
From
From
:10.0.0.169,
:10.0.0.169,
:10.0.0.169,
TimeTaken
TimeTaken
TimeTaken
:
:
:
20
20
20
--10.0.0.169
Ping
Statistics
3 Packets Transmitted, 3 Packets Received, 0% Packets Loss
msecs
msecs
msecs
---

Reply
Received
From
:10.0.0.152,
TimeTaken
Reply
Received
From
:10.0.0.152,
TimeTaken
:
:
20
20
msecs
msecs
5. Configure Switch 2 based on the topology shown in Figure 4:

SEFOS(config)# set gvrp disable
SEFOS(config-if)# no ip address
SEFOS(config)# vlan 40
SEFOS(config-vlan)# exit
SEFOS(config-if-range)# shutdown
47

6. Create a 10GbE link between the two switches. Use one of the P3 10GbE links from the first QSFP ports
as the interconnect. Also, create and configure the VLAN interfaces on the connection link.
Run the following commands on Switch 1:
You should see the following message:
% Access VLAN does not exist. Creating vlan
SEFOS#
Run the following commands on Switch 2:
You should see the following message:
% Access VLAN does not exist. Creating vlan
SEFOS#
You can now use this configuration as a starting point for different router configurations.
7. Save the configuration information so that the switch will start with the saved configuration on reboot.
Run the following command on Switch 1:

Building configuration
[OK]

Building configuration
[OK]
8. Verify that the existing routes are available in Switch 1.

SEFOS# show ip route
Codes: C - connected, S - static, R - rip, B - bgp, O - ospf
C 10.0.0.0/24 is directly connected, vlan10

If there is no known route from Switch 1 to Blade Server B, the ping from Blade Server A
to Blade Server B will fail.
9. Configure the static route in Switch 1:

SEFOS(config)# ip route 40.0.0.0 255.255.255.0 20.0.0.200
SEFOS(config)# end
10. Configure the static route in Switch 2:
SEFOS(config)# ip route 10.0.0.0 255.255.255.0 20.0.0.201
SEFOS(config)# end
49
11. Verify that the routes are known to Switch 1 and Switch 2.
S 40.0.0.0/24 [-1] via 20.0.0.200

S 10.0.0.0/24 [-1] via 20.0.0.201
C 40.0.0.0/24 is directly connected, vlan40 iss# show ip route
12. Ping from Blade Server A to Blade Server B.


Add static routes on Blade Server A:

# route add -net 20.0.0.0 netmask 255.255.255.0 gw 10.0.0.201
Ping the VLAN 20 interface on Switch 1:
# ping 20.0.0.201

rtt min/avg/max/mdev = 0.722/0.816/0.910/0.094 ms, pipe 2

# ping 20.0.0.200

# ping 40.0.0.200
Add static routes on Blade Server B:

Now that Blade Server A is able to communicate to the VLAN 40 interface on Switch 2, ping Blade
Server B on Switch 2.
# ping 40.0.0.110
64 bytes from 40.0.0.110: icmp_seq=0 ttl=62 time=0.100
ms
ms
ms
ms
From Blade Server B, ping the VLAN 20 interface on Switch 2:

# ping 20.0.0.200
51

# ping 20.0.0.201

# ping 10.0.0.201
From Blade Server B, ping Blade Server A:

# ping 10.0.0.169
ms
ms
ms
ms
To reach Blade Server A from Blade Server B, packets should be routed between VLAN 40 and VLAN 20, and
between VLAN 20 and VLAN 10. You can now test the basic routing functionality of SEFOS now that the static
routes have been configured.
Follow these instructions to remove the static routes on Switch 1 and Switch 2:
1. Remove the static route on Switch 1:
SEFOS(config)# no ip route 40.0.0.0 255.0.0.0 20.0.0.200
SEFOS(config)# end
2. Remove the static route on Switch 2:
SEFOS(config)# no ip route 10.0.0.0 255.0.0.0 20.0.0.201

SEFOS(config)# end

This procedure describes how to configure dynamic routing using the redistribution feature of RIP protocol in
SEFOS. This procedure verifies that accessibility between the two hosts are established after learning the inbetween
routes dynamically through the RIP protocol.
Enable and configure RIP on NEM 1 and NEM 2.
1. Verify that Switch 1 and Switch 2 still have the configuration information that you saved in the section
Configuring Static Unicast Route Entries.
The basic VLAN configuration for dynamic routing is similar to the basic VLAN configuration for static routing. If
the configuration information was not saved on Switch 1 and Switch 2, follow the procedures in Steps 1 6 in
Configuring Static Unicast Route Entries, then return here.
2. Enable RIP protocol on Switch 1:
SEFOS(config)# router rip
SEFOS(config-router)# neighbor 20.0.0.200
SEFOS(config-router)# network 20.0.0.201
SEFOS(config-router)# redistribute all
SEFOS(config-router)# end
3. Enable RIP protocol on Switch 2 using the following commands. If Switch 2 is a third-party switch, verify
that RIP protocol is enabled on Switch 2 and that related configurations are applied accordingly.
SEFOS(config)# router rip
SEFOS(config-router)#
neighbor 20.0.0.201

4. Check the routing database on Switch 1:

SEFOS# show ip rip database
53
Vrf default
10.0.0.0/8
10.0.0.0/24
20.0.0.0/8
20.0.0.0/24
40.0.0.0/8
40.0.0.0/8
[1]
[1]
[1]
[1]
[2]
[2]
auto-summary
directly connected, vlan10
auto-summary
auto-summary
via 20.0.0.200, vlan20
5. Check the routing database on Switch 2:

SEFOS# show ip rip database
Vrf default
10.0.0.0/8
10.0.0.0/8
20.0.0.0/8
20.0.0.0/24
40.0.0.0/8
40.0.0.0/24
[2]
[2]
[1]
[1]
[1]
[1]
auto-summary
via 20.0.0.201, vlan20
auto-summary
auto-summary
6. Ping the VLAN interface on VLAN 10 on Switch 1 from Blade Server B:

# ping 10.0.0.201
....
7. Ping Blade Server A from Blade Server B:

# ping 10.0.0.169
8. Ping Blade Server B from Blade Server A:

# ping 40.0.0.110
ms
ms
ms
ms
You can now test the basic routing functionality of SEFOS now that SEFOS is able to dynamically learn the routing
entries using the redistribution feature of RIP.
Follow these instructions to disable RIP on Switch 1 and Switch 2:
1. Disable RIP on Switch 1:
SEFPS(config)# no router rip
SEFOS(config)# end
2. Disable RIP on Switch 2.
If Switch 2 is a third-party switch, follow the instructions that came with the switch to disable RIP. Otherwise,
follow the instructions given above to disable RIP on Switch 2.

This procedure describes how to configure dynamic routing using the redistribution feature of the OSPF protocol in
SEFOS. This procedure verifies that the reachability between two hosts is established after learning the inbetween
routes dynamically through the OSPF protocol.
Enable and configure RIP on NEM 1 and NEM 2.
1. Verify that Switch 1 and Switch 2 still have the configuration information that you saved in the section
Configuring Static Unicast Route Entries.
The basic VLAN configuration for dynamic routing is similar to the basic VLAN configuration for static routing. If
the configuration information was not saved on Switch 1 and Switch 2, follow the procedures in Steps 1 6 in
Configuring Static Unicast Route Entries, then return here. If Switch 1 or Switch 2 is a third-party switch, configure
the basic VLAN and the dynamic routing as recommended by the switch's manufacturer.
2. Enable the OSPF protocol on Switch 1:
SEFOS(config)# router ospf
SEFOS(config-router)# asBR router
SEFOS(config-router)# network
20.0.0.201 area 0.0.0.0
SEFOS(config-router)# router-id 20.0.0.201

SEFOS(config-router)# network
10.0.0.201 area 0.0.0.0

3. Enable the OSPF protocol on Switch 2 using the following commands. If Switch 2 is a third-party switch,
verify that the OSPF protocol is enabled on Switch 2 and that related configurations are applied
accordingly.
SEFOS(config)# router ospf
SEFOS(config-router)# asBR router
SEFOS(config-router)# network 20.0.0.200 area 0.0.0.0
SEFOS(config-router)# network 40.0.0.200 area 0.0.0.0
55

4. Check the neighbor router status of Switch 1:
SEFOS# show ip ospf neighbor
Vrf
default
Neighbor-ID Pri
----------- --40.0.0.200 1
State
----FULL/DR
DeadTime
39
Address
Interface
-----------------------20.0.0.200 vlan20

Codes: C - connected, S - static, R - rip, B - bgp, O ospf
O 40.0.0.0/24 [2] via 20.0.0.200
5. Verify the neighbor router status of Switch 2 using the following commands. If Switch 2 is a third-party
switch, use appropriate command to verify the neighbor router status.
SEFOS# show ip ospf neighbor
Vrf default
Neighbor-ID Pri
----------- --10.0.0.201
1
State
----FULL/BACKUP
DeadTime
-------32
Address Interface
------- --------20.0.0.201 vlan20

O 10.0.0.0/24 [2] via 20.0.0.201
The connectivity between Blade Server A and Blade Server B can be verified using ping. Follow these procedures to
disable OSPF on Switch 1 and Switch 2:
1. Disable OSPF on Switch 1:
SEFOS(config)# no router ospf
SEFOS(config)# end
2. Disable OSPF on Swich 2.

If Switch 2 is a third-party switch, follow the instructions that came with the switch to disable OSPF. Otherwise,
follow the instructions given above to disable OSPF on Switch 2.

The following figure shows a sample topology for the SFP+ LAG and backplane portface connection.
Figure 5. Sample Topology For the SFP+ LAG and Backplane Portface Connection

This procedure describes how to provide SFP+ two-port 10GbE trunk uplinks using a single Sun Blade 6000
Ethernet Switched NEM 24p 10GbE installed in a Sun Blade 6000 Modular System with up to ten blade servers.
The blade server ports (backplane ports 15-24) will use portfast spanning tree for fast convergence. Note that you
can also disable the spanning tree on these backplane ports.
1. Enable a trunk on SFP+ ports P1 and P2 on the Sun Blade 6000 Ethernet Switched NEM 24p 10GbE.
In this procedure, the trunk is connected to another layer 2 switch, labeled RSW1. A remote server is also connected
to the RSW1 layer 2 switch.
SEFOS(config-if-range)# channel-group 12 mode active
57
SEFOS(config-if-range)# end
Port-channel System Identifier is 00:14:4f:6c:67:0f
Flags:
D down
P - in port-channel
I stand-alone

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(P)
LACP
Ex0/1(P),Ex0/2(P)

SEFOS(config-if-range)# spanning-tree portfast
Warning: portfast should only be enabled on ports connected to a single host.
Connecting hubs, concentrators,switches, bridges, etc... to this interface when
portfast is enabled can cause temporary bridging loops.
Use with CAUTION
Root Id
Priority
32768
Address
00:14:4f:6c:63:0f
Cost
1900
Port
25 [po12]
MST00

Bridge Id
Priority
32768
Address
00:14:4f:6c:67:0f

Name
Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/15
Designated Forwarding 2000
128
P2P
Ex0/17
128
P2P
Ex0/19
128
P2P
Ex0/24
128
P2P
po12
Root
128
P2P
Forwarding 1900
2. Disable spanning tree on these ports, if necessary.

SEFOS(config-if-range)# spanning-tree disable
Root Id
Priority
32768
Address
00:14:4f:6c:63:0f
Cost
1900
Port
25 [po12]

MST00
Bridge Id
Priority
32768
Address
00:14:4f:6c:67:0f
59

Name
Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/15
Disabled
Forwarding
2000
128
P2P
Ex0/17
Disabled
Forwarding
2000
128
P2P
Ex0/19
Disabled
Forwarding
2000
128
P2P
Ex0/24
Disabled
Forwarding
2000
128
P2P
po12
Root
Forwarding
1900
128
P2P
3. Verify that the server that is connected to the remote switch or router is reachable.
[root@nsn57-20 ~]# ping 10.2.2.184
[root@nsn57-20 ~]# ping 10.2.2.46

[root@nsn57-20 ~]# ping 10.2.2.44

-- 10.2.2.44 ping statistics --3 packets transmitted, 3 received, 0% packet loss, time 1999ms
SEFOS#
SEFOS# show mac-address-table

Vlan
Mac Address
Type
Ports
----
-----------
----
-----
00:14:4f:6c:63:0f
Learnt
po12
00:14:4f:6c:63:10
Learnt
po12
00:14:4f:6c:63:27
Learnt
po12
00:21:28:0b:25:4f
Learnt
Ex0/17
00:21:28:2d:f5:c7
Learnt
Ex0/15
1 00:21:28:2d:f6:27 Learnt
1
00:21:28:2d:f7:47 Learnt
po12
Ex0/24
Total Mac Addresses displayed: 7

The following figure shows a sample topology for the Layer 2 switch LAG and QSFP splitter SAN connection.
Figure 6. Sample Topology For the Layer 2 Switch LAG and QSFP Splitter SAN Connection

This procedure describes how to use a single Sun Blade 6000 Ethernet Switched NEM 24p 10GbE installed in a Sun
Blade 6000 Modular System with up to ten blade servers to provide:
SFP+ two-port trunk 10GbE uplinks
One QSFP port with a QSFP to SFP+ splitter connected to four 10GbE storage units
61
The blade server ports (backplane ports 15-24) will use portfast spanning tree for fast convergence. Note that you
can also disable the spanning tree on these backplane ports.
1. Enable a trunk on SFP+ ports P1 and P2 on the Sun Blade 6000 Ethernet Switched NEM 24p 10GbE.
In this procedure, the trunk is connected to another layer 2 switch, labeled RSW1. A remote server is also connected
to the RSW1 layer 2 switch.
Flags:
D down
P - in port-channel
I stand-alone

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(P)
LACP
Ex0/1(P),Ex0/2(P)
2. Enable QSFP ports 11 P14 on the Sun Blade 6000 Ethernet Switched NEM 24p 10GbE.
In this procedure, the cable connected to the QSFP ports 11-14 has the following connections:
A single QSFP connector on one end, connected to the QSFP ports 11-14 on the Sun Blade 6000 Ethernet
Switched NEM 24p 10GbE
On the other end, four SFP+ connectors from the QSFP to SFP+ splitter, onnected to four 10GbE ports on the
server, simulating a storage box

Warning: portfast should only be enabled on ports connected to a single host.
Connecting hubs, concentrators,switches, bridges, etc... to this interface when
portfast is enabled can cause temporary bridging loops.
Use with CAUTION
3. Run the following commands on port 15 of the blade server installed in slot 0 in the Sun Blade 6000
Modular System:
[root@nsn57-28 ~]# ifconfig eth2:2 10.2.2.184/24 up
4. Verify the four 10GbE connections on the server simulating the storage box connections.
Addresses 10.2.X.28 are configured on these 10GbE ports.
[root@nsn57-28 ~]# ifconfig eth2
eth2 Link encap:Ethernet HWaddr 00:1B:21:51:26:EC
inet addr:10.2.2.28 Bcast:10.2.2.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe51:26ec/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4776 errors:0 dropped:0 overruns:0 frame:0
TX packets:423 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:507165 (495.2 KiB) TX bytes:28021 (27.3 KiB)

eth3 Link encap:Ethernet HWaddr 00:1B:21:51:26:ED
inet6 addr: fe80::21b:21ff:fe51:26ed/64 Scope:Link
63


eth4 Link encap:Ethernet HWaddr 00:1B:21:51:26:BC
inet6 addr: fe80::21b:21ff:fe51:26bc/64 Scope:Link

eth5 Link encap:Ethernet HWaddr 00:1B:21:51:26:BD
inet6 addr: fe80::21b:21ff:fe51:26bd/64 Scope:Link
5. Ping the four connections to the simulated storage device from the Blade Server in slot 0 in the Sun Blade
6000 Modular System:
[root@nsn57-184 ~]# ping 10.2.2.28
[root@nsn57-184 ~]# ping 10.2.3.28

[root@nsn57-184 ~]# ping 10.2.4.28

[root@nsn57-184 ~]# ping 10.2.5.28

[root@nsn57-184 ~]# ping 10.2.5.28

6. Ping the remote switch or router from the remote server:

[root@nsn57-28 ~]# ping 10.2.2.20
65

The following figure shows a sample topology for the Switched NEM high availability connection.
Figure 7. Sample Topology For the Switched NEM High Availability Connection

This example configuration illustrates a High Availability configuration where the 10GbE connections are failed
over between two NEMs, one in slot NEM0 and the other in slot NEM1.
In the example, two Sun Blade 6000 Modular System chassis are used. The first Sun Blade 6000 Modular System
chassis (SBC6000_1) is used to show the sample High Availability configuration using Linux and Solaris OS-based
blade servers. The other Sun Blade 6000 Modular System chassis (SBC6000_2) is used to simulate a remote
switch/router and server resource that needs to be reached from the blade servers on chassis SBC6000_1, and it is
configured in a similar manner. The QSFP ports used for interconnections are configured as a link aggregation.
Each blade server has a dual-port 10GbE FEM card, with each 10GbE interface connected to the NEMs. One of the
FEMs 10GbE interface is connected to the NEM in slot NEM0, and the other FEM interface is connected to the
NEM in slot NEM1.
For Linux hosts, these two FEM interfaces are configured into a single bonding inteface with the High Availability
configuration option. For Solaris hosts, the two FEM interfaces are configured into the same IMPM failover group.
1. Configure the Blade Servers and QSFP ports on the SBC6000_1 chassis.
Enable bonding on the Linux server:
[root@linux-184 ~]# modprobe bonding arp_interval=1000
arp_ip_target=38.38.38.210 mode=1
[root@linux-184 ~]# ifconfig bond0 38.38.38.184/24 up
Interface)
(monitor
[root@linux-184 ~]# ifenslave bond0 eth eth2

[root@linux-184 ~]# ifconfig bond0:5 5.5.5.184/24 up
Interface)
(test
Enable the QSFP ports P3-6 and P7-10 on the NEM in slot NEM0. Configure the link aggregation
interfaces Po100 and Po200 on these QSFP ports:
SEFOS(config)# exit
Flags:
D down
P - in port-channel
I stand-alone
67
Group Port-channel Protocol
Ports
------------------------------------------------------------------------100
200
Po100(D)
Po200(D)
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
SEFOS(config)# exit
Flags:
D down
P - in port-channel
I stand-alone

Ports
------------------------------------------------------------------------100
Po100(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
200
Po200(D)
LACP
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
2. Configure the Blade Servers and QSFP ports on the SBC6000_2 chassis.
Enable bonding on the Linux server:
[root@linux-210 ~]# modprobe bonding arp_interval=1000
arp_ip_target=38.38.38.184 mode=1
[root@linux-210 ~]# ifconfig bond0 38.38.38.210/24 up
Interface)
(monitor
[root@linux-210 ~]# ifenslave bond0 eth eth2

[root@linux-210 ~]# ifconfig bond0:5 5.5.5.210/24 up
Interface)
(test
SEFOS(config)# exit
Port-channel System Identifier is 00:14:4f:7c:63:11
Flags:
69
D down
P - in port-channel
I stand-alone

Ports
------------------------------------------------------------------------100
200
Po100(P)
Po200(P)
LACP
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
SEFOS(config)# exit
Flags:
D down
P - in port-channel
I stand-alone

Ports
------------------------------------------------------------------------100
Po100(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
200
Po200(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
3. Check the link aggregation and spanning tree status on each of the four NEMs.
Check the link aggregation and spanning tree status on the NEM in slot NEM0 in the SBC6000_1
chassis:
Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone
Admin Status is enabled

Oper Status is enabled
Identifier is 00:14:4f:6c:56:0f
P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(P)
Po200(P)
LACP
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
Note that the entries in the Port-channel and Ports columns changed from D (down) to P (in port-channel)
for the Sun Blade 6000 Ethernet Switched NEM 24p 10GbE installed in NEM slot 0 in the first Sun Blade
6000 Modular System.
Root Id
Priority 0
Address 00:00:00:00:00:00
Cost 0
Port 0 [0]
MST00
Spanning tree Protocol has been disabled
Bridge Id
Priority 32768
Address 00:00:00:00:00:00
Name
Role
State
Cost
Prio
Type
71
----
----
-----
----
----
------
Ex0/15
Disabled
Unknown
2000
128
P2P
Ex0/17
Disabled
Unknown
2000
128
P2P
Ex0/19
Disabled
Unknown
2000
128
P2P
Ex0/24
Disabled
Unknown
2000
128
P2P
po100
Disabled
Unknown
1900
128
P2P
po200
Disabled
Unknown
1900
128
P2P
chassis:
Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(P)
Po200(P)
LACP
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
Note that the entries in the Port-channel and Ports columns changed from D (down) to P (in port-channel)
for the NEM installed in slot NEM 1 in the SBC6000_1 chassis.
Root Id
Priority 32768
Address 00:14:4f:6c:63:0f
Cost 1900
Port 25 [po100]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po100
Root
Forwarding
1900
128
P2P
po200
Designated
Forwarding
1900
128
P2P
4. Check the bonding interface status for the Linux Blade Server linux-184 in the SBC6000_1 chassis.
[root@nsn57-184 ~]#
more /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.0.3 (March 23, 2006)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth2
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0
ARP Polling Interval (ms): 1000
ARP IP target/s (n.n.n.n form): 38.38.38.210
Slave Interface: eth
MII Status: up
Link Failure Count: 4
Permanent HW addr: 00:21:28:2d:f5:c6
Slave Interface: eth2
MII Status: up
This output confirms that the switched NEM installed in NEM slot 1 in the first Sun Blade 6000 Modular System is
the active path.
5. Unplug the cables from QSFP ports 3-6 and 7-10 on the NEM in slot NEM1 in the SBC6000_1 chassis.
6. Check the interface status on the SBC6000_1 chassis.
Check the link status of the link aggregation interfaces and the spanning tree status on the NEM in slot
NEM1 in the SBC6000_1 chassis. The link aggregation interfaces should be down and the spanning tree
root should move somewhere else.
Root Id
Priority 32768
73
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Name

Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(D)
Po200(D)
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
NEM0 in the SBC6000_1 chassis:
Root Id
Priority 0
Address 00:00:00:00:00:00
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Address 00:00:00:00:00:00
Name

Role
State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/15
Disabled
Unknown
2000
128
P2P
Ex0/17
Disabled
Unknown
2000
128
P2P
Ex0/19
Disabled
Unknown
2000
128
P2P
Ex0/24
Disabled
Unknown
2000
128
P2P
po10
Disabled
Unknown
1900
128
P2P
po200
Disabled
Unknown
1900
128
P2P

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(P)
Po200(P)
LACP
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
Check the bonding status on the the linux-184 Blade Server in the SBC6000_1 chassis. The active
bonding interface should be changed from eth2 to eth. The 10GbE interface eth corresponds to
the NEM in slot NEM0.
[root@linux-184 ~]#

Primary Slave: None
Currently Active Slave: eth
MII Status: up
Up Delay (ms): 0
Down Delay (ms): 0
MII Status: up
75

MII Status: down
7. Check the bonding interface status for the Blade Server in the SBC6000_2 chassis.
[root@nsn57-20 ~]#
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)

Primary Slave: None
MII Status: up
Up Delay (ms): 0
Down Delay (ms): 0
MII Status: up
Permanent HW addr: 00:21:28:2d:f6:26
MII Status: up

Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Name
Role

State
Cost
Prio
Type
----
----
-----
----
----
------
Ex0/1
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po200
Designated
Forwarding
1900
128
P2P

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(D)
Po200(P)
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
Root Id
Priority 32768
Cost 1900
Port 25 [po100]
MST00
MST00 is executing the mstp compatible Multiple Spanning Tree Protocol
Bridge Id
Priority 32768
Address 00:14:4f:7c:63:11
State
Cost
Prio
Type
Name
Role
----
----
-----
----
----
------
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po100
Root
Forwarding
1900
128
P2P
77

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

Identifier is 00:14:4f:7c:63:11
P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(P)
Po200(D)
LACP
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
9. Unplug the cables from QSFP ports 3-6 and 7-10 on the NEM in slot NEM0 in the SBC6000_2 chassis..
10. Check the bonding status on the Blade Server in the SBC_60002 chassis.
[root@nsn57-20 ~]#
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)

Primary Slave: None
MII Status: up
Up Delay (ms): 0
Down Delay (ms): 0
MII Status: down
MII Status: up

Root Id
Priority 32768
Address 00:14:4f:7c:63:11
Cost 0
Port 0 [0]

MST00
MST00 is executing the mstp compatible Multiple Spanning Tree Protocol
Bridge Id
Priority 32768
Address 00:14:4f:7c:63:11
State
Cost
Prio
Type
Name
Role
----
----
-----
----
----
------
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

Identifier is 00:14:4f:7c:63:11
P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(D)
Po200(D)
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
79

State
Cost
Prio
Type
Name
Role
----
----
-----
----
----
------
Ex0/1
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po200
Designated
Forwarding
1900
128
P2P

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(D)
Po200(P)
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)

Check the bonding interface status on the for the Linux Blade Server linux-184 in the SBC6000_1
chassis.
[root@nsn57-184 ~]#

Primary Slave: None
Currently Active Slave: eth
MII Status: up
Up Delay (ms): 0
Down Delay (ms): 0
MII Status: up
MII Status: down

chassis:
Root Id
Priority 0
Address 00:00:00:00:00:00
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Address 00:00:00:00:00:00
State
Cost
Prio
Type
Name
Role
----
----
-----
----
----
------
Ex0/15
Disabled
Unknown
2000
128
P2P
Ex0/17
Disabled
Unknown
2000
128
P2P
Ex0/19
Disabled
Unknown
2000
128
P2P
Ex0/24
Disabled
Unknown
2000
128
P2P
po200
Disabled
Unknown
1900
128
P2P

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(D)
Po200(P)
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
81
chassis:
Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
State
Cost
Prio
Type
Name
Role
----
----
-----
----
----
------
Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P

Port-channel Module
Port-channel Module
Port-channel System
Flags:
D down
I stand-alone

P - in port-channel

Ports
------------------------------------------------------------------------100
200
Po100(D)
Po200(D)
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
13. Configure and verify High Availability on the Solaris Blade server on the SBC6000_1 chassis.
These steps will reestablish a connection on the unplugged QSFP cable and will verify High Availability on the
Linux Blade Server. The Solaris Blade Server has two 10GbE interfaces, just as the Linux Blade Server has.
Interface ixgbe0 is connected to the NEM in slot NEM0 and interface ixgbe1 is connected to the NEM in slot NEM1
in the SBC6000_1 chassis.
Configure and verify High Availability on the ixgbe0 10GbE interface:
[root@Solaris-190
[root@Solaris-190
broadcast + group
-failover netmask
~]# ifconfig ixgbe0 plumb

~]# ifconfig ixgbe0 38.38.38.190 netmask +
testgroup1 up \ addif 38.38.38.192 deprecated
+ broadcast + up
Configure and verify High Availability on the ixgbe1 10GbE interface:

[root@Solaris-190
[root@Solaris-190
broadcast + group
-failover netmask
~]# ifconfig ixgbe1 plumb

~]# ifconfig ixgbe1 38.38.38.191 netmask +
testgroup1 up \ addif 38.38.38.193 deprecated
+ broadcast + up
14. Add the target for probe-based monitoring:

[root@nsn57-184 ~]# route add -host 38.38.38.210 38.38.38.210 -static
15. Ping the target Blade Server in the SBC6000_2 chassis.
The ping should succeed.
16. Remove the NEM from slot NEM0 in the SBC6000_1 chassis.
Refer to the Sun Blade 6000 Ethernet Switched NEM 24p 10GbE User's Guide for those instructions, then return
here. Removing the switched NEM from a running system will simulate having a failed switched NEM in that slot.
17. Monitor the console output of the Solaris Blade Server in the SBC6000_1 chassis:
Feb 25 18:29:36 Solaris-190 in.mpathd[858]: [ID 594170 daemon.error] NIC
failure detected on ixgbe0 of group testgroup1
Feb 25 18:29:36 Solaris-190 in.mpathd[858]: [ID 832587 daemon.error]
Successfully failed over from NIC ixgbe0 to NIC ixgbe1
Note that the ixgbe1 10 GbE interface has become the active IPMP interface.
18. Re-insert the NEM into slot NEM0 in the SBC6000_1 chassis.
repair detected on ixgbe0 of group testgroup1
Successfully failed back to NIC ixgbe0
83
Note that the active link has failed back over to the ixgbe0 10GbE interface.
20. Disable the 10GbE interface corresponding to the ixgbe0 link on the NEM in slot NEM0 in the
SBC6000_1 chassis:
SEFOS#
Feb 25 18:35:07 Solaris-190 ixgbe: [ID 339653 kern.notice] NOTICE: ixgbe0:
xcvr addr:0x00 - link is down
Feb 25 18:35:07 Solaris-190 in.mpathd[858]: [ID 215189 daemon.error] The
link has gone down on ixgbe0
failure detected on ixgbe0 of group testgroup1
Successfully failed over from NIC ixgbe0 to NIC ixgbe1
Feb 25 18:35:07 Solaris-190 ixgbe: [ID 752849 kern.warning] WARNING: ixgbe0
: portn 0
Note that the active link has failed over to the ixgbe1 10GbE interface.
22. Enable the 10GbE interface corresponding to the ixgbe0 link on the NEM in slot NEM0 in the
SBC6000_1 chassis:
SEFOS#
Link ixgbe0 recovered
Feb 25 18:35:57 Solaris-190 ixgbe: [ID 339653 kern.notice] NOTICE: ixgbe0:
xcvr addr:0x00 - link is up 10000 Mbps full duplex
Feb 25 18:35:57 Solaris-190 in.mpathd[858]: [ID 820239 daemon.error] The
link has come up on ixgbe0
Feb 25 18:35:58 Solaris-190 in.routed[350]: [ID 464608 daemon.error] route
38.38.38.192 --> 0.0.0.0 nexthop is not directly connected
repair detected on ixgbe0 of group testgroup1
Successfully failed back to NIC ixgbe0
Note that the active link has failed back over to the ixgbe0 10GbE interface.

The following figure shows a sample topology for four Switched NEMs, high availability connection.
Figure 8. Sample Topology For Four Switched NEMs, High Availability Connection

In this example, the configuration consists of a cluster of servers within four Sun Blade 6000 Modular System
chassis. Each Sun Blade 6000 Modular System chassis has one or two NEMs (Sun Blade 6000 Ethernet Switched
NEM 24p 10GbEs from Oracle) installed. The NEMs are interconnected in the topology shown in Figure 8 using the
QSFP ports. The four ports on each QSFP port are configured as a link aggregation.
For the purposes of this exercise, the following components will use the following Ethernet addresses:
NEM A, installed in Chassis A: Ethernet address 1.1.1.184
NEM B, installed in Chassis B: Ethernet address 1.1.1.190
NEM C, installed in Chassis C: Ethernet address 1.1.1.156
NEM D, installed in Chassis D: Ethernet address 1.1.1.20
85
Setting Up QSFP Ports Into a Link Aggregation

This procedure describes how to configure the QSFP ports as a link aggregation. Note that you can copy and paste
this information to a SEFOS console or you can use this information to create a script.
Checking the Connectivity Status on the NEMs

1. Check the connectivity status on NEM A:
Root Id
Priority 32768
Cost 1900
Port 26 [po36]
MST00
Bridge Id
Priority 32768
State
Cost
Prio
Type
Name
Role
----
----
-----
----
----
------
Ex0/15
Ex0/17
Ex0/19
Ex0/21
Ex0/22
Ex0/23
Ex0/24
po36
po710
po1114
Designated
Designated
Designated
Designated
Designated
Designated
Designated
Root
Alternate
Designated
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
Discarding
Forwarding
2000
2000
2000
2000
2000
2000
2000
1900
1900
1900
128
128
128
128
128
128
128
128
128
128
P2P
P2P
P2P
P2P
P2P
P2P
P2P
P2P
P2P
P2P

87
Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)
2. Check the connectivity status on NEM B:

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Ex0/21
Ex0/22
Ex0/23
Ex0/24
po36
po710
po1114
Designated
Designated
Designated
Designated
Designated
Designated
Designated
Designated
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
Forwarding
2000
2000
2000
2000
2000
1900
1900
1900
128
128
128
128
128
128
128
128
P2P
P2P
P2P
P2P
P2P
P2P
P2P
P2P
3. Check the connectivity status on NEM C:

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 1900
Port 28 [po1114]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po36
Designated
Forwarding
1900
128
P2P
po710
Designated
Forwarding
1900
128
P2P
po1114
Root
Forwarding
1900
128
P2P
89
4. Check the connectivity status on NEM D:

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 1900
Port 27 [po710]
MST00
Bridge Id
Priority 32768
Address 00:14:4f:7c:63:11
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po36
Alternate
Discarding
1900
128
P2P
po710
Root
Forwarding
1900
128
P2P
po1114
Alternate
Discarding
1900
128
P2P
The spanning tree and link aggregation status indicates that there is full mesh connectivity between the NEMs and
the Sun Blade 6000 Modular System chassis.
5. Ping the other chassis from Chassis B (1.1.1.190):
# ping 1.1.1.184; ping 1.1.1.156; ping 1.1.1.20
1.1.1.184 is alive
1.1.1.156 is alive
1.1.1.20 is alive
The following steps will verify that redundant paths exist between the multiple chassis.
6. Disconnect the QSFP cable connecting NEM A in Chassis A to NEM B in Chassis B (QSFP ports 3-6).
7. Check the connectivity on NEM A installed in Chassis A:
Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)
Note that trunk link Po36 is now down.

Root Id
Priority 32768
Cost 3800
Port 27 [po710]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po710
Root
Forwarding
1900
128
P2P
po1114
Alternate
Discarding
1900
128
P2P
91
8. Enter the following commands for NEM B installed in Chassis B:

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po710
Designated
Forwarding
1900
128
P2P
po1114
Designated
Forwarding
1900
128
P2P
9. Enter the following commands for NEM C installed in Chassis C:


Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 1900
Port 28 [po1114]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po36
Designated
Forwarding
1900
128
P2P
po710
Designated
Forwarding
1900
128
P2P
po1114
Root
Forwarding
1900
128
P2P
10. Enter the following commands for NEM D installed in Chassis D:

Flags:
D down
I stand-alone
P - in port-channel

93
Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 1900
Port 27 [po710]
MST00
Bridge Id
Priority 32768
Address 00:14:4f:7c:63:11
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po36
Alternate
Discarding
1900
128
P2P
po710
Root
Forwarding
1900
128
P2P
po1114
Designated
Forwarding
1900
128
P2P

# ping 1.1.1.184; ping 1.1.1.156; ping 1.1.1.20
1.1.1.184 is alive
1.1.1.156 is alive
1.1.1.20 is alive
This step in the procedure verifies that even though there is a link failure of the link aggregation between the chassis
(in this case, between NEM A and NEM B, QSFP ports 3-6), there is still network connectivity between the two
NEMs.
The following steps will introduce an additional link failure to verify that redundant paths continue to exist between
the multiple chassis.
12. Disconnect the QSFP cable connecting NEM B in Chassis B to NEM D in Chassis D (QSFP ports 7-11).

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 3800
Port 27 [po710]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po710
Root
Forwarding
1900
128
P2P
po1114
Designated
Forwarding
1900
128
P2P
14. Check the connectivity on NEM B installed in Chassis B:

Flags:
D down
I stand-alone
P - in port-channel
95

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(D)
LACP
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po1114
Designated
Forwarding
1900
128
P2P
15. Check the connectivity on NEM C installed in Chassis C:

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
-------------------------------------------------------------------------
12
36
710
1114
Po12(D)
Po36(P)
Po710(P)
Po1114(P)
LACP
LACP
LACP
LACP
Ex0/1(D),Ex0/2(D)
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 1900
Port 28 [po1114]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po36
Designated
Forwarding
1900
128
P2P
po710
Designated
Forwarding
1900
128
P2P
po1114
Root
Forwarding
1900
128
P2P
16. Check the connectivity on NEM D installed in Chassis D:

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(P)
LACP
Ex0/3(P),Ex0/4(P),Ex0/5(P),Ex0/6(P)
710
Po710(D)
LACP
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

97
Root Id
Priority 32768
Cost 3800
Port 26 [po36]
MST00
Bridge Id
Priority 32768
Address 00:14:4f:7c:63:11
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po36
Root
Forwarding
1900
128
P2P
po1114
Alternate
Discarding
1900
128
P2P

# ping 1.1.1.184; ping 1.1.1.156; ping 1.1.1.20
1.1.1.184 is alive
1.1.1.156 is alive
1.1.1.20 is alive
This step in the procedure verifies that even though there are now two link failures of the link aggregation between
the chassis (in this case, between NEM A and NEM B, ports 3-6, and from between NEM B and NEM D, ports 711), there is still network connectivity between the chassis.
The following steps will introduce another link failure to verify that redundant paths continue to exist between the
multiple chassis.
18. Disconnect the QSFP cable connecting NEM C in Chassis C to NEM D in Chassis D (QSFP ports 3-6).
Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
710
1114
Po36(D)
Po710(P)
Po1114(P)
LACP
LACP
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 3800
Port 27 [po710]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po710
Root
Forwarding
1900
128
P2P
po1114
Designated
Forwarding
1900
128
P2P

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(D)
LACP
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)
99
Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po1114
Designated
Forwarding
1900
128
P2P

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 1900
Port 28 [po1114]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po710
Designated
Forwarding
1900
128
P2P
po1114
Root
Forwarding
1900
128
P2P

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(D)
LACP
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
101
Name
---Ex0/15
Ex0/17
Ex0/19
Ex0/21
Ex0/22
Ex0/23
Ex0/24
po1114
Max age is 20 sec, forward delay
Role
State
Cost
----------Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
1900
is 15 sec
Prio
---128
128
128
128
128
128
128
128
Type
-----P2P
P2P
P2P
P2P
P2P
P2P
P2P
P2P

# ping 1.1.1.184; ping 1.1.1.156; ping 1.1.1.20
1.1.1.184 is alive
1.1.1.156 is alive
1.1.1.20 is alive
This step in the procedure verifies that even though there are now three link failures of the link aggregation between
the chassis, there is still network connectivity between the chassis:
QSFP cables are disconnected between NEM A and NEM B, ports 3-6
QSFP cables are disconnected between NEM B and NEM D, ports 7-11
QSFP cables are disconnected between NEM C and NEM D, ports 3-6
The following steps will introduce another link failure to verify that even if more trunks fail, there could still be
limited connection between some of the NEMs.
24. Disconnect the QSFP cable connecting NEM A in Chassis A to NEM D in Chassis D (QSFP ports 11-14).
NEM A, NEM B and NEM C can still communicate with each other with this simulated trunk failure.
Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
-------------------------------------------------------------------------
12
36
710
1114
Po12(D)
Po36(D)
Po710(P)
Po1114(D)
LACP
LACP
LACP
LACP
Ex0/1(D),Ex0/2(D)
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
Ex0/11(D),Ex0/12(D),Ex0/13(D),Ex0/14(D)

Root Id
Priority 32768
Cost 3800
Port 27 [po710]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po710
Root
Forwarding
1900
128
P2P

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(D)
LACP
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)
103

Root Id
Priority 32768
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/17
Designated
Forwarding
2000
128
P2P
Ex0/19
Designated
Forwarding
2000
128
P2P
Ex0/21
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
Ex0/23
Designated
Forwarding
2000
128
P2P
Ex0/24
Designated
Forwarding
2000
128
P2P
po1114
Designated
Forwarding
1900
128
P2P

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(P)
LACP
Ex0/7(P),Ex0/8(P),Ex0/9(P),Ex0/10(P)
1114
Po1114(P)
LACP
Ex0/11(P),Ex0/12(P),Ex0/13(P),Ex0/14(P)

Root Id
Priority 32768
Cost 1900
Port 28 [po1114]
MST00
Bridge Id
Priority 32768
Name
Role
State
Cost
Prio
Type
---------------------Ex0/15
Designated
Forwarding
2000
128
P2P
Ex0/16
Designated
Forwarding
2000
128
P2P
Ex0/18
Designated
Forwarding
2000
128
P2P
Ex0/22
Designated
Forwarding
2000
128
P2P
po710
Designated
Forwarding
1900
128
P2P
po1114
Root
Forwarding
1900
128
P2P

Flags:
D down
I stand-alone
P - in port-channel

Group Port-channel
Protocol
Ports
------------------------------------------------------------------------12
Po12(D)
LACP
Ex0/1(D),Ex0/2(D)
36
Po36(D)
LACP
Ex0/3(D),Ex0/4(D),Ex0/5(D),Ex0/6(D)
710
Po710(D)
LACP
Ex0/7(D),Ex0/8(D),Ex0/9(D),Ex0/10(D)
1114
Po1114(D)
LACP
Ex0/11(D),Ex0/12(D),Ex0/13(D),Ex0/14(D)

Root Id
Priority 32768
Address 00:14:4f:7c:63:11
Cost 0
Port 0 [0]
MST00
Bridge Id
Priority 32768
Address 00:14:4f:7c:63:11
105
Name
---Ex0/15
Ex0/16
Ex0/18
Ex0/22
Max age is 20 sec, forward delay

Role
State
Cost
----------Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
2000
Designated
Forwarding
2000
is 15 sec
Prio
---128
128
128
128
Type
-----P2P
P2P
P2P
P2P

# ping 1.1.1.184; ping 1.1.1.156; ping 1.1.1.20
1.1.1.184 is alive
1.1.1.156 is alive
no answer from 1.1.1.20
This step in the procedure verifies that even though there are several failures of the link aggregation between the
chassis, there is still network connectivity between the two or three of the NEMs installed in the chassis.

Sun Blade 6000 Ethernet Switched Software Configuration Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sun Blade 6000 Ethernet Switched Software Configuration Guide

Uploaded by

Copyright:

Available Formats

Sun Blade 6000 Ethernet Switched

NEM 24p 10GbE

Part Number E26660-01

Setting Up a Four NEM/Two Chassis High Availibility Configuration

All Oracle products

Sun Blade 6000 Ethernet Switched

Sun Network 10GbE Switch 72p

Sun Blade 6000 Modular System

Oracle Integrated Lights Out

Acronyms and Abbreviations

Access Control List

Command Line Interface

Dynamic Host Configuration Protocol

Differentiated Services Code Point

Hyper Text Transfer Protocol

Internet Control Message Protocol

Media Access Control

Maximum Transfer Unit

Non-Volatile Random Access Memory

Port based Network Access Control

Remote Authentication Dial-In User Service

Reverse Address Resolution Protocol

Sun Ethernet Fabric Operating System

Simple Network Management Protocol

Transmission Control Protocol

CLI Command Modes

Access SEFOS from the

Use the logout or exit

Access SEFOS from

Use the logout or exit

Use the end command to

From Global Configuration SEFOS(config-if)#

Use the exit command to

Support and Accessibility

Access electronic support through

Learn about Oracle's commitment

Configuring Switch Clock

cd: The session with /NEM/fs_cli has ended.

Configuring SEFOS System Features

Figure 1. Configuration of Basic System Features

Figure 2. Configuration of Advanced System Features

Dynamic IP Allocation Protocol

Login Authentication Mode

All users are allowed to configure the switch

Restoration File Name

HTTP Port Number

Trap Generation on an Interface

Checksum on Tunnel Interface

MTU Discovery on Tunnel Interface

Max number of ISID

Max number of ISID per context

Max number of ports per ISID

Max number of ports per ISID_per_context

Max number of SVLANs per ISID

Max number of CVLANs per ISID

Configuring the Default IP Address

Default Subnet Mask

Default IP Address Config Mode

Default IP Address Allocation Protocol

Switch Base MAC Address

Default Interface Name

Default RM Interface Name

Config Restore Option

Config Save Option