Bill Wiegand Consulting I/T Specialist June 2013

IBM Flex System V7000 Technical Overview

2013 IBM Corporation

Agenda
Flex System Hardware Overview SAS Networking Clustering and Network Integration Management Virtualization Overview Array Configuration Advanced Functions

2013 IBM Corporation

The IBM Flex System Platform

The Flex System V7000 Storage node fits within the Flex Enterprise chassis to provide integrated storage and servers in a single package
Runs the exact same software as the Storwize V7000 and SAN Volume Controller (SVC)

2013 IBM Corporation

The IBM Flex System Platform

The Flex Enterprise Chassis provides the following services to the Flex System V7000 Storage Node:
1 Gb/s Management network 4 host attachment networks
10 Gb/s Ethernet using FCoE or iSCSI 8 Gb/s or 16 Gb/s Fibre Channel

The host attachment networks must always be configured in 2 pairs when used with the Flex System V7000 A reliable power supply source and cooling

2013 IBM Corporation

Flex System V7000 Enclosures - Front View

Control Enclosure

Expansion Enclosure
5 2013 IBM Corporation

Internal Components
HIC card 2 FC or 10gigE Full Wide Node Canisters HIC card 1 10gigE only

Control Enclosure Drive Caddy

24, 2.5 HDD Bays

2013 IBM Corporation

Drive Options
The following drives are supported in the Flex System V7000:
2.5" 10k SAS HDD, 300GB (Seagate / Hitachi) 2.5" 10k SAS HDD, 600GB (Seagate / Hitachi) 2.5" 10k SAS HDD, 900GB (Seagate / Hitachi) 2.5" NL SAS, 500GB (Seagate) 2.5" NL SAS, 1TB (Seagate) 2.5" 15k SAS, 146GB (Seagate) 2.5" 15k SAS, 300GB (Seagate) 2.5" SSD SAS, 200GB (Hitachi) 2.5" SSD SAS, 400GB (Hitachi)

LFF 3.5 NL_SAS drives in 2/3/4TB capacities are available by using Storwize V7000 expansion enclosures

2013 IBM Corporation

Control Enclosure Node Canister

Intel Jasper Forest quad core 64-bit CPU running at 2.1 GHz 8GB Memory per node canister 16GB per control enclosure 32GB internal flash drive for Linux and virtualization software One or two Host Interface Cards (HIC) Slot 1 HIC must always be:
Two port 10 Gb/s Ethernet Adapter supporting FCoE and/or iSCSI

Slot 2 HIC can be either:

Four 2/4/8 Gb/s Fibre Channel ports Two port 10 Gb/s Ethernet Adapter supporting FCoE and/or iSCSI

One internal 10/100/1000 Mb/s Ethernet for management (no iSCSI) One external 6 Gb/s SAS wide port (24Gb aggregate) and usage optional Two external USB Ports (not used for normal operation) One Battery

2013 IBM Corporation

Expansion Enclosure Canister

Two 6 Gb/s SAS ports with usage of port 1 is mandatory and use of port 2 is optional The SAS port on the Flex System V7000 expansion canisters are HD Mini SAS ports, and the Storwize V7000 canister SAS ports are Mini SAS
Both are functionally the same, but have different sized connectors

2013 IBM Corporation

IBM Flex System V7000 Technical Overview

SAS Networking

10

2013 IBM Corporation

SAS Network Cabling

11

2013 IBM Corporation

SAS Network Cabling

12

2013 IBM Corporation

SAS Network Cabling

The SAS chain can contain up to 10 enclosures in any one of the following configurations:
Flex System V7000 Control Enclosure only Flex System V7000 Control Enclosure + 1 Flex System V7000 Expansion Enclosure in the same Enterprise chassis + 0 - 8 Storwize V7000 Expansion Enclosures Flex System V7000 Control Enclosure + 2 Flex System V7000 Expansion Enclosure in the same Enterprise chassis + 0 - 7 Storwize V7000 Expansion Enclosures

The control enclosure MUST be cabled with Flex System V7000 expansion enclosures before the Storwize V7000 expansion enclosures All Flex System V7000 expansion enclosures must be in the same Flex Enterprise chassis as control enclosure
13 2013 IBM Corporation

Flexible Configuration
Integrated into the Flex Enterprise chassis
Flex Chassis
Storage ITE Dual Controller Storage ITE JBOD Storage ITE JBOD

Scalable to external JBODS

Flex Chassis

Storage ITE JBOD

Flex System V7000 Storage Node

14 2013 IBM Corporation

Standard Rack Enclosure

Scalable within a Flex Enterprise chassis

IBM Flex System V7000 Technical Overview

Clustering and Network Integration

15

2013 IBM Corporation

Scale Multiple Ways

Flex System V7000 One I/O Group System
Control Enclosure

Flex System V7000 2-4 I/O Groups Clustered System

Control Enclosure Control Enclosure

Cluster

Expand

Expand

An I/O Group is a control enclosure and its associated SAS attached expansion enclosures Clustered system can consist of 2-4 I/O Groups Scale capacity/throughput 4x
Up 960 internal drives in 40 enclosures within two 42U racks

No interconnection of SAS chains between control enclosures as control enclosures communicate via FC Expansion Enclosures Expansion Enclosures Expansion Enclosures

Non-disruptive upgrades
From smallest to largest configurations Purchase hardware only when you need it
No extra feature to order and no extra charge for a clustered system

Cluster

An I/O Group is a control enclosure and its associated SAS connected expansion enclosures
16 2013 IBM Corporation

Virtualize external storage arrays behind Flex System V7000 for even greater capacity and throughput

Cluster Control Enclosures

All four control enclosures must be interconnected using a minimum of two FC ports per node canister
FCoE is not currently supported for clustering, only FC is today Flex chassis must have FC switch modules and control enclosures must have FC HICs

Recommended FC connectivity is via a FC top of rack switch or via an existing Brocade/Cisco FC SAN
Can interconnect FC switch modules in Flex Enterprise chassis via ISls if desired in lieu of using a TOR switch

17

2013 IBM Corporation

Flex System V7000 Storage Node with Fiber Channel

Switch Configurations (per switch) 8GB QLogic 14 internal ports 1 to each Node Canister
Means max of 2 FC ports used per node canister
FC enabled Storage Node to integrated FC switches Connect to externally virtualized FC storage

16GB Brocade 12 Ports active with Ports On Demand

All internal 10 Compute Nodes + 1 to each Node Canister 10 internal 6 Compute Nodes + 2 to each Node Canister + 2 uplink ports
Virtualized External Storage

16GB Brocade 24 Ports active with Ports On Demand

10 Compute nodes (1 port each) + 2 ports to each node canister + 10 uplink ports Red line is Fibre Channel
Fiber Channel Adapter Flex System V7000 Cisco or Brocade SAN

Brocade or QLogic FC switches

18

2013 IBM Corporation

Flex System V7000 Storage Node with Integrated FCoE

First truly FCoE optimized storage
No cost wasted on non-FCoE ports
Virtualized External Storage

All FCoE traffic via mid-plane to integrated switch Easily integrated into customer FC SAN
Virtualize other FC storage via FCoE connection
Black line is FCOE Red line is Fibre Channel Green line is Ethernet
LOM or CN4054 (pNIC, vNIC mode) Flex System V7000 Storwize V7000 FCoE

Cisco or Brocade SAN

Ethernet Network G8264/G8316

CN4093 Flex System Fabric Converged Switch

19

2013 IBM Corporation

IBM Flex System V7000 Technical Overview

Management

20

2013 IBM Corporation

Auto-Discovering a New Storage Node

Storage Node 01

21

2013 IBM Corporation

Auto-Discovering a New Storage Node

Insert Flex System V7000 Node Wait 5 minutes for power up Use chassis map task to launch browser to Easy Setup Wizard

22

2013 IBM Corporation

System Manager Chassis Map Operations

Chassis map location Flyover status

Rapid access to details V7000 GUI a click away

23

23

2013 IBM Corporation

Native GUI Launched from Remote Access Task

Deeper device specific functions

24

24

2013 IBM Corporation

Update Management Capability for Flex System V7000 Storage Node

Centrally manage the device firmware

25

2013 IBM Corporation

IBM Flex System V7000 Technical Overview

Virtualization Overview

26

2013 IBM Corporation

Flex System V7000 Built on Virtualization

Servers

Servers

Servers

Flex System V7000 Virtualized Storage Pools

Flex System V7000 Virtualized Storage Pools

Flex System V7000 Virtualized Storage Pools

External Array

Internal
Increases disk utilization and reduces hot spots Improves administrative productivity by up to 2X

External
External arrays inherit all Storwize V7000 functions Extends life of existing storage arrays Optional feature

Migrate
Migrate data from external LUNs to Flex System V7000 Thin or fully provisioned target volumes Included at no cost for 45 days from migration start

27

2013 IBM Corporation

Virtualization The Big Picture

Designed to be redundant, modular and scalable solution

System consists of one to four I/O Groups managed as a single entity

Volumes NC NC Volumes NC

Storage Area Network

Volumes NC NC

Control enclosure with 2 node canisters (NC) makes up an I/O Group and owns given volumes
Volumes NC NC

NC

Control Enclosure

Control Enclosure

Control Enclosure

Control Enclosure

Managed Disks a Cluster Resource

28

2013 IBM Corporation

Virtualization The Big Picture

Volumes: Max 8192 Volumes, 2048 per I/O Group, with each up to 256TB in size and each assigned to: A specific I/O Group Built from a specific Storage Pool I/O Group 0 I/O Group 1 Cluster: Max 4 I/O Groups built from 4 control enclosures Managed Disks (MDisks): Internally or externally provided Max 4096 MDisks per System Storage Pools: Max 128 Storage Pools Max 128 MDisks per Pool

Control Enclosure

Control Enclosure

Flex System V7000 System MDG1 MDG3

Pool 1 Pool 2 Pool 3

29

2013 IBM Corporation

Logical Building Blocks

The Flex System V7000 uses basic storage units called managed disks and collects them into one or more storage pools These storage pools then provide the physical capacity to create volumes for use by hosts

Volumes

Storage Pool

Managed Disks (MDisks)

30

2013 IBM Corporation

Managed Disks

The basic unit of storage in the Flex System V7000 is the managed disk or MDisk for short A managed disk must be protected by RAID to prevent loss of the entire storage pool In Flex System V7000 we can have two different types: Internal Array MDisk The internal RAID implementation inside the system takes drives and builds a RAID array with protection against drive failures External SAN attached MDisk An external storage system provides the RAID function and presents a Logical Unit (LU) to the Flex System V7000

31

2013 IBM Corporation

Storage Pools
A storage pool is a collection of managed disks The primary property of a storage pool is the extent size which by default using the GUI is 256MB
This extent size is the smallest unit of allocation from the pool

When you add managed disks to a pool they should have similar performance characteristics:
Same RAID level Roughly the same number of drives per array Same drive type (SAS, NL SAS, SSD except if using Easy Tier) Similar performance characteristics for external storage system MDisks

This is because data from each volume will be spread across all MDisks in the pool, so the volume will perform approximately at the speed of the slowest MDisk in the pool
The exception to this rule is that if using Easy Tier you can have 2 different tiers of storage in the same pool but the MDisks within the tiers should still have the same performance characteristics

32

2013 IBM Corporation

Volumes
A1 A2 A3 A4 A5 A6 A7 A8 A9 B1 B2 B3 B4 B5 B6 B7 B8 C1 C2 C3 C4 C5 C6 C7 C8
Volumes Managed Disks Volume A

Volume B Thin Provisioned Volume C

A1 A5 A9 B4 B8

A2 A6 B1 B5 C1

A3 A7 B2 B6

A4 A8 B3 B7

When the managed disks are added into storage pools, the managed disks are split into chunks of storage known as extents The size of these extents is a property of the storage pool Default is 256MB Whenever you create a new volume you must pick a single storage pool to provide the physical capacity By default the created volume will stripe all of its data across all the managed disks in the storage pool as shown on the left diagram Thin provisioned volumes consume extents only as actual data is written to disk

Storage Pool
33 2013 IBM Corporation

IBM Flex System V7000 Technical Overview

Array Configuration

34

2013 IBM Corporation

Internal Storage RAID

The Flex System V7000 system will contain a number of drive objects, but these drives cannot be directly added to storage pools The drives need to be included in a Redundant Array of Independent Disks (RAID) to provide protection against the failure of individual drives

35

2013 IBM Corporation

35

Supported RAID levels

Flex System V7000 will support the following RAID levels:

RAID 0 (striping, no redundancy) RAID 1 (mirroring between 2 drives) RAID 5 (striping, can survive one drive fault) RAID 6 (striping, can survive two drive faults) RAID 10 (RAID 0 on top of RAID 1)

More details about the what the RAID levels actually mean is available at URL below:
http://en.wikipedia.org/wiki/RAID

36

2013 IBM Corporation

36

Creating a RAID Array

In Flex System V7000 a RAID Array is created at the point that you add the array into the storage pool When the array is created a managed disk also called a MDisk is created the full useable capacity of the array Wizards and Presets are available to suggest configurations to users based on the hardware attached to the system If the MDisk built on internal storage is ever removed from the pool, then the array and the associated managed disk is unconfigured Drives go back to candidate status to be used to create another array

37

2013 IBM Corporation

37

One Click RAID Setup

A one click GUI setup exists to configure all drives into RAID arrays
Solid state drives will be configured as RAID 1 with 2 members and 1 spare SAS drives will be configured as RAID 5 with 8 members and 1 spare Near line SAS drives will be configured as RAID 6 with 12 members and 1 spare

All members of an array will have the same drive class

All SAS or all SSD or all NL SAS Same RPM Same capacity

One storage pool will be created for each drive class

38

2013 IBM Corporation

38

Manual RAID Array Creation (1): Drive Class

Internal Storage configuration

Fully automatic configuration of the whole system Semi-automatic configuration of a subset of drives Same type, RPM and capacity

39

2013 IBM Corporation

39

Array Creation (2): Preset

All RAID configuration in the GUI is done via presets
Collection of RAID properties which have been preselected based on best practice guidelines

Wizard shows what it will create at the bottom of the panel

40

40

2013 IBM Corporation

Array Creation (3): Width, Optimization, Spares

Decide whether to configure spares

Choose the optimization policy

Modify the number of drives to configure into array(s)

41

2013 IBM Corporation

41

Spare Drive Design

Spare drives are global spares

This means that any spare which is at least as big as the drive which is being replaced can be used in an array
Thus a SSD array with no SSD spare available would use a HDD spare

The spare system will prefer best possible match based on the following parameters:
Technology type
SAS vs SSD vs. NL-SAS

Speed/RPM Capacity

Each array has a spare goal property

Indicates how many spares must be available to protect this array If the number of spares drops below the goal then alert sent to notify admin

42

2013 IBM Corporation

42

IBM Flex System V7000 Technical Overview

Advanced Function

43

2013 IBM Corporation

Flex System V7000 Advanced Functions

Data Migration Included
Move data between storage pools an/or storage tiers non-disruptively Migrate to new disk hardware when old hardware comes off lease

Thin Provisioning Included

Allows for storage optimization by consuming real capacity only when data is written

Volume Mirroring Included

Mirror data between storage pools built from internal or external disk enclosures Add mirror to existing volume as thin-provisioned to reclaim capacity and/or migrate to another storage pool

Easy Tier Included

Allows for management of hotspots automatically by migrating extents from spinning drives to solid state drives as needed for higher performance

FlashCopy Included
Allows use of FlashCopy function with internal Flex System V7000 volumes as well as external storage under Flex System V7000 management FlashCopy Manager optionally available to help automate function

44

2013 IBM Corporation

Flex System V7000 Advanced Functions

Metro and/or Global Mirror Optional
Allows you to replicate data synchronously or asynchronously between up to four Flex System V7000 systems and/or Storwize V7000/SVC clusters

Compression Optional
Allows for in-band compression of data to minimize use of physical capacity

External Virtualization Optional

Allows you to bring external fibre channel disk systems under Flex System V7000 control providing access to all the functions of the above features

Questions?
45 2013 IBM Corporation

Flex System V7000: Simplified Experience

Advanced Storage Management Time to Value with IBM PureSystems Automated deployment When placed into the Flex System Enterprise Chassis, Flex System V7000 is immediately recognized and added to IBM Flex System Manager and Chassis Map Integrated event monitoring Seamless visibility to events and status Intelligent storage provisioning Integrated storage provisioning for virtual machine creation, deployment and cloning Automated firmware update Update Manager provides firmware level compliance monitoring and management
46 2013 IBM Corporation IBM Flex V7000 Storage System L&L

47

2013 IBM Corporation

Trademarks
The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.
Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market. Those trademarks followed by are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400, e business(logo), DBE, ESCO, eServer, FICON, IBM, IBM (logo), iSeries, MVS, OS/390, pSeries, RS/6000, S/30, VM/ESA, VSE/ESA, WebSphere, xSeries, z/OS, zSeries, z/VM, System i, System i5, System p, System p5, System x, System z, System z9, BladeCenter

The following are trademarks or registered trademarks of other companies.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.
* All other products may be trademarks or registered trademarks of their respective companies. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

48

2013 IBM Corporation