Professional Documents
Culture Documents
7400
White Paper
Printed in the United States of America Copyright 2012 Edison Group, Inc. New York. Edison Group offers no warranty either expressed or implied on the information contained herein and shall be held harmless for errors resulting from its use. All products are trademarks of their respective owners. First Publication: September 2012 Produced by: Chris M Evans, Sr. Analyst; John Nicholson, Sr. Analyst; Barry Cohen, Editor-inChief; Manny Frishberg, Editor
Table of Contents
Overview _____________________________________________________________ 3
Thin Provisioning Overview ________________________________________________ 3 Thin Provisioning Drawbacks_______________________________________________ 4
NetApp ___________________________________________________________________ 8
Background _____________________________________________________________________ 8 HP 3PAR StoreServ ComparisonStart Thin ________________________________________ 8 HP 3PAR StoreServ ComparisonGet Thin _________________________________________ 9 HP 3PAR StoreServ ComparisonStay Thin _________________________________________ 9
Executive Summary
As the drive to "do more with less" becomes a mantra for many organizations, optimizing space utilization is a key goal of many IT departments. Storage continues to be one of the major cost components of today's infrastructure deployments. Thin technology, including thin provisioning, offers efficiency benefits that can significantly reduce both capital and operational costs. However implementations of thin technologies differ with the storage vendors. HP 3PAR StoreServ is seen as a leader in thin technology, with three key aims: 1. Start Thinensure thin provisioned storage occurs with minimum overhead. 2. Get Thinensure data moved to HP 3PAR StoreServ remains thin on migration. 3. Stay Thinensure data is kept at optimal efficiency over time. To validate this statement, a literature review, extensive customer interviews, and two tests were performed: 1. Zero-Page-Reclaim Performancevalidation of the ability to reclaim freed resources as part of normal operations. 2. Large Pre-allocationtest of the ability to create new storage volumes with minimal overhead. Overall, HP 3PAR StoreServ was the best performer in achieving the goals of "start thin," "get thin," and "stay thin."
Page 1
Introduction
Objective
This report looks at thin provisioning technology from the major storage vendors in today's marketplace. It compares the thin implementations from seven storage array platforms, including Hewlett Packard's 3PAR storage arrays. In particular this white paper highlights three important differentiating aspects of HP 3PAR StoreServ's thin technology: 1. The ability to "start thin"provisioned storage is thin at deployment time. 2. Getting thinthe ability to move data from thick to thin. 3. Staying thinmaintaining thin LUNs.
Audience
Decision makers in organizations that are considering implementing a thin technology strategy will find this paper provides high level information on vendor offerings. Technical professionals looking to understand more about the implementation of vendor thin technology solutions will also find the content of this paper useful.
Terminology
This white paper makes reference to the following common terminology:
"Thick" LUNa storage volume presented from an array in which all of the space representing the logical size of the LUN (logical unit number) is reserved on the array for exclusive use by that volume. "Thin" LUNa storage volume presented from an array that is not tied to any physical storage allocation and in which only the physically written space is consumed on the array. Thin technologiesa suite of features, including thin provisioning, that optimize the use of a storage array.
Page 2
Overview
In recent years storage has become one of the major cost components within the data center. Although the price of storage continues to fall, the rate of data growth in many organizations continues to rise steeply, resulting in increasing costs for managing the storage systems. Every year there is a requirement to "do more with less," using storage more efficiently without increasing the operational budget. There are a number of key initiatives being undertaken by organizations in order to reduce their storage consumption. These relate directly to the use of thin technology.
Reducing WasteStorage utilization never reaches 100 percent of the physical space provisioned from an array, as each level of configurationfrom the array to the hostintroduces some inefficiency. Reducing waste increases utilization and allows the deferral of additional capital expenditure. Reducing OverheadDeploying storage isnt a quick task; from purchase order to deployment on the data center floor, the process can take months to achieve. Storage administrators usually keep storage in reserve in order to manage the purchase process. FlexibilityEnd users want the minimum disruption to their applications and as a result, many over-order storage resources, in many cases up to 36 months ahead of when the space is actually needed. Ideally, end users should be able to lay out their storage needs based on growth plans and then allocate that storage on-demand. Improving Cost EfficiencyStorage Tiering (placing data on the most cost-effective media for the I/O profile required) is a key technology in reducing storage costs. Dynamic tiering can be used to automate the process of data placement, based on the use of storage pools for LUN creation. Thin provisioned LUNs directly aid the deployment of a tiered storage model. A thin LUN is built from blocks of physical disk capacity from within a pool of storage with metadata to associate the logical LUN to the physical space. The physical blocks can therefore be taken from multiple pools, where each pool represents a different tier.
Page 3
In traditional "thick" storage deployments, physical space on disk is immediately reserved for the entire size of a volume (or "LUN") at creation time, regardless of how much space will subsequently be used by the host. In thin storage deployments, no space is reserved in advance for the LUN. As the host writes data to the LUN, physical space is assigned on-demand from the array, usually in blocks that vary from 4KB to 42MB, according to the vendor. Thin provisioned LUNs are therefore much more efficient and more closely track the actual space in use on the host. For many reasons, storage utilization on hosts never reaches 100 percent utilization. However with "thick" LUNs, physical space is reserved out on an array for the entire size of a volume. Thin provisioned deployments can take advantage of all physical storage available by creating more logical storage capacity than is physically available. So called "over-provisioning" enables the utilization of physical space to be pushed to levels higher than can be achieved in normal deployments.
Zero-Page-Reclaim (ZPR)A storage array identifies an entire block of storage consisting of binary zeros, the block will be assumed to be unused and is released back to the free pool. The ability to find unused blocks depends on a number of factors, including the file system and array block-size and the level of file fragmentation. Smaller array block-sizes are better for ZPR operations. SCSI UNMAPThe UNMAP command is a low-level I/O operation that can be used by the host to signal to the storage array that a block of storage is no longer in use and can be released to the free pool. Unfortunately very few operating systems currently support this feature.
Page 4
RAIDHP 3PAR StoreServ arrays offer a unique RAID technology that provides chassis high availability, and divides physical disks into "chunklets" of either 256MB or 1GB in size. Chunklets are combined to form Logical Volumes (LVs) and Common Provisioning Groups (CPGs) from which Virtual Volumes are created. Thin Provisioning Virtual Volumes use a block size increment of 16KB, which is the minimum reclaimable unit of storage within the array. Hardware ASICNow at Generation Four, HP 3PAR StoreServ uses dedicated custom ASIC (Application Specific Integrated Circuit) processors to perform the identification and recovery of unused resources that can be reclaimed from thin provisioned virtual volumes. An ASIC enables processor-intensive tasks to be offloaded to dedicated hardware, removing the performance impact of features such as space reclamation from the array and ensures consistent host I/O response times. The HP 3PAR StoreServ ASIC provides a range of functions, including inline ZPR. Thin PersistenceAn operating system task that identifies and recovers freed resources. Thin ConversionPerforms the migration of thick to thin volumes through a process of inline zero detection. As data is copied to the array, zeroed blocks of data are identified and logically mapped rather than physically written to disk. Thin Copy ReclamationPerforms space recovery on volume copies within the array. Thin Reclamation APIHP 3PAR StoreServ developed the Thin Reclamation API in partnership with Symantec. This feature allows the file system to signal when freed resources can be released on the array. It is supported by Symantec Veritas Storage Foundation from Version 5 onwards. Virtualization SupportHP 3PAR StoreServ supports the VMware VAAI API, including the "block zeroing" command. ManagementHP 3PAR StoreServ arrays provide alerts for specific thin provisioning space issues. Alerts are issued based on pre-defined thresholds and enable efficient monitoring of capacity in thin environments.
Page 5
Competitive Analysis
EMC VMAX
Symmetrix VMAX, EMC's flagship enterprise storage platform, is the first enterpriseclass storage product to move away from custom hardware design. It uses commoditybased Intel processors with customized hardware managing the interconnect between storage modules. The VMAX operating systemEnginuityis an evolution of the code developed for the first Symmetrix ICDA (Integrated Cache Disk Arrays) in 1991, and it still retains many of the original architectural design features and constraints. The discussion of VMAX in this section covers the latest 10K, 20K and 40K models.
Background
Thin provisioning in VMAX is implemented as a feature called Virtual Provisioning EMC's brand name for their thin provisioning technology. Thin provisioned LUNs are known as thin devices and take physical storage from thin pools. A thin pool is created using standard "thick" LUNs (termed data devices), which are subdivided into allocation units called thin device extents. Thin pools must use the same emulation and RAID protection type and EMC recommend building them from disks of the same rotational speed and data device size. A thin extent is 12 tracks or 768KB in size and represents both the initial minimum assigned to all thin devices when they are bound to a thin pool and also the lowest increment of granularity when the capacity of a thin device is extended. HP 3PAR StoreServ thin technology uses the much smaller increment of 16KB, which results in less wastage, particularly with fragmented and thin-hostile file systems. Thin devices are effectively cache-based objects that simply reference the underlying physical pool of standard LUNs in array. These LUNs in turn, map to physical disks. A single VMAX system supports up to 512 pools and 64,000 thin devices.
With HP 3PAR StoreServ thin technology, physical disks are simply assigned to a pool from which either thin or thick LUNs can be provisioned. When VMAX thin devices are bound to a thin storage pool, a minimum allocation of one thin extent (768KB) is reserved. As thin devices are effectively cache objects, each device consumed an additional 148KB of cache, plus 8KB per 1GB based on the size of the thin device. With HP 3PAR StoreServ thin technology, no initial space reservations are made.
VMAX Restrictions
EMC recommends a utilization level of between 60-80 percent per thin pool in order to prevent "out of space" issues. With multiple pools (which are required for different RAID data protection types) this can result in significant waste. HP 3PAR StoreServ thin technology does not require separate pools for multiple protection types. When using Synchronous SRDF with VMAX, only one active write is permitted per thin device. Where thin devices are created into meta-devices, this can result in a performance impact. There is also a limit of eight read-requests-per-paths for each thin device, which can result in slow performance with high read miss rates.
Edison: HP Thin Technologies Comparison Page 7
NetApp
NetApp storage appliances were originally developed to deliver network-attached storage using either the CIFS or NFS protocols. Over time, NetApp have developed their platform to cater for block storage, using either iSCSI or Fibre Channel. The current versions of NetApp filers can be configured in either 7-mode or cluster-mode and represent two distinct product lines based on the original Data ONTAP operating system, and the codebase from the acquisition of Spinnaker, Inc., respectively.
Background
NetApp filers implement block-based storage within Data ONTAP by emulating LUNs within volumes known as FlexVols. FlexVols are then created on aggregates (pools of physical storage) and physical disk RAID groups. The underlying architecture uses a data layout called WAFL (Write Anywhere File Layout) that operates a "write-new" policy for both new data and updates; no block or file data is ever updated in place. WAFL uses a page size of 4KB, storing updates in non-volatile RAM before writing an entire "stripe" of data to disk. In this way, writes are optimized on commit-to-disk using a RAID-4 physical disk configuration. NetApp LUNs are emulated through files on volumes; therefore, both block and file data can be mixed within the same storage pool. LUN creation is a simple process to achieve, however the use of block-based LUNs involves significant complexity.
Page 8
Hitachi VSP
The VSP is Hitachi's current enterprise-level storage array and is the evolution of previous Lightning and USP-V models. The VSP retains the use of custom ASIC technology, in which the management of storage processes is handled by Virtual Storage Directors connected to the back-end switch matrix. Custom ASIC usage has been a feature of all of the Hitachi storage platforms; however it isn't used directly in the thin provisioning approach or in managing the efficiency of thin provisioned storage.
Background
VSP thin provisioning technology is known as HDP--Hitachi Dynamic Provisioning. HDP thin LUNs (called LDEVs or logical devices) are created from a HDP pool that comprises standard LDEV devices. In turn, LDEVs are created from RAID groups, built from up to 16 disks in one of seven RAID-5 or RAID-6 variations. At the physical level, data is written in tracks of 256KB per physical disk, which results in a standard logical
Edison: HP Thin Technologies Comparison Page 9
page size of 42MB, in order to accommodate all possible RAID levels. This means initial volume allocations and volume expansion of thin LUNs is in 42MB-page increments that can result in inefficient use of space with small file block size and thin-unfriendly file systems. With HP 3PAR StoreServ thin technology, space allocations are made in 16KB increments, which results in much less wastage in thin-unfriendly environments.
Page 10
EMC VNX
EMC's VNX platform is an evolution of the previous CLARiiON and Celerra products (serving block and file protocols respectively). The two platforms were brought together and marketed as a single platform, using one management tool, called Unisphere. Blockbased storage LUNs are presented from the base hardware unit, with file access implemented on x-blade modules. Thin provisioning technology is implemented using the Virtual Provisioning (VP) feature. VP extends the capabilities of LUN configuration to include both thick and thin LUNs on the same disk pool. VP disk pools can be comprised of large numbers of disks (greater than the standard disk pool which is limited to 16 devices), but still configures disks in RAID groups for resiliency.
Performance Considerations
EMC highlight that Virtual Provisioning Thin LUNs do provide more flexibility, but offer lower performance than traditional thick LUNs and so recommend their use only for applications requiring "moderate" performance. HP 3PAR StoreServ thin technology has no performance restrictions.
IBM XIV
The IBM XIV storage array platform was acquired from an Israeli startup, founded by the inventor of the EMC Symmetrix, Moshe Yanai. The platform takes a radical departure from traditional arrays and uses only high-capacity SATA or SAS hard drives, although the configuration has recently been expanded to accelerate I/O using an SSD cache layer. XIV is now at the third generation of hardware, utilizing either 2TB or 3TB drives, with 6TB of SSD cache. Each array is comprised of between six and 15 server nodes, which hold 12 hard drives each, resulting in a maximum configuration of 180 drives. Each node subdivides disks into 1MB chunks, which are then distributed across all disks as a single large pool of mirrored data. XIV uses the terms "soft size" and "hard size" to refer to the logical and physical size of a LUN respectively. These terms also apply equally to pools that can be allocated physical capacity. It is possible for a pool to deplete hard (physical capacity) and lock access to a volume, despite there being free physical space in other pools. The overall capacity of an XIV array is referred to as the "system hard size." A "system soft size"the degree of over-provisioning permitted at the array levelis also defined, but can't be modified by the system administrator. This value has to be set by an IBM engineer and requires the customer to indemnify IBM against any issues that occur as a result of the change.
and the placement of the XIV "inline" with the host and the original volume. This task requires an outage to achieve. With HP 3PAR StoreServ thin technology, data can be migrated into the array via the host, identifying zero-pages in line and without requiring a host outage.
Dell Compellent
Compellent Technologies, Inc. was founded in 2002 and subsequently acquired by Dell in 2011, from which time it was marketed under the Dell Compellent brand name. Based on commodity components, Compellents unique offering is called Data Progression, an automated tiered storage feature enabling migration of data between storage tiers at the block level. Compellents thin provisioning technology is known as Dynamic Capacity. A thin LUN is allocated using 2MB blocks that can be assigned from any of the physical capacity within the array. Although block size defaults to 2MB, the administrator may override this value to either 512KB or 4MB.
Page 13
no minimum allocation. Dell Compellent can use 512KB or 4MB block sizes; however, Dell recommends not mixing block sizes in a single system, as this can result in waste of physical space.
Page 14
Page 15
Page 16
Test Results
Test 1Zero-Page-Reclaim Performance
The data in these tables represents the performance for each array capable of zero-pagereclaim within the test parameters. NetApp FAS was not included in this test, as the system has no native support for zero-page-reclaim. Data for IBM XIV was not included, because that system performs reclaim over a very long period of time that was outside the test parameters.1
Platform EMC VMAX Dell Compellent EMC VNX Hitachi VSP HP 3PAR
Table 1 - Test 1 - IOPS Performance during ZPR
According to an IBM Redbook, IBM XIV Storage System: Copy Services and Migration, it, "could take up to three weeks for used space value to decrease This is because recovery of empty space runs as a background task."(Page 264). Not only is the time required for ZPR outside the parameters of our research, enabling over-provisioning is, "not within the scope of the administrator role."(Page 30) This suggests that an IBM engineer must perform an overprovisioned configuration.
1
Page 17
Platform EMC VNX EMC VMAX Dell Compellent Hitachi VSP HP 3PAR
Table 1 - Test 1 - I/O Latency during ZPR
Page 18
The results of this test show that ZPR activity has an impact on both the latency and throughput of each platform except HP 3PAR StoreServ. The greatest effect was seen on EMC VMAX performance and the latency increase with Dell Compellent. Edison was able to determine that performance impact shown on EMC VMAX was because the platform needs to read each thin device extent into cache in order to perform ZPR processing. This cache load clearly has a direct impact on array performance. Edison was unable to diagnose the causes of the increase latency on the Dell Compellent system. The HP 3PAR StoreServ array has dedicated ASICs to handle the ZPR workload without impacting on delivering I/O to hosts.
Page 19
Page 20
The results of this test show EMC VNX and IBM XIV performed poorly in preallocations. EMC VNX reserves a minimum of 3GB per LUN; IBM XIV reserves a minimum of 17GB per LUN. The other platforms performed well. Clearly when systems have large volumes of LUNs, the minimum reserve can have a detrimental impact on the aims of "starting thin," resulting in large amounts of unusable storage.
Page 21
Best Practices
The testing and research in this white paper highlights a number of best practice considerations: 1. Implement Zero-Page-ReclaimThis feature should be used to ensure LUNs stay thin, however on most platforms (except 3PAR because of its custom ASIC and XIV
Page 22
because it runs so slowly) needs to be scheduled out of normal production hours to minimize performance impact. 2. Be aware of minimum LUN sizesWhen setting a standard for thin provisioned LUNs, ensure that the minimum configured LUN size is not likely to waste capacity.
Page 23
TR-3505 - NetApp Deduplication for FAS and V-Series Deployment and Implementation Guide TR-3563 NetApp Thin Provisioning Increases Storage Utilization with On Demand Allocation TR-3483 Thin Provisioning in a NetApp SAN or IP SAN Enterprise Environment GC27-3913-03 - IBM XIV Storage System Planning Guide GC27-3912-02 IBM XIV Storage System Product Overview 4AA3-3516ENW HP 3PAR Architecture 300-006-718 Best Practices for Fast, Simple Capacity Application with EMC Symmetrix H2222.3 EMC VNX Virtual Provisioning White Paper 300-011-798 EMC VNX Series Release 7.0 VNX System Operations Dell Compellent Data Progression Data Sheet
Page 24
Arrays
Hitachi VSP NetApp FAS3140, running Data ONTAP 8.0.2, RAID-DP across 28 drives. Dell CompellentRAID-5 across 72x 600GB SAS drives EMC VNX5700 running microcode 5.31, RAID-5 across 24x 300GB 15K SAS drives. HP 3PAR F400 InForm OS 3.1.1 (MU1) EMC VMAX-20K IBM XIV Gen2, 72x 1TB SATA drives
Servers
HP BL Blade Servers, 2x Intel X5650 CPU, 16GB RAM, HP Flex10 I/O Windows 2008R2 SP1 & CentOS 6.2 IOMeter v2006.07.27
4AA4-4079ENW
Page 25