Session: 2331

Storage Subsystem Tech FAQ

Deep Dive
James Wright
TME Storage Product Management

Confidentiality Notice
The information in this presentation is confidential and proprietary to NetApp and may not be disclosed
without the written permission of NetApp.
The information is intended to outline our general product direction. It is intended for information purposes
only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing decisions. NetApp makes no warranties,
expressed or implied, on future functionality and timeline. The development, release, and timing of any
features or functionality described for NetApps products remains at the sole discretion of NetApp. NetApp's
strategy and possible future developments, products and or platforms directions and functionality are all
subject to change without notice. NetApp has no obligation to pursue any course of business outlined in
this document or any related presentation, or to develop or release any functionality mentioned therein.

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

The NetApp Data Fabric Ecosystem

OnCommand
Management Suite
NetApp Private
Storage
E-/EF-Series

AltaVault

FAS | AFF

StorageGRID
Webscale

FlexArray
and FLI
Non-NetApp Storage
EMC, Hitachi, Others
3

2015 NetApp, Inc. All rights reserved. NetApp Proprietary Internal Use Only

FlexPod

1. Storage Subsystem Technical FAQ

Origins

Agenda

2. Best Practice Vs. Supported

Configurations?
3. Raid Layout, Does Balance Matter?
4. Stack Transitions, How Many And
Why?
5. Additional Storage Tips

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Subsystem Storage Technical FAQ

Origins

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Storage Subsystem Technical FAQOrigins

History
Originated about 8 years ago by TMEs supporting FAS storage hardware
Compilation of best practices and supported configuration information
Initiated as a result of need to respond to common inquiries from youNetApp System Engineers,
Professional service reps, Partners and Customers

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Storage Subsystem Technical FAQOrigins

History continued
Facts gathered from internal and external sources throughout the years and vetted independently by
engineering and support
Updated more frequently than complimentary collateral TR-3838 Storage Subsystem Configuration Guide
Structured in informal Q&A style, which helps you find answers quickly to common concerns

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Best Practices vs. Supported Configurations

What Are They and What They Mean To You

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Best Practice Recommendation

Websters defines Best Practice as:
A procedure shown by research and experience to produce optimal results and is
established or proposed as a standard suitable for widespread adoption.

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Best Practice Recommendation

NetApp leverages a best-practice approach in the design of default configurations and
solutions
Architected with these factors in mind

10

Optimized for performance

Increased resilience and uptime
Solutions well designed and tested, validated by myriad of methods
Simplicity, ease of use baked in

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Supported Configuration
Definition

A solution in which the arranged components have been validated by engineering and
support to interoperate functionally, though not necessarily at optimum performance or
resilience
Or as Websters states for supportTo keep something going
Its a solution whose purpose is to just work, nothing more

11

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Best Practice Recommendation Versus Supported Configuration

12

Solution features

Best-practice recommendation

Supported configuration

Performance based

Yes

Partially

Resilient

Yes

Partially

Gold standard

Yes

No

Functionally tested to work

Yes

Partially

Simple to deploy

Yes

Partially

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Best Practice Versus Supported ConfigurationWhat Does It

Mean?
Leverage Best-Practice recommendations when available through use of Defaults to get
Best-in-breed performance
Supported configurations will yield results but, again, not optimized

13

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

RAID Layout
What Is RG Balance and Does It Really Matter?

14

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Raid LayoutOverview
RAID group (RG)Arrangement of whole or partitioned SSD or HDDs that share the same
parity drives for data protection (also known as a parity group)
Includes data (D) and parity (P) drives

AggregateCollection of RAID groups (RGs) within the same physical member set
Data (D)Drive or partition which holds user data
Parity (P)Drive or partition which is leveraged to reconstruct user data in the event of
disk failure in a RG
Spare (S)Replacement drive or partition available in the event of disk failure, parity
information transferred to this drive during reconstruction

15

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Raid LayoutOverview Continued

RAID 4Data protection scheme with a single parity drive in a RG
RAID DP (DP)Data protection scheme with two parity drives in RG equivalent to RAID-6

16

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

RAID Layout BalanceWhat Is It?

RAID group layoutDistribution of RG in an aggregate
BalancedDistributed approach, where configurations are provisioned with equally (or nearly equal) sized
spindle count RGs within an aggregate
ImbalancedDistributed approach, where configurations are provisioned with RGs that are not of equal size
spindle count within an aggregate

17

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

RAID BalanceWhat Is It?

One approach is to have a balanced layout as shown below
Note: Best-practice recommendation for SAS drives would deploy a balanced layout with (2) RGs of 16
default RG size, instead of (4) RGs of 8 to maximize the write throughput (with additional drives) and reduce
the parity tax as youd have 4 drives for parity with the larger RG instead of 8 for the smaller RG.
Aggregate

18

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

RAID BalanceWhat Is It?

An imbalanced layout is shown below
Note: NetApp Data ONTAP is resilient enough to serve data well in a imbalanced RG layout. However, we
suggest limiting the difference in drive count to half the size of the other RGs, if possible. The larger the
number of RGs in the aggregate, the smaller the impact will be in performance.
Aggregate

19

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

RAID Default RG size

Default and Maximum
Use Defaults as Best Practice
Data ONTAP Default & Maximum RAID Group

Size by Drive Type

Drive Type
SSD
SAS/FC
SATA/NL-SAS
6TB NL-SAS/MSATA

20

RAID Type

Default RAID Group

Size

Maximum RAID
Group Size

RAID-DP (default)
RAID 4
RAID-DP (default)
RAID 4
RAID-DP (default)
RAID 4
RAID-DP (default)

23 (21+2)
8 (7+1)
16 (14+2)
8 (7+1)
14 (12+2)
7 (6+1)
14(12+2)

28 (26+2)
14 (13+1)
28 (26+2)
14 (13+1)
20 (18+2)
7 (6+1)
14 (12+2)

RAID 4

7 (6+1)

7 (6+1)

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

RAID LayoutWhat Does It Mean?

Choose balanced layout where possible, using default RG sizes
Performance optimized when taking this Best Practice approach

Imbalanced layout functions sufficiently well, however one should maintain threshold of
at least half the size of the other member RGs to avoid performance issues

21

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack Transitions
What Is It and How Much Is Too Much?

22

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack TransitionOverview
SASSerial-attached SCSI, method that NetApp storage leverages to transport data
SAS HBA Host Bus Adapter component that initiates connection between the storage
shelves and the controller
Embedded unit on Platform controller board or installed on PCIe Card

SAS CablingOptical wire or copper cable which physically attaches shelves and
controllers to one another

23

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack TransitionOverview continue

IOM (I/O Module) Component in rear of shelf that is cabled to a Controllers SAS HBA or
other shelves in the stack enables shelf communication.
IOM3Legacy, 3Gb/s SAS storage technology
IOM6 Current, 6Gb/s SAS storage technology

StackA group of SAS shelves attached via copper cable or optical wiring to a controllers
SAS HBA.
*Current maximum depth is subject to platform support and spindle type (Entry platforms will have lower to
equal limits compared to mid and high end controllers)

24

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack TransitionWhat Is It?

Stack TransitionOccurs when IOMs within a stack are heterogeneous, where IOM3 are
mixed with IOM6 or vice versa. Each shift between the different types/speeds is
considered a transition.
Storage Link Process:
After startup of shelves and controller (in order specified) stack is probed by storage HBA
HBA Identifies speed capability of each shelf in stack based on I/O Module type.
IOM3 supports 3Gb/s
IOM6 supports 6Gb/s

25

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack TransitionWhat Is It?

SAS Storage Link Process - continue
Controller HBA determines if there are any changes in IOM technologies present in the
stack.
HBA will establish all connections at the lowest common speed

26

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack Transition What is it???

Single Transition Example: FAS8080: shelf #1 DS4243(IOM3) + shelf #2 DS2246 (IOM6)
1. Stack is probed by Controller HBA
2. HBA identifies IOM3 on shelf#1,
supports 3Gb/s

FAS8080

3. HBA identifies IOM6 on shelf#2,

supports 6Gb/s
4. HBA calculates single transition
shift is present in stack: IOM3
IOM6
5. HBA Establishes all
connections at lowest common
speed of 3Gb/s

27

1 2

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Shelf#1
DS4243
IOM3
3 Gb/s

Shelf#2
DS2246
IOM6
6 Gb/s

Stack Transition What is it???

Two Transition Example: FAS8080: shelf #1 DS2246 (IOM6) + shelf #2 DS4243 (IOM3)
shelf #3 DS2246 (IOM6)
1. Controller HBA probes stack

FAS8080
1

2. HBA identifies IOM6 on

shelf#1, supports 6Gb/s
3. HBA identifies IOM3 on
shelf #2, supports 3Gb/s

Shelf#1
DS2246
IOM6
6 Gig/s

4. HBA identifies IOM6 on

shelf#3, supports 6Gb/s
5. HBA calculates two IOM shift
stack transitions present: IOM6
IOM3 IOM6

6. HBA Establishes all connections

at lowest common speed of 3Gb/s
1

28

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

5 6

Shelf#2
DS4243
IOM3
3 Gig/s
Shelf#3
DS2246
IOM6
6 Gig/s

Stack Transition What is it???

- Internal Shelf Controller Example FAS2552: shelf #0 internal DS2246 (IOM6) shelf #1
DS4243 (IOM3) shelf #2 DS2246 (IOM6): two transitions
Shelf#0
FAS2552
IOM6
6 Gb/s

1. Controller HBA probes stack

1

2 5 6

2. HBA identifies IOM6 on

shelf#0, support 6Gb/s
3. HBA identifies IOM3 on
shelf#1, supports 3Gb/s
4. HBA identifies IOM6 on
shelf#2, support 6Gb/s
5. HBA calculates two
IOM shift stack transitions
present: IOM6 IOM3
IOM6

Shelf#1
DS4243 IOM3
1 3

3 Gb/s

4 5

6. HBA Establishes all

connections at lowest common
speed of 3Gb/s
29

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Shelf#2
DS2246
IOM6
6 Gb/s

Stack TransitionWhy?
Best-practice recommendationIs and has been to have no more than a single transition
from IOM3 IOM6 in a stack
Ensures optimal performance in a storage stack
Provides the best resiliency possible in mixed IOM3 and IOM6 scenarios

30

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack TransitionWhats new?

Policy shift in current environments
Multiple IOM3IOM6 transition configurations now a supported configuration
Best practice recommendation remains to limit stack transitions to one however.
Change due to a review in ASUP data and extensive engineering testing showing minimal
risk in downtime events or serious performance degradation for multi-transition
environments
Two or more transitions allowed up to the 10-stack depth limit*

31

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stack TransitionWhat It Means

Best practice for IOM3 IOM6 is still a single transition for performance and
resilience
Supported configuration for IOM3 IOM6 environments beyond 2 transitions, though
performance not optimal as all permutations are not continually validated.

32

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Storage Tips
-

33

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Tips
Use flash with Flash Cache, Flash Pool where possible as price points for SSD vs.
Performant SAS hard disk drives are converging
cDOT enables flexibility with Flash
Provides greater performance and resilience

Disk Maintenance Center/Pitstop

MC/Pitstop avails early HDD/SDD failure mechanism for your whole drive environments, enabling greater
uptime for your systems

Disk Qualification Package (DQP)

Netapp recommends downloading the latest disk qualification package firmware bundles from the Support
site periodically (bi-annually or quarterly) to enable new features in our latest released HDD/SSD.

34

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Key Takeaways
Understand why there are best practices and where they are applicable
Learn how following recommendations can improve your storage performance
Gain knowledge on supported storage configurations

35

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Related Sessions and Resources

Learn more about this topic

For additional information after the event visit:

36

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Tell Us What You Thought

Take an Insight survey on your mobile device
Completed surveys are entered into a drawing to
win an Apple Watch, NetApp gear, and more
Dont have the app? Download the Insight Mobile App
at the Apple Store and on Google Play.

37

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Stay Connected
Follow NetApp Insight on Facebook and Twitter
Speaker Name Here
https://www.facebook.com/Add your business Facebook address here
https://twitter.com/Add your business Twitter address here
Insight US

Insight EMEA

38

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Q&A
Storage Subsystem Technical FAQ

39

Insight 2015 NetApp, Inc. All rights reserved. NetApp Confidential Limited Use Only

Thank you

2015 NetApp, Inc. All rights reserved. No portions of this presentation may be reproduced without prior written consent of NetApp, Inc. Specifications are subject
to change without notice. NetApp and the NetApp logo are registered trademarks of NetApp, Inc. in the United States and/or other countries. All other brands or
products are trademarks or registered trademarks of their respective holders and should be treated as such.