managing the unimaginable: a practical approach to petabyte data storage randy cochran,...
TRANSCRIPT
Managing the Unimaginable:A Practical Approach to Petabyte Data Storage
Randy Cochran, Infrastructure Architect, IBM Corporation, [email protected] -1366 - Information on Demand Infrastructure
2
Data Storage is Getting Out-of-Hand
Are storage demands starting to overpowering you?
3
Most Research Firms Agree
“It is projected that just four years from now, the world’s information base will be doubling in size every 11 hours.” (“The toxic terabyte; How data-dumping threatens business efficiency”, Paul Coles, Tony Cox, Chris Mackey, and Simon Richardson, IBM Global Technology Services white paper, July 2006)
“Our two-year terabyte CAGR of 52% is 3ppt (percentage points) below rolling four quarter results of 55%.”("Enterprise Hardware: 2007-08 storage forecast & views from CIOs", Richard Farmer and Neal Austria, Merrill Lynch Industry Overview, 03 January 2007)
“With a 2006–2011 CAGR nearing 60%, there is no lack in demand for storage…”("Worldwide Disk Storage Systems 2007–2011 Forecast: Mature, But Still Growing and Changing", Research Report # IDC206662, Natalya Yezhkova, Electronics.ca Publications, May 2007)
“According to TheInfoPro…..the average installed capacity in Fortune 1000 organizations has jumped from 198 TB in early 2005 to 680 TB in October 2006. …..TIP found that capacity is doubling every 10 months.”
(InfoStor Magazine, Kevin Komiega, October 19, 2006)
4
What’s Driving Petabyte Level Storage?
The “Perfect Storm”General Increase
in demand
New digital data technologies
More regulatory requirements
Better protection from litigation
Disaster Recovery plans
Proliferation of Sophisticated applications
Declining storage media costs
A desire for greater storage
efficiency
Storage technical skills scarcity
A growing understanding of retained data’s business value
According to IDC, between 2006 and 2010 information added annually to the digital universe will increase more than six fold from 161 to 988 exabytes.
5
Just How Big is a Petabyte?
Terminology IEC Notation Value Bits Bytes
Bit bit - - - - 1 - -
Byte B - - - - 8 1
Kilobyte KB 10241=210 8,192 1,024
Megabyte MB 10242=220 8,388,608 1,048,576
Gigabyte GB 10243=230 8,589,934,592 1,073,741,824
Terabyte TB 10244=240 8,796,093,022,208 1,099,511,627,776
Petabyte PB 10245=250 9,007,199,254,740,990 1,125,899,906,842,620
Exabyte EB 10246=260 9,223,372,036,854,780,000 1,152,921,504,606,850,000
Zettabyte ZB 10247=270 9,444,732,965,739,290,000,000 1,180,591,620,717,410,000,000
Yottabyte YB 10248=280
9,444,732,965,739,290,000,000,000 1,180,591,620,717,410,000,000,000
Data Storage Size Relationships
Petabyte storage had been around for years – online Petabyte storage has not.
“Ninety-two percent of new information is stored on magnetic media, primarily hard disks.” “How Much Information 2003”, UC Berkeley's School of Information Management and Systems
6
How Big is That in Human Terms?
Terminology Value
Bit Two
Byte Eight
Kilobyte Thousand
Megabyte Million
Gigabyte Billion
Terabyte Trillion
Petabyte Quadrillion
Exabyte Quintillion
Zettabyte Sextillion
Yottabyte Septillion
A unit of storage capable of holding a single alpha-numeric character
Data Storage Size Relationships
A "bit" is short for "binary digit" and can hold only two states, 0 or 1
One page of a book having around a thousand characters on it
A medium resolution color photograpy taken by a digital camera
A gigabyte is equal to the contents of about 10 yards of books on a shelf
A Terabyte could hold 1,000 copies of the Encyclopedia Britannica
Approximately 100 times the printed collection of the Library of Congress
Estimated as 1/5th of all of words spoken since the beginning of history
Estimated size of data storage for all computers in the world by 2010
(Difficult to equate to a meaningful example)
According to Britannica.com the U.S. Library of Congress contains approximately 18 million books, 2.5 million recordings, 12 million photographs, 4.5 million maps, and more than 54 million manuscripts.
7
Why is Petabyte Storage a Challenge?
Areas Impacted by Petabyte Storage:
• Content and File Management• Application & Database Characteristics• Storage Management • Architectural Design Strategy• Performance and Capacity• SAN Fabric Design• Backup and Recovery Methods• Security System Complexity• Compliance with Regulatory Requirements• Operational Policies and Processes• Maintenance Requirements
8
Content and File Management
9
Management Starts With Data Classification
Data Classification Assumptions • Not all data is created equal• The business value of data changes over time• Performance can be improved by re-allocating data to an
optimized storage configuration• The value of most business data is not fixed; it is expected to
change over time
Understanding the business value of data is a crucial in designing an effective data management strategy
Which data has a greater value to the business - a client’s purchase
record, or a memo about last year’s phone system upgrade?
10
Data Classification Example
There are no universally accepted standard definitions for Tier Levels.
Data ClassificationPercent of Active Data
Storage Class
Disk Drive Characteristics Vendor Models
Relative Cost/GB
Typical Tier
Level Recovery
Requirement
Life Critical / Business Critical 10% - 15%
Enterprise Class
FC SCSI 15K or 10K RPM,
low to medium capacity
IBM DS8300EMC DMX1/2/3000HDS Tagmastore x45 Tier 1
Recovery in <15 minutes
Business Important 20% - 30% Mid-Range
FC SCSI 10K RPM, medium to
high capacity
IBM DS6800, DS4800EMC DMX800HDS 9970/80 x25 Tier 2
Recovery in < 4-hours
Business Standard 50% or more Mid-RangeSATA, FATA, or SAS
high capacity
IBM DS48000, DS4700 EMC CX300/500/700HDS 9530 Thunder x5 Tier 3
Recover in < 72-hours
Nearline / Reference 50% or more Low End Optical or Tape Device
IBM DR550 or TS3500EMC Centera, Hitachi WMS100, HP 7100ux
Optical Jukebox x3 Tier 4Recover as soon
as practical
Low Value / Archived Inactive data Archived Archived data
Iron Mountain,internal archiving
x1 Tier 5Recover when
required
11
Control Your File Content
Implement file agingSet data retention periodsEliminate low value data
• Clean out old backup files
• Eliminate outdated information
• Deploy de-duplication technology
• Reduce storage of low value data
• Locate and purge corrupt files
Crack down on unauthorized storage usagePeriodically review log files and archive or
delete obsolete information
12
Application and Database Characteristics
13
Know your applications needs• User expectations• Workload complexity• Read or write intensity• Sequential files usage• IOPS dependence• Stripe size optimization• Throughput requirements• Service prioritization• Growth expectations
Don’t allow databases to “lock up” vast amounts of storage
Know Your Application and Database Needs
14
Applications characteristics will drive storage decisions• Value to the business• Number of users• Usage patterns
Steady Bursty Cyclical Variable
• 7x24 or 9x5 access• Domestic or global access• Distributed or self-contained• High or low security data• Architectural constraints
Significant performance gains (or losses) can be achieved by matching requirements to storage characteristics
Applications Will Drive Storage Requirements
15
Storage Management
16
Large Storage Systems Must Be Managed
Information Lifecycle Management (ILM) Hierarchical Storage Management (HSM) Storage Resource Management (SRM) Storage Virtualization
"Enterprises can achieve better and more targeted utilization of resources by first establishing the value of their information assets and then using storage management software to execute the policies that define how resources are utilized."
Noemi Greyzdorf, research manager, Storage Software, IDC
17
Information Lifecycle Management
“(ILM is) the process of managing business data throughout its lifecycle from conception until disposition across different storage media, within the constraints of the business process.”
(courtesy of Veritas Corporation, Nov. 2004)
ILM is not a commercial product, but a complete set of products and processes for managing data from its initial inception to its final disposition.
18
Information Lifecycle Management
Information has business value• It’s value changes over time
• It ages at different rates
• It has a finite life-cycle
• As data ages its performance needs change
Some Information is subject to different security requirements, due to government regulatory or legal enforcements
Outdated information has different disposal criteria
A combination of processes and technologies that determine how information flows through a corporate environment
Encompasses management of information from its creation until it becomes obsolete and is destroyed
19
“Best Practices” for ILM Implementations
Know exactly where information is stored Be able to retrieve information quickly and efficiently Limit access to only those who need to view data Create policies for managing and maintaining data Do not destroy important documents Avoid keeping multiple copies of the same data Retain information only until it is no longer useful Destroy outdated files on a regular basis Document all processes and keep them up-to-date
20
Hierarchical Storage Management
“HSM is a policy-based data storage management system that automatically moves data between high-cost and low-cost storage media, without requiring the knowledge or involvement of the user.”
(courtesy of http://searchstorage.tedchtarget.com)
IBM has been involved in providing HSM solutions for over 30-years and offer a wide variety of products with automated data movement capabilities.
21
File Access Activity Over Time
AIX File Accesses Per Day
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
1,600,000
1,800,000
<= 1 day 1 day - 1-wk. 1-wk. - 1-mo 1-mo. - 2-mo. 2-mo. - 3-mo. 3-mo. - 6-mo. 6-mo. - 9-mo. 9-mo. - 1-yr > 1-yr
Number ofAccesses
Last Accessed
Expon. (Last Accessed)
75% 10% 15%
Files accessed within the last 2-months.
Files accessed between 2 and 6 months.
Files accessed beyond 6 months
22
Hierarchical Storage Management
HSM Concepts• Only 10%-15% of most data is actively accessed• The business value of data changes over time• Between 80% and 90% of all stored data is inactive• High performance storage (FC disks) are expensive• Lower performance media (tape, optical platters, and SATA
disk) are comparatively inexpensive
10% 20% 70% Archive
23
Hierarchical Storage Management
HSM Concepts (cont.)• Enterprise class storage is not required for all data• Policies can be set to establish the proper frequency for
transitioning aging data to less expensive media• HSM allows optimal utilization of expensive disk storage• Low cost, high density disks consume fewer resources• Overall storage system performance may improve
$$$$ $$$ $$ $
24
IBM Products with HSM Capabilities
General Parallel File System (GPFS)
IBM Content Manager for Multiplatforms
Tivoli Storage Manager HSM for Windows
Tivoli Storage Manager for Space Management (AIX)
SAN File System (SFS)
DFSMShsm (Mainframe)
High Performance Storage System (HPSS)
25
Storage Resource Management
“Storage Resource Management (SRM) is the process of optimizing the efficiency and speed with which the available drive space is utilized in a storage area network (SAN). Functions of an SRM program include data storage, data collection, data backup, data recovery, SAN performance analysis, storage virtualization, storage provisioning, forecasting of future needs, maintenance of activity logs, user authentication, protection from hackers and worms, and management of network expansion. An SRM solution may be offered as a stand-alone product, or as part of an integrated program suite.”
(Definition Courtesy of http://searchstorage.techtarget.com)
IBM’s primary tool for Storage Resource Management is their TotalStorage Productivity Center suite of tools for disk, data, fabric, and replication.
26
Storage Resource Management Functions
DeploymentManagement
§Asset Management§Change Management§Capacity Planning Service Level
Management
§ Policy Management§ Security Management§ Automation
§ Backup & Recovery § HSM Operations§ Point-in-Time Copies§ Disaster Recovery§ Data Migration§ Data Archiving
§ Event Management§ Performance Management§ Accounting Management§Quota Management
ComplianceManagement
OperationalManagement
27
Storage Virtualization
Virtualization
“The act of integrating one or more (back end) services or functions with additional (front end) functionality for the purpose of providing useful abstractions. Typically virtualization hides some of the back end complexity, or adds or integrates new functionality with existing back end services. Virtualization can be nested or applied to multiple layers of a system.”
(Definition Courtesy of http://www.snia.org/education/dictionary)
Virtualization allows most of the complexity of a storage infrastructure to be hidden from the user.
28
Virtualization Makes Storage One Large Pool
Virtualization Characteristics• Makes storage configuration details
invisible to the user• Improves overall manageability of
the system• Aggregates isolated storage
“islands” into a unified view• Facilitates greater flexibility and
scalability • Optimizes utilization of storage capacity• Provides the ability to move data on-the-fly• Improves storage subsystems flexibility• Allows rapid re-allocation of storage resources • Improves performance by providing another layer of caching• May provide additional functionality for the SAN
29
Architectural Design Strategy
30
Key Architectural Design Considerations
Resource Consumption Storage Economics RAID Allocation Performance Objectives Other Design Issues
The integrity of the architectural design will determine the overall performance, stability, economic efficiency, manageability and future scalability of the system.
31
Power Consumption vs. Storage Capacity
** National retail price of electricity per KwH from “Power, Cooling, Space Efficient Storage”, page 2, ESG white paper, Enterprise Strategy Group, July. 2007.
These disks all have very similar power consumption requirements, even though the largest one features 28 times the capacity of the smaller one.
In addition, each disk will require approximately 0.4-0.6 watts of electrical power to cool each BTU of heat produced.
Disk Type
Disk Capacity
in GB
Average Power
per Disk in Watts
# of Disks per
Petabyte
Total Power in
KW
Total Power in KW/hr.
**Cost per KW/hr. @
$.0874
Power Cost Per
Year
Cost Over a 5-yr Period
Power Consumption
Efficiency Index (Watts/GB)
BTUs per disk
BTU/hr per Petabyte
FC 36.7 9.9 27,248 270.0 20.25 $1.77 $15,505 $77,523 0.270 33.8 3,314,653,798
FC 73.4 9.3 13,624 126.8 9.51 $0.83 $7,283 $36,417 0.127 31.7 1,557,078,474
FC 146.8 10.8 6,812 73.7 5.53 $0.48 $4,232 $21,158 0.074 36.9 904,644,196
FC 300 13.8 3,333 46.0 3.45 $0.30 $2,642 $13,209 0.046 47.1 564,777,840
SATA 250 9.3 4,000 37.2 2.79 $0.24 $2,136 $10,680 0.037 31.7 456,667,200
SATA 320 9.3 3,125 29.1 2.18 $0.19 $1,669 $8,344 0.029 31.7 356,771,250
SATA 400 9.3 2,500 23.3 1.74 $0.15 $1,335 $6,675 0.023 31.7 285,417,000
SATA 500 9.3 2,000 18.6 1.40 $0.12 $1,068 $5,340 0.019 31.7 228,333,600
SATA 750 9.3 1,333 12.4 0.93 $0.08 $712 $3,560 0.012 31.7 152,222,400
SATA 1000 9.3 1,000 9.3 0.70 $0.06 $534 $2,670 0.009 31.7 114,166,800
Disk Power Consumption - Cost per Petabyte.
32
Comparing Storage Subsystem Power Costs
Significant power savings may be realized by redistributing data to the appropriate type and size of disk drive.
All on One TierPercent of Total
1000 TB of Storage
Storage Type
Storage Frames In KW
Disk Type
Disk Size in GB
Number of Disks
Disk Power in KW
Total Power in
KW
Power Cost per Year
All Data 100% 1000 (11) DS8300 143.0 FC 146 6850 74.1 217.1 $166,219
Tiered by ActivityPercent of Total
1000 TB of Storage
Storage Type
Storage Frames In KW
Disk Type
Disk Size in GB
Number of Disks
Disk Power in KW
Total Power in
KW
Power Cost per Year
Frequently accessed 10% 100 (2) DS8300 26.0 FC 146 685 7.4 33.4 $25,580
Infrequently accessed 20% 200 (3) DS4800 7.9 FC 300 667 9.2 17.1 $13,097
Seldom accessed 70% 700 (9) DS4200 6 SATA 750 934 8.7 14.7 $11,244
= = = = = =
$49,921
Power Cost - Traditional Storage
Power Cost - Tiered Storage
33
Comparing Storage Subsystem Cooling Costs
Additional power savings may be realized from the reduced cooling requirements provided by high capacity, lower wattage disk drives.
All on One TierPercent of Total
1000 TB of
Storage
Storage Type
Storage Frame Heat in BTUs
Disk Size in GB
Number of Disks
Individual Disk Heat in BTUs
Total Disk
Heat in BTUs
Total System Heat in BTUs
Total power for Cooling in KW
Cooling Cost per
Year
All Storage 100% 1000 (11) DS8300 262801 146 6850 36.9 252765 515566 74 $56,702
Tiered by ActivityPercent of Total
1000 TB of
Storage
Storage Type
Storage Frame Heat in BTUs
Disk Size in GB
Number of Disks
Individual Disk Heat in BTUs
Total Disk
Heat in BTUs
Total System Heat in BTUs
Total power for Cooling in KW
Cooling Cost per
Year
Frequently accessed 10% 100 (2) DS8300 47782 146 685 36.9 25277 73059 21 $16,389Infrequently accessed 20% 200 (3) DS4800 2412 300 667 47.1 31416 33828 10 $7,588
Seldom accessed 70% 700 (9) DS4200 13644 750 934 31.7 29608 43252 13 $9,703 = = = =
86300 150,138 44 $33,680
Cooling Cost - Traditional Storage
Cooling Cost - Tiered Storage
34
Comparing Storage Floor-Space Cost
Model Scenario
Total Number of Units
Unit Width in Inches
Unit Depth in Inches
Front -Depth in Inches
Rear - Depth in Inches
Footprint per Unit (sq. ft.)
Total Footprint
(sq. ft.)
Typical Cost per
sq. ft. per Month
Total Cost per Month
Total Cost Per Year
Percent Difference
DS8300Traditional
Storage 11 104.1 46.6 48 30 90.08 990.83 $65 $64,404 $772,847 - - - -
DS8300 2 104.1 46.6 48 30 90.08 180.15
DS4800 3 25.4 43.3 36 36 20.34 61.01
DS4200 9 25.4 43.3 36 36 20.34 183.04 = = = =
424.20 $65 $27,573 $330,878 57%
Service Clearance
Tiered Storage
The DS4800 and DS4200 storage subsystems include the required number of disk expansion trays mounted in standard equipment racks.
35
How Do the Costs Add Up?
21 43 65 87 48471615109 14131211 222120191817 282726252423 3029 3231 4645444342414039383736353433
CONSOLECatalyst 2948G-GE-TX
STATUS PSI RPSU
49 50 51 5210/100 MGT
System Storage EXP420
4G B/s 2G B/s
System Storage EXP420
4G B/s 2G B/s
System Storage EXP420
4G B/s 2G B/s
System Storage DS4200
4G B/s 2G B/s
DS4200
System Storage DS4700
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
DS4700
System Storage DS4700
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
TotalStorage
DS4800
DS4800
System x3455
System x3455
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESET DS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESET DS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
DS8300 DS4800 DS4200s with SATA Disk
21 43 65 87 48471615109 14131211 222120191817 282726252423 3029 3231 4645444342414039383736353433
CONSOLECatalyst 2948G-GE-TX
STATUS PSI RPSU
49 50 51 5210/100 MGT
System Storage EXP420
4G B/s 2G B/s
System Storage EXP420
4G B/s 2G B/s
System Storage EXP420
4G B/s 2G B/s
System Storage DS4200
4G B/s 2G B/s
DS4200
System Storage DS4700
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
DS4700
System Storage DS4700
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
TotalStorage
DS4800
DS4800
System x3455
System x3455
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESET DS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESET DS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
21 43 65 87 48471615109 14131211 222120191817 282726252423 3029 3231 4645444342414039383736353433
CONSOLECatalyst 2948G-GE-TX
STATUS PSI RPSU
49 50 51 5210/100 MGT
System Storage EXP420
4G B/s 2G B/s
System Storage EXP420
4G B/s 2G B/s
System Storage EXP420
4G B/s 2G B/s
System Storage DS4200
4G B/s 2G B/s
DS4200
System Storage DS4700
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
DS4700
System Storage DS4700
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
System Storage EXP810
4G B/s 2G B/s
TotalStorage
DS4800
DS4800
System x3455
System x3455
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESET DS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESET DS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
Traditional Approach
Everything on DS8300s
Tiered Storage Approach
Power Cost $166,219Cooling Cost $56,702Floor Space $772,847
=======$995,768
Power Cost $49,921Cooling Cost $33.68Floor Space $330,878
=======$380,833
Savings: $614,935 / yr.
36
A Look at Older Disk Subsystem Efficiency
Front Rear
ESS800 w/frame 1 55.9 145.6 GB 13,112 47,000 115.7 35.8 34 45 5672.06ESS800 w/frame 1 55.9 - - - - 13,112 47,000 115.7 35.8 - - - - - - - - 4142.06
= = = = = = = = - - - - = = = = = = = = = = = = = = = = = = = = = = = = = = = = 2 111.8 - - - - 26224 94000 231.4 35.8 34 45 9814.12
DS8300 1 115.2 300 GB 7000 23,891 104.1 44.6 48 30 6082.862107-9AE 1 - - - - - - - - 6,000 20,478 - - - - - - - - - - - - - - - - - - - -
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 2 115.2 - - - - 13000 44369 104.1 44.6 6082.86
DS8300 Benefits 100% 103% - - - - 50% 53% 45% 125% - - - - - - - - 38%
BTUs/hr. Width DepthTotal
Sq. Ft.
100 TB of Storage
Access Clearance Model Frames
Capacity in TB
Largest FC Drives Watts
Storing 100 TB of data on more modern storage subsystems results in 50% less power consumption, a 53% reduction in BTUs per hr., and a reduction in required floor space of 38%.
In addition, a DS8300 system has over 7x the throughput of the ESS800.
37
Why is Tiered Storage Important?
Maps data’s business value to disk characteristics
Places data on storage appropriate to its usage
Incorporates lower cost disks Reduces resource usage
(power, cooling, etc.) Matches user access needs to storage characteristics Capitalizes on higher capacity disk drive technology Increases overall performance of the system
38
A Typical Tiered Storage Architecture
21 43 65 87 48471615109 14131211 222120191817 282726252423 3029 3231 4645444342414039383736353433
CONSOLECatalyst 2948G-GE-TX
STATUS PSI RPSU
49 50 51 5210/100 MGT
System Storage EXP420
4GB/s 2GB/s
System Storage EXP420
4GB/s 2GB/s
System Storage EXP420
4GB/s 2GB/s
System Storage DS4200
4GB/s 2GB/s
DS4200
System Storage DS4700
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
DS4700
System Storage DS4700
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
TotalStorage
DS4800
DS4800
System x3455
System x3455
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESETDS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESETDS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
21 43 65 87 48471615109 14131211 222120191817 282726252423 3029 3231 4645444342414039383736353433
CONSOLECatalyst 2948G-GE-TX
STATUS PSI RPSU
49 50 51 5210/100 MGT
System Storage EXP420
4GB/s 2GB/s
System Storage EXP420
4GB/s 2GB/s
System Storage EXP420
4GB/s 2GB/s
System Storage DS4200
4GB/s 2GB/s
DS4200
System Storage DS4700
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
DS4700
System Storage DS4700
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
TotalStorage
DS4800
DS4800
System x3455
System x3455
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESETDS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESETDS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
21 43 65 87 48471615109 14131211 222120191817 282726252423 3029 3231 4645444342414039383736353433
CONSOLECatalyst 2948G-GE-TX
STATUS PSI RPSU
49 50 51 5210/100 MGT
System Storage EXP420
4GB/s 2GB/s
System Storage EXP420
4GB/s 2GB/s
System Storage EXP420
4GB/s 2GB/s
System Storage DS4200
4GB/s 2GB/s
DS4200
System Storage DS4700
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
DS4700
System Storage DS4700
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
System Storage EXP810
4GB/s 2GB/s
TotalStorage
DS4800
DS4800
System x3455
System x3455
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESETDS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
STATUS
P/S
FAN
LINK ACT
CONSOLE MGMT10/100
RESETDS - C9140 - K9
MDS 9140 MULTILAYER INTELLIGENT FC SWITCH
5 8
LN- -SP
1 4
LN- -SP
37 40
LN- -SP
33 36
LN- -SP
29 32
LN- -SP
25 28
LN- -SP
21 24
LN- -SP
17 20
LN- -SP
13 16
LN- -SP
9 12
LN- -SP
Business Critical
High Performance / Very High Availability
Business Important
Good Performance / High Availability
Business Standard
Average Performance / Standard Availability
Reference / Historical
Near-line or Off-line
Normally a tiered storage strategy is based on data’s business value.
DS8300 DS4800 DS4200s with SATA Disk
TS3500 Tape
Library
39
Choosing the Right Controller FrameStorage Controller Cost Comparison
$0
$50,000
$100,000
$150,000
$200,000
$250,000
DS8300 BaseSystem
DS6800Controller
DS4800Controller
DS4700Controller
DS4200Controller
List Price
DS8300 Base System
DS6800 Controller
DS4800 Controller
DS4700 Controller
DS4200 Controller
40
Choosing the Right Disk Characteristics
Ultra High Performance
High Performance (15K RPM) orLow capacity – High Spindle Count (10K RPM)
Medium Performance SCSI or SAS (10K RPM) orHigh capacity SCSI or SAS (10K RPM)
Solid State Disk
Tier 1
Tier 2
Tier 3
Tier 4
FC or SAS SCSI Disk
FC SCSI Disk
High capacity, very low cost medium for long-term archiving
UDO, DVD, MO, or TAPE Library(Near-Line Media)
Tier 0
High capacity, low cost SATA Disk
SATA Disk
Notes:
q Tiers 0 and 1 must be physically close to the client equipment for latency reasons. Tiers 2 – 4 can be more remote.
q Active data in Tiers 0 & 1 is normally less than 20% of the total data.
q Inactive data (80%) is not significantly affected by latency issues from asynchronous connections.
q Management and capacity planning for the entire tiered structure can be executed from any geographic location.
System Primary Memory (DRAM and Cache )
41
Comparing Disk Drive Attributes Cost-per-GB By Disk Drive Type
$22.87
$16.89
$13.51
$24.51
$22.87
$16.89
$15.25$13.69
$10.66
$2.56 $2.56 $2.30
$0.00
$5.00
$10.00
$15.00
$20.00
$25.00
$30.00
4GB F
C 73.
4GB 1
5K
4GB F
C 146
.8G
B 15K
4GB F
C 300
GB 1
5K
2GB F
C 36.
4GB/1
5K
2GB F
C 73.
4GB/1
5K
2GB F
C 146
.8G
B/15K
2GB F
C 73.
4GB/1
0K
2GB F
C 146
.8G
B/10K
2GB F
C 300
GB/10K
400
GB S
ATA II
500
GB S
ATA II
750
GB S
ATA II
4GB FC 73.4GB 15K
4GB FC 146.8GB 15K
4GB FC 300 GB 15K
2GB FC 36.4GB/15K
2GB FC 73.4GB/15K
2GB FC 146.8GB/15K
2GB FC 73.4GB/10K
2GB FC 146.8GB/10K
2GB FC 300GB/10K
400 GB SATA II
500 GB SATA II
750 GB SATA II
42
The Cost Impact of Adding Disk TraysDeclining Cost-Per-GB
(DS4800, in 4-disk increments)
$0.00
$10.00
$20.00
$30.00
$40.00
$50.00
$60.00
$70.00
$80.00
$90.00
$100.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Disk Trays
Per GB
Cost per GB
Note: Calculations based on 146 GB, 10K RPM Drives
43
Tiered Storage Design Pros and Cons
Advantages• Lower initial purchase price• Higher capacity per square foot• Reduced power consumption• Decreased requirement for cooling• Increased equipment flexibility• Potentially a higher performance solution
Disadvantages• Inherently a more complex architecture• Greater up-front effort to design and implement• Requires advanced storage design skills and knowledge
44
RAID Selection Decision Drivers
Application or Database characteristics• Read/write mix
• Dependency on IOPS
RAID Performance characteristics• Appropriate RAID level
• Number of disks per array
• Stripe size
• Available bandwidth
Configuration rules and recommendations
Loss from data parity and hot sparing Disk failure probability RAID parity rebuild times
45
Loss from Mirroring, Striping, and Sparing
RAID10
S
S
Usable
468.8 TB
Unusable
531.2 TB
Loss
53.1%Stripe
Mirror
RAID1
S
S
Usable
468.8 TB
Unusable
531.2 TB
Loss
53.1%
Mirror
RAID10 =
Mirror plus Stripe
RAID1 =
Mirror Only
46
Loss from RAID5 Parity and Sparing
RAID5 - 3+P Array Usable559.5 TB
Unusable 531.2 TB
Loss44.1%
P P P P
SP P P P
RAID5 - 7+P Array Usable791.5 TB
Unusable 208.5 TB
Loss20.9%
P P
SP P
RAID5 - 14+P Array Usable857.1 TB
Unusable 142.9 TB
Loss14.3%
P
SP
S
Note: The second tray has one 2+P array to allow for one spare drive per two trays.
Note: The second tray has one 6+P array to allow for one spare drive per two trays.
Note: Each tray has one spare drive per tray.
47
Other Architectural Considerations
Compatibility High availability Architectural robustness Flexibility and scalability Stability of the technology Vendor’s financial standing Well defined product line
roadmap Support for industry
standards
48
Performance and Throughput
49
Storage Subsystem Performance Drivers
Business objectives and user expectations Applications and database characteristics Server characteristics SAN fabric characteristics Storage controller characteristics Caching characteristics Configuration characteristics Disk latency characteristics
"We can't solve problems by using the same kind of thinking we used when we created them." Albert Einstein
50
Storage Performance Enhancers
Data Mover – Reassigning data transfer tasks to a specialized “engine” reduces the workload on the host processing system.
Search Engines – Systems dedicated to executing searches in vast amounts of stored data to satisfy specific requests.
Directory Services – Stores and organizes information about resources and data objects.
High Speed Interconnections – Dedicated “behind the scenes” networks dedicated to the transfer of large amounts of data.
Autonomic Computing – must have an ability to reconfigure itself under varying and possibly unpredictable conditions.
51
Other Storage Performance Tips
Method Description
RAID selectionStriping data across multiple disks to reduce the response latency of individual disks and increase the available bandwidth of the array.
Thin ProvisioningAllows applications and databases to be allocated more capacity than is physically reserved on the storage subsystem.
Parallel file systems
Provides individual file block striping across multiple disks to greatly improve I/O throughtput.and storage capacity.
VirtualizationProvides some caching of data, spreads I/O across multiple storage controller channels, and can be used as a "data mover" for remote data replication.
Increased spindle count
Deploying a large number of low capacity, high speed disks in a RAID array to increase the number of spindles and improve the array's IOPS capability.
High RPM disk drives
Implementing 15K RPM drives to reduce the amount of disk latency and increase the transfer rate.
Striping across trays
Building a RAID array by vertically striping it across multiple disk trays and storage controller channels.
Increasing cache size
Installing additional storage cache to increase the percentage of "read hits" from cache and avoiding much slower data read requests from disks.
Sequential file isolation
Isolate sequential write files on dedicated physical disks to minimize the amount of head seek time and disk rotational latency.
1:1 SAN port relationships
Assign high performance servers and storage controllers to SAN Fabric ports with a 1:1 relationship to ensure maximum bandwidth is always available.
Hot spot elimination
Monitor disk storage for "hot spots" (over-worked array areas) and reallocate storage to eliminate the usage imbalance.
52
SAN Fabric Network
53
SAN Fabric Overview
SAN Fabric is the interconnecting structure between associated servers and storage devices
Proper fabric design will directly impact on:• Performance• Availability• Equipment cost• Manageability• Maintainability
Communications protocol can be either Fibre Channel, Ethernet, or a combination of both
Ability of the fabric to scale is critical Monitoring of SAN fabric traffic is a necessity
54
Designing a High Performance Fabric
Select an optimal SAN fabric topology• Mesh• Waterfall• Core / Edge• Fat tree• Butterfly• Torus• Hypercube• Hybrid
Ensure the design is driven by business application requirements
Keep it as simple as possible for manageability
55
Common SAN Fabric Examples
Server
Server
Storage
StorageBasic Dual Switch
Server
Server
Storage
StorageFabric Mesh
Server Server
Director Director
Highly Availability Core - Edge Fabric
Server Server
StorageStorage StorageStorageFat Tree Fabric Architecture
56
SAN Fabric Design Considerations
SAN Fabric design issues:• Throughput requirements• Potential bottlenecks• Port speed / port count• Port subscription rate• Maximum hop count• Redundancy for High Availability• Modularity (flexibility/scalability)• Future upgradeability• Complexity vs. overall manageability• Isolation vs. unification• Wide Area Network interconnections• Power consumption and footprint• Component cost
57
Backup and Recovery
58
Backup and Recovery Challenges
Size of the backup window Ability to recover files The time to recover files Integrity of the data backups Required frequency of backups Stored data retention period
• Functional• Legal
Available bandwidth for backup resources Media deterioration over time Technological obsolescence
59
The Traditional Storage Backup Approach
1.0 PB of Storage
TS3500 Tape Library
(192) of the newest LTO 4 tape drives running at a maximum native transfer rate of 120 MB/sec. would need at least 13-hours to back up 1.0 PB of data.
60
Fast Tape Drives can Saturate the Network
0
20
40
60
80
100
120
MB/sec.
Drive Type
Native Transfer Rate in MB/sec.
LTO-4
LTO-3
LTO-2
LTO-1
1120 1/2"
3592 1/2"
3590 1/2"
9940B
9940A
9840C
9840B
9840A
Effective transfer rate of Gigabit Ethernet
One LTO-4 tape drive run in native mode is 20% faster than the effective usable bandwidth of Gigabit Ethernet!
Four LTO-4 tape drives run at a 2:1 compressed mode will take most of the usable bandwidth of 10Gbps Ethernet!
61
Large Systems Backup Approaches
Point-in-Time Copies and Replication
• Snapshots Most popular method of large storage backup Snapshots create an exact copy of the source data Once a bitmap is created, storage access can resume While copying, new data is written to both source and target Requires minimal downtime for production systems
• Replication Replication creates a mirror image of data over distance May be synchronous (consistent) or asynchronous (lagging) Synchronous is distance-limited, asynchronous is not
62
Point-in-Time Copy
p5
xSeries 366
ESD
0
3
1
4
2
5
xSeries 366
ESD
0
3
1
4
2
5
SVC Virtualization
Engine
200MB/sec. x 4 ports x 4 Clusters3.2 GB/sec. bandwidth
P590 server LPARs
Redundant 200MB/sec. HBAs
200MB/sec. bandwidth
Internal Bandwidth400MB/sec. x 32 adapter ports
128 GB/sec. (64GB/sec. usable)
DS4800 Storage
Internal Bandwidth400MB/sec. x 4 adapter ports
1.6 GB/sec. (1.2 GB/sec. effective rate)
Internal Bandwidth400MB/sec. x 32 adapter ports
128 GB/sec. (64GB/sec. usable)
Internal Bandwidth400MB/sec. x 4 adapter ports
1.6 GB/sec. (1.2 GB/sec. effective rate)
200MB/sec. x 8 ports1.6 GB/sec.bandwidth DS4800
StorageTotalStorage
DS4800
TotalStorageEXP 710
TotalStorageEXP 710
TotalStorageEXP 710
TotalStorageEXP 710
TotalStorageEXP 710
TotalStorageEXP 710
TotalStorageEXP 710
TotalStorageEXP 710
DS8300 Storage
200MB/sec. x 8 ports1.6 GB/sec.bandwidth
DS8300 Storage
21
TotalStorage
FlashCopy BitMap
ACS Initiated
FlashCopy Backup Process
SVC Virtualization
Engine
200MB/sec. x 4 ports x 4 Clusters3.2 GB/sec. bandwidth
FlashCopy Backup and Cloning
Process
To Tape Library
1
2
3
4b
Mirror Side BMirror Side A
FlashCopy BitMap
5b
6a/b200MB/sec. x 2
channel bandwidth(400MB/sec. usable)
4a
5a
6a/b
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
WS-X9016
1/2 Gbps FC Module
SUPERVISOR
WS-X9530 SFI
CONSOLEMGMT
STATU
S
SYSTEM
RESET
ACTIVE
PWR M
GMT
COM 1CFI
10/100
STATUS
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
WS-X9016
1/2 Gbps FC Module
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
WS-X9016
1/2 Gbps FC Module
SUPERVISOR
WS-X9530 SFI
CONSOLEMGMT
STATU
S
SYSTEM
RESET
ACTIVE
PWR M
GMT
COM 1CFI
10/100
STATUS
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
WS-X9016
1/2 Gbps FC Module
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS2 8 10 16 18 24 26 32
1 7 9 15 17 23 25 31
WS-X9032-SMV
FC Services Module
STATUS
63
Data Replication Structures
MainframeStorage
Wintel Storage
UnixStorage
MissionCritical Storage
(DS8300)
Business Standard Storage
(DS4000 Series with
SATA)
Business ImportantStorage
(DS6800)
On-line file recovery and
backups
Disk Array
(check-points)
TSM Lan-less and
Server-less Backups
Hot FailoverDisk Array
(synchronous)
Warm FailoverDisk Array
(asynchronous)
Metro Mirror Global Mirror
Bunker East Facility
North Building
Bunker West Facility
Local Tape
Backups
TS
M S
erver-less B
ackups
MainframeStorage
Wintel Storage
UnixStorage
MissionCritical Storage
(DS8300)
Business Standard Storage
(DS4000 Series with
SATA)
Business ImportantStorage
(DS6800)
FlashCopy
On-line file recovery and
backups
Disk Array
(check-points)
Local Tape Backups
TSM Lan-less and
Server-less Backups
TS
M S
erver-less B
ackups
Hot FailoverDisk Array
(synchronous)
Warm FailoverDisk Array
(asynchronous)
Metro MirrorGlobal Mirror
Primary Data Center
Metro Area Storage Site
Remote Storage Site
TS
M S
erver-less B
ackups
TS
M S
erver-less B
ackups
Area Backups
???
Remote Backups
AndArchiving
TSM S
erve
r-les
s Bac
kups
???
Hypothetical Backup & Recovery Approach01/18/06
Tiered Storage
Service Level 1 -Continuous Availability
Service Level 2 – Under 15-Minute Recovery
Service Class 3 -Under 4-Hour Recovery
NOTE: This is about a 50,000 ft. overview of one possible approach. It is designed in a triple redundant disaster recovery architecture, so the primary structure could sustain a failure and the secondary structure could also sustain a disaster before the primary structure was restored, and the structure would still continue to support key business operations.
Local Site Area Site(within 170 miles)
Remote Site(anywhere in the World)
64
Other Large Systems Backup Approaches
Object-based Backups• Backs up only new blocks that have changed• Copies only files it has never seen before• Inserts pointers if file exists somewhere else• Provides instant recoveries by presenting a mountable volume
Delta-Block Incremental Backups• Evaluates changed data by breaking a file down into discrete blocks• Block-for-block comparison of a modified file with an existing file• When a difference is detected it extracts a copy of that block only• Usually copies a number of blocks, but not the entire file
Continuous Data Protection (CDP)• Copies blocks to the backup system as soon as they change• Stores data in a log that allows recovery from any point in time • Performs fast recoveries by restoring only blocks that have changed• Provides instant recoveries by presenting a mountable volume
65
Security Requirements
66
Impact of Petabyte Storage on Security
Traditional distributed access control techniques are designed for smaller systems with general or random workloads
Petabyte storage may service tens of thousands of clients and hundreds of storage devices
Storage design must be capable of supporting I/O patterns that are highly parallel and very bursty by nature
Security solutions must be kept highly scalable to keep up with storage growth patterns
67
Impact of Petabyte Storage on Security (Cont.)
Authentication and authorization requirements can dramatically impact server performance
Performance could be further reduced if data is encrypted
Traditional security protocols perform poorly because they do not scale well
The number of security operations is closely tied to the number of devices and requests
68
Regulatory Compliance
69
The Challenge Of Regulatory Compliance
What’s driving storage regulatory legislation?
• Corporate fraud and illegal practices• Increased emphasis on the security
of personal data• The threat of terrorist activities• The global impact of the Internet• Increased reliance on stored data
for defense against litigation • Increased business dependence on
electronic communications (e-mail, digital voice messaging, instant messaging, VoIP, etc.)
70
Regulatory Requirements Continue to Grow
According to the Storage Networking Industry Association (SNIA) there are over 20,000 regulations worldwide addressing the storage of data
If you do business overseas, you must also be aware of the applicable foreign regulatory requirements
The number of government regulatory requirements increases every year
• There is little chance this upward trend will reverse itself in the future
• Regulatory guidelines do not dictate how you should maintain your data, only what the expected outcome should be.
71
Common Regulatory Compliance Goals
Most regulatory requirements are based upon:
• Security: Maintain data in a secure environment
• Efficiency: Rapid location and retrieval of data
• Legibility: Recovered documents must be in a readable format that is clear and concise
• Authenticity: The recovered document must be verifiable as the original
• Validation: Documentation process must be available for review by a neutral third party
Regulatory compliance becomes more challenging as storage subsystems grow in size and complexity.
72
Regulatory Legislation Examples
Sarbanes-Oxley HIPAA USA Patriot Act Gramm-Leach-Bliley Act FRCP CFR 240 17a(f) NASD 3010 and 3110 21 CFR Part 11 (FDA) DOD 5015.2 California Senate Bill 1386 Florida Sunshine Law
PCI ISO 17799 CFR Title 18 (FERC) E-SIGN EU Directive 95/46/EC Basel II NARA GRS20 CFR Title 47, Part 42 NASD 2711/NYSE Rule 472 JCAHO FPC 65 COOP compliance
73
Maintenance Requirements
74
Disk Drive Reliability Misconceptions
Actual disk failure rate is usually higher than published– Vendors indicate a .58% - .88% failure
rate – Actual field usage suggests a 1.5% - 3.5%
(or greater) failure rate
Field studies show no appreciable difference in reliability between SCSI, FC, and SATA drives
Heat and high duty cycles do not appear to have as detrimental of an effect on disk life as once thought
75
Disk Drive Reliability Misconceptions (Cont.)
Infant Mortality doesn’t appear to be a significant issue for newly installed disks
Disks exhibit a fairly linear rate of failure over time, which is contrary to the standard “bathtub” model
Self-Monitoring, Analysis and Reporting Technology (S.M.A.R.T.) diagnostics appear to predict only about 50% of all disk failures
76
Disk Failures Are an Expected Occurrence
2
7 6
20
0
5
10
15
20
25
30
1% Rate 3.5% Rate
Projected Disk Failures per Month
Tiered
Monolythic
Tiered
TieredTraditional
Traditional
Using the disk count from our previous Traditional vs. Tiered models, it’s easy to see that disk failures will occur on a regular basis.
77
Other Considerations
78
Other Issues to Consider
Design for minimal human intervention Maintain extensive monitoring of the environment Exercise control over information propagation Architect for maximum performance and throughput Ensure robustness and high availability Configure for scalability and flexibility Develop well defined SLA objectives Implement a structured support operation
79
Emerging Technologies
80
Emerging Technologies to Watch
Thin Provisioning Data de-duplication SAS Interface InfiniBand NPIV (N_Port ID Virtualization) Large-platter storage technologies 2.5” disk technologies Solid state disk drives Virtualized file systems (i.e.- ZFS, SOFS) Grid Storage
81
Summary
82
Some Parting Thoughts
Online multi-petabyte storage is a reality Data will double every two to three years Storage media cost-per-GB will continue to decline Storage Operational management is a growing issue Governmental regulations will increase over time New technologies will demand additional storage Experienced storage designers and administrators will
grow increasingly harder to find Scarce data center resources (bandwidth, floor space,
power, cooling, etc.) will become more expensive A carefully designed architecture is the key to efficient
storage operations
83
Putting It All Together
Primary Storage
Primary Storage
Primary Storage
Primary Storage
Synchronous Replication
(DR Hot Site)
Remote Data Sources
Asynchronous Replication
(DR Warm Site)
Data Movers
High SpeedSearch Engine
FlashCopyStorage
Primary Storage
Primary Storage
Primary Storage
Secondary Storage
Linux Server GPFS Clusters
Virtualization Cluster
Virtualization Cluster
Data Movers
SAN Routers
Edge
Edge
Near-LineStorage(Tape
Library)
Storage Management
Event Reporting
PerformanceManagement
Fabric Management
Infiniband Core
Edge
Edge
Long-Term Archive
(Bunker Storage)
FlashCopyStorage
Backup & Recovery
(Tape Library)
Within 300 KM Any Distance
Re
dun
da
nt W
AN
Pro
viders
HSM MetadataDatabase
HSMStructure
Redundant Backup Capability
Network Data Compression
Multi-Petabyte Storage Infrastructure05/12/06
Fast File Recovery
Local Server Internal Backups
Data Reduction Techniques
Infiniband Core
Infiniband Core
Captain Kirk: Scotty - We need more power!!!
Mr. Scott: Capn, I'm gi'in ya all I got, she can na take much more!
84
Questions?
Randy Cochran, Infrastructure Architect
IBM Global Technical Services - Cell: (630) 248-0660 - [email protected]