future of data storage in the cloud
DESCRIPTION
Presentation slides for a session at Cloud Computing Expo West 2010 on the growth of data and our need to store it more efficiently.TRANSCRIPT
Community StackerBret Piatt
Future of Data Storage in the CloudCloud Computing Expo West 2010
Twitter: @bpiatt
OBLIGATORY
“WHAT IS THE CLOUD?”
GLOBAL
ON DEMAND
INFRASTRUCTURE
70’s – 80’sMainframe Era
90’s-2000’sClient Server Era
2010-beyondCloud Era
[Based on a Gartner Study]
2010 IT budgets aren’t getting cut....but CIOs expect their spend to go further.
• #1 Priority is Virtualization• #2 is Cloud Computing
A NEW ERA OF COMPUTING
HOW ABOUT NOW..
“..YOU TALK ABOUT SOMETHING REAL?”
HOW BIG IS A..
GIGABYTE500,000 pages of text
15 minutes of HD Video
TERABYTE10,000 hours of high quality audio
35 Blu-ray discs
PETABYTEAll of the data for World of Warcraft™
62,400 hours of HD video
EXABYTEThe amount of data sent on the global
wireless networks per month
ZETTABTYEAll of the data on Earth today
150GB of data per person
ZETTABTYE2% of the data on Earth in 2020
HOW DO WE STORE IT TODAY?NOT THIS EFFICIENTLY
STATE OF THE ART DENSITY
3TB DRIVES AT 15 PER RU
529,101 42U CABINETS FOR A ZETTABYTE
DATA IS OUTGROWING NETWORK
1 PETABTYE OF DATA TAKES OVER 10 DAYS TO MOVE..
NOT
..ON A 10 GIGABIT NETWORK!
With the cost to store data approaching zero......we're ending up with digital Hoarders™.
JUST BECAUSE YOU CAN DOESN’T MEAN YOU SHOULD
If we stored all of the global data as “an average” enterprise..
..it would take....38.5% of the World GDP!
EFFICIENT STORAGE IS KEY
ITEM MONTHLY FIGURES
ENTERPRISE AVGERAGE STORAGE COST $1.98 PER GIGABYTE
WORLD GDP $5.13 TRILLION
COST TO STORE A ZETTABYTE $1.98 TRILLION
THE OPPORTUNITY
MOST DATA IS AT REST
STORAGE I/O IS EXPENSIVE
ADVANCED FEATURES ARE EXPENSIVE
USE HYBRID DATA STORAGE
MAXIMIZE BENEFITS
OBJECT STORAGE
Distributed, REST-based API, No central database
Hardware agnostic - commodity hardware, RAID not required
Account/Container/Object structure (not file system, no nesting)
Replication (N copies of accounts, containers, objects)
Data distributed evenly throughout system
Scalable to multiple petabytes, billions of objects
EVOLUTION OF OBJECT STORAGE
SMALL SCALE DEPLOYMENT
5 Storage Nodes, $0.13 per GB (monthly)
490TB of disk, 160TB usable
18.5kVA, 35 RU, 5 “half cabinets”
$0.114 per GB OPEX (monthly)
$0.014 per GB CAPEX ($200k, 36 month refresh)
$0.38/GB (with 3 copies), < 20% of “average”
LARGE SCALE DEPLOYMENT
