cloud storage: infrastructure - etouches€¦ · agenda (1) the need for cloud storage data centers...

42

Upload: doanhuong

Post on 12-Apr-2018

220 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack
Page 2: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Storage:

Key to Future of IT

Infrastructure Anil Vasudeva

President & Chief Analyst [email protected]

408-268-0800

Page 3: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Agenda

(1) The Need for Cloud Storage Data Centers Infrastructure

IT Roadmap to Clouds

Cloud Computing Ecosystem & Architectural Stack

Storage Usage in Data Centers & in Cloud Mega-Centers

Applications mapped by Key Workload & Storage Metrics

(2) Cloud Infrastructure Technology Enablers Virtualization: Key to Cloud Storage Infrastructure

SSDs: A New SCM Filling Price Performance Gaps

Networks: Fast March to 100Gb/sec

Hadoop: Meeting Big Data/BI/Data Base Paradigm

(3) Drivers for Cloud Storage Adoption Storage Efficiencies and Costs Reduction

Tier 4 Storage for Data Protection

Real-Time Online Analytics

Technology Innovations HW, SW 2005-2015

(3) Cloud Segments: Challenges & Solutions Adopting Public/Private/Hybrid Clouds: Pros & Cons

(4) Cloud Storage Requirements Adopting Public/Private/Hybrid Clouds: Pros & Cons

Best Practices: Getting Ready for the Cloud

(5) Key Take-Aways

Page 4: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

4 4

Data Centers & Cloud Infrastructure

Enterprise VZ Data Center On-Premise Cloud

Home Networks

Web 2.0

Social Ntwks.

Facebook,

Twitter, YouTube…

Cable/DSL…

Cellular

Wireless

Internet ISP

Core Optical

Edge

ISP

ISP

ISP

ISP

ISP

Supplier/Partners

Remote/Branch Office

Public CloudCenter©

Servers VPN IaaS, PaaS

SaaS

Vertical

Clouds

ISP

Tier-3

Data Base

Servers

Tier-2 Apps

Management Directory Security Policy

Middleware Platform

Switches: Layer 4-7,

Layer 2, 10GbE, FC Stg

Caching, Proxy,

FW, SSL, IDS, DNS, LB,

Web Servers

Application Servers

HA, File/Print, ERP,

SCM, CRM Servers

Database Servers,

Middleware, Data

Mgmt

Tier-1

Edge Apps

FC/ IPSANs

Source:: IMEX Research - Cloud Infrastructure Report ©2009-12

Page 5: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

IT Industry Journey - Roadmap

Cloudization On-Premises > Private Clouds > Public Clouds DC to Cloud-Aware Infrast. & Apps. Cascade migration to SPs/Public Clouds.

Integrate Physical Infrast./Blades to meet CAPSIMS ®IMEX Cost, Availability, Performance, Scalability, Inter-operability, Manageability & Security

Integration/Consolidation

Standard IT Infrastructure- Volume Economics HW/Syst SW

(Servers, Storage, Networking Devices, System Software (OS, MW & Data Mgmt SW)

Standardization

Virtualization Pools Resources. Provisions, Optimizes, Monitors Shuffles Resources to optimize Delivery of various Business Services

Automatically Maintains Application SLAs (Self-Configuration, Self-Healing©IMEX, Self-Acctg. Charges etc)

Automation

IT Industry

Roadmap IMEX Research

Analytics – BI

Predictive Analytics - Unstructured Data From Dashboards Visualization to Prediction Engines using Big Data.

Page 6: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Examples

eMail - Yahoo!,Google…

Collaboration - Facebook,Twitter …

Bus.Apps - SalesForce, GoogleApps, Intuit…

Examples

Amazon EC2

Force.com

Navitaire

Examples

Amazon S3

Nirvanix

Infrastructure HW &

Services - Servers, Network, Storage

- Management, Reporting

SaaS

PaaS

IaaS

Platform Tools & Services - Deploy developed platforms

ready for Application SW on

Cloud Aware Infrastructure

Software-as-a-Service - Servers, Network, Storage

- Management, Reporting

Service Providers

Examples

Public - BT, Telstra, T-Systems France Telecom

Private –

Hybrid – IBM/Cloudburst,

Cloud Services Providers Public – Mutitenancy,OnDemand

Private - On Premises, Enterprise

Hybrid – Interoperable P2P

Cloud Computing Ecosystem

Page 7: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Platform Tools & Services

Operating Systems

Cloud Computing Public Cloud Service

Providers

Private Cloud

Enterprise

App SLA

SaaS Applications

…..

.Net

Pyt

ho

n

EJB

Ru

by

PH

P

….. PaaS

IaaS

SaaS

Virtualization

Resources (Servers, Storage, Networks)

Hybrid

Cloud

App SLA

App SLA

App SLA

App SLA

Man

ag

em

en

t

Virtualized Cloud Infrastructure

Application’s SLA dictates the Resources Required to meet specific

requirements of Availability, Performance, Cost, Security, Manageability etc.

Page 8: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

8 8

Data Storage Usage –

In Corporate Data Centers

I/O Access Frequency vs. Percent of Corporate Data 100%

Cache • Logs

• Journals

• Temp Tables

• Hot Tables

SS

D

Disk

Arrays • Tables

• Indices

• Hot Data

Tape

Libraries • Back Up Data

• Archived Data

• Offsite DataVault

2% 10% 50% 100% 1% % of Corporate Data

65%

75%

95%

% o

f I/O

Access

es

Source:: IMEX Research - Cloud Infrastructure Report ©2009-12

Page 9: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

9

Applications Best Suited for Cloud

• Temporary Storage

– Test & Dev Data, Excess Storage from Data Centers

• Internet Web Content – Web 2.0 user-generated content, Digital Catalogs, Public Files, iPhone App Content

• Data Archiving & Records Retention – Regulatory compliance, Corporate governance, Audits & searches, Legal, Intellectual

Property, Risk Management

• Data Back Up & Restore – Offsite data protection, Historical Versioning, Online Access, Automated Retrieval

• Intranet File Storage – Employee Portals, Digital Libraries, Document Collaboration, Training Media, File

Shares, Historical Versioning, Online Access, Automated Retrieval

Source: IMEX Research SSD Industry Report ©2010-122

Page 10: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

10 10

I/O Access Frequency vs. Percent of Corporate Data

SSD • Logs

• Journals

• Temp Tables

• Hot Tables

FCoE/ SAS

Arrays

• Tables

• Indices

• Hot Data

Cloud

Storage SATA

• Back Up Data

• Archived Data

• Offsite DataVault

2% 10% 50% 100% 1% % of Corporate Data

65%

75%

95%

% o

f I/O

Access

es

NextGen Data Storage Usage in Cloud

Source:: IMEX Research - Cloud Infrastructure Report ©2009-12

Page 11: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Market Segments by Applications

*IOPS for a required response time ( ms) *=(#Channels*Latency-1)

(RAID - 0, 3)

500 100

MB/sec 10 1 50 5

Data

Warehousing

OLAP

Business

Intelligence (RAID - 1, 5, 6)

IOP

S*

(*L

ate

nc

y-1

)

Web 2.0 Audio

Video

Scientific Computing

Imaging

HPC

TP

HPC

10K

100 K

1K

100

10

1000 K OLTP

eCommerce

Transaction

Processing

Source:: IMEX Research - Cloud Infrastructure Report ©2009-12

Page 12: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Storage Requirements

Source: IMEX Research SSD Industry Report ©2011

DRAM

Flash

SSD

Performance

Disk

Capacity Disk

Tape

Auto Tiering System using SSDs

Data Class (Tiers 0,1,2,3)

Storage Media Type (Flash/Disk/Tape)

Policy Engines (Workload Mgmt)

Transparent Migration (Data Placement)

File Virtualization (Uninterrupted App.Opns.in Migration)

Replication

RAID – 0,1,5,6,10

Virtual Tape

Back Up/Archive/DR

Data Protection

Storage Virtualization

MAID

Deduplication

Thin Provisioning

Storage Efficiency

Auto Tiered Storage

Page 13: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Workloads Consolidation using VZ

Source: Dan Olds & IMEX Research 2009

• A single server 1.5x larger than

standard 2-way server will handle

consolidated load of 6 servers.

• VZ manages the workloads +

important apps get the compute

resources they need automatically

w/o operator intervention.

• Physical consolidation of 15-20:1

is easily possible

• Reasonable goal for VZ x86

servers – 40-50% utilization on

large systems (>4way), rising as

dual/quad core processors

becomes available

• Savings result in Real Estate,

Power & Cooling, High Availability,

Hardware, Management

Page 14: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

TCO Savings with Virtualization

$-

$4,000

$8,000

$12,000

$16,000

w/o VZ w VZ

Provisioning

HardwareSAN

Network

Power & Cooling

DC Real Estate

Disaster Recovery

Downtime

Co

st

over

3 y

ears

995 Pre-Virtualization (VZ) Servers 78 VZ

Servers

VZ SW

&

Support Fo

r T

CO

An

aly

sis

, E

M: im

ex@

im

exre

se

arc

h.c

om

(

40

8)

26

8-0

800

Page 15: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Storage Requirements

Source: IMEX Research SSD Industry Report ©2011

DRAM

Flash

SSD

Performance

Disk

Capacity Disk

Tape

Auto Tiering System using SSDs

Data Class (Tiers 0,1,2,3)

Storage Media Type (Flash/Disk/Tape)

Policy Engines (Workload Mgmt)

Transparent Migration (Data Placement)

File Virtualization (Uninterrupted App.Opns.in Migration)

Replication

RAID – 0,1,5,6,10

Virtual Tape

Back Up/Archive/DR

Data Protection

Storage Virtualization

MAID

Deduplication

Thin Provisioning

Storage Efficiency

Auto Tiered Storage

Page 16: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

16 16

SSD Technology –

Filling Price/Perf Gaps

HDD

Tape

DRAM

CPU

SDRAM

Performance I/O Access Latency

HDD becoming

Cheaper, not faster

DRAM getting

Faster (to feed faster CPUs) &

Larger (to feed Multi-cores &

Multi-VMs from Virtualization)

SCM

NOR

NAND

PCIe

SSD

SATA

SSD

Price $/GB

Source: IMEX Research SSD Industry Report ©2010-12

SSD segmenting into

PCIe SSD Cache

- as backend to DRAM &

SATA SSD

- as front end to HDD

Page 17: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

17 17

SSD in SAN Storage - Perf & TCO

SAN TCO using HDD vs. Hybrid Storage

14.25.2

75

28

0

64

0

36

145

0

0

50

100

150

200

250

HDD Only HDD/SSD

Co

st

$K

Power & Cooling RackSpace SSDs HDD SATA HDD FC

Pwr/Cool

RackSpace

SSD

HDD-

SATA

HDD-FC

SAN PerformanceImprovements using SSD

0

50

100

150

200

250

300

FC-HDD Only SSD/SATA-HDD

IOP

S

0

1

2

3

4

5

6

7

8

9

10

$/I

OP

Performance (IOPS) $/IOP

$/IOPS

Improvement

800%

IOPS

Improvement

475%

Source: IMEX Research SSD Industry Report ©2010-12

SAN Storage – Performance & TCO Improvements using SSD

Page 18: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

SSD Challenges & Solutions:

Goals & Best Practices

Best Practices – By leveraging Error Avoidance Algorithms, and Best

Practices of Verification Testing, to keep total functional failure rate <=3% (with defects and wear-outs issues combined)

– In practice, endurance ratings are likely to be significantly higher than typical use, so data errors and failures will be even less.

– Capacity Over-provisioning will provide large increases in random performance and endurance.

– Select SSD based on confirmed EVT Ratings – Use MLC within requirements of Endurance Limits

20%

15%

10%

5%

0% % o

f D

rive

s F

ail

ing

(A

FR

%)

5 3 2 1 0 4

Years of Use JEDEC

SSD

Std.

<=3%

Be aware of Tools & Best Practices … And you should be OK !!

Concerned about SSD

Adoption in your Enterprise ?

Using Best-of-Breed Controllers to achieve <=3% AFR and JEDEC

Endurance Verification Testing should allow Enterprise Capabile SSDs

Source: Intel IDF’10 & IMEX Research SSD Industry Report 2011 ©IMEX 2010-12

Page 19: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

19 19

I/O Forensics for Auto Storage-Tiering

Source: IBM & IMEX Research SSD Industry Report 2011 ©IMEX 2010-12

LBA Monitoring and Tiered Placement • Every workload has unique I/O access signature • Historical performance data for a LUN can

identify performance skews & hot data regions by LBAs

Storage-Tiering at LBA/Sub-LUN Level

Storage-Tiered Virtualization

Physical Storage Logical Volume

SSDs

Arrays

HDDs

Arrays

Hot Data

Cold Data

Automatic

Migration (Policy Based)

Page 20: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

20

AutoSmart Storage-Tiering SW:

Enhancing Database Throughput

Source: IBM & IMEX Research SSD Industry Report 2011 ©IMEX 2010-12

• DB Throughput Optimization

– Every workload has unique I/O access signature and historical behavior

– identify hot “database objects” and smartly placed in the right tier.

– Scalable Throughput Improvement - 300%

– Substantial IO Bound Transaction Response

time Improvement - 45%-75%

• Productivity (Response Time) Improvement

– Using automated reallocation of hot spot data to SSDs (typically 5-10% of total data), significant performance improvements is achieved :

• Response time reduction of around 70+% or

• IOPS increase of 200% for any I/O intensive loads

– Verticals benefitting from Online Transactions :

• Airlines Reservations, Investment Banking Wall St. Stock Transactions Financial Institutions Hedge Funds etc. plus Low Latency seeking HPC Clustered Systems etc.

Page 21: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Big Data – Driving New Storage

Data Stored by Large US Enterprises

Big Data Storage Potential Data Stored by Large US Enterprise

14%

12%

10%

10%

9%

6%

6%

6%

5%

4%

4%

3%

3%

3%

2%

2%

1%

Discrete Manafacturing

Government

Communications and Media

Process manufacturing

Banking

Health Care Providers

Securities & Investment Srvcs

Professional Services

Retail

Education

Insurance

Transportation

Wholesale

Utilities

Resource Industries

Consumer and Recreational

Services

Construction 967

1,312

1,792

831

1,931

370

3,866

278

697

319

870

801

536

1,507

825

150

231

Big Data Cloud Storage Potential Data Stored by Large US Enterprises

Stored Data by Industry (in US 2009 PB)

Stored Data TB/Firm (>1K Employees US)

Page 22: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Harnessing Big Data

for Business Insights

Majority of data growth is being

driven by unstructured data

and billions of large objects

Information is at the center of

New Wave of opportunity

80% of world’s data is unstructured

driven by rise in Mobility devices,

collaboration machine generated data.

Data

Sources

Big Data

Infrastructure

Business

Insights

Page 23: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Type of Data Organizations Analyze Types of Data Organizations Analyze

59%

44%

69%

64%

41%

33%

51%

28%

46%

36%

36%

18%

18%

21%

8%

3%

3%

68%

68%

37%

23%

33%

34%

26%

32%

21%

15%

11%

15%

11%

9%

3%

6%

5%

Customer/member data

Transactional data from applications

Application Logs

Other Types of Event Data

Network Monitoring/Network Traffic

Online Retail Transactions

Other Log Files

Call Data Records

Web Logs

Text data from social media and online

Search logs

Trade/quote data

Intelligence/defense data

Multimedia (audio/video/images)

Weather

Smartmeter data

Other (please specify)

Hadoop

Non Hadoop

Page 24: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

24 24

Data

: IM

EX

Researc

h &

Panasas

Smart Mobile

Devices

Commercial

Visualization

Bio/Health

Analytics

Predictive Bus.

Intelligence Entertainment-

On Demand

Instant On Boot Ups

Rugged, Low Power

1GB/s, __ms

Rendering (Texture & Polygons)

Very Read Intensive, Small Block I/O

10 GB/s, __ms 4 GB/s, __ms

Data Warehousing

Random IO, High OLTPM

Most Accessed Videos

Very Read Intensive

10 GB/s, __ms

NextGen Drivers: Big Data Cloud

Page 25: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Big Data – Architectural Goals

Big Data

Platform

Meet Enterprise Criterion Meet Requirements of V3

Analyze Data in Native Format

Page 26: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Rise of Big Data Analytics

Page 27: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Demand for Real Time Analytics

Page 28: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Technology Innovations - HW

Page 29: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Technology Innovations: DB SW

Tech Innovation 1985 1990 1995 2000 2005 2010 2015

OLTP Transactions DB SW

Rows Locking Optimizer Parallel Query Clustering XML Grid OpenSource /

Hadoop

OLAP- Analytics DB SW

Indexing Partitioning Columnar Materialized

View Bit Mapped

Index In-Memory Query Binding

Hardware 32 bit SMP NUMA 64 bit Multi-core/ Blades

Flash MPP

Big Data Multi-core Columnar In-Memory MPP

Visualization

OLTP Database Innovation Progress

0.1

1

10

100

1000

10000

100000

1985 1990 1995 2000 2005 2010 2015

0.01

0.1

1

10

100

1000

10000

$/TPMc

TPMc/Processor

$ / T

PM

c

TP

Mc

/ P

roc

es

so

r

Big Data: Analytics DB Technology Impact

0

0

1

10

100

1,000

10,000

100,000

1985 1990 1995 2000 2005 2010 2015

0.0001

0.001

0.01

0.1

1

10

100

1000

10000

100000

.1

.01

DW Size TB $/GB

Page 30: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Big Data – Product Segments

Big Data - Product Metrics

Data Set Size

PB

TB

GB

Data Structure

Transaction

Machine

Unstructured

Other

Access/Use Transaction

Search

Analytics

Parallel Processing

Appliance

Cluster < 1K

Cluster > 1K

Memory In-Memory

Flash

DB Technique Columnar

Zero Sharing

No SQL

Data Cataloging SW

Text

Image

Audio

Video

Page 31: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Big Data Ecosystem

Page 32: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Big Data Stack

Servers

Operating System

Hypervisor/VMs

Big Data Storage Framework (HDFS)

Big Data Processing Framework (MapReduce)

Big Data Access Framework

Pig Hive Sqoop

Big Data (Connectors)

Big Data Orchestration Framework

HBase Avro Flume ZooKeeper

BI APPLICATIONS (Query, Analytics, Reporting, Statistics)

EDW B

ac

ku

p &

Re

co

ve

ry

Ma

na

gem

en

t

Se

cu

rity

Network

BI Framework - Interoperable with Enterprise Data Warehousing

Page 33: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Types - Benefits & Risks

Private Cloud (On Premises)

Public Cloud (Remote)

Hybrid Cloud (Co-Oper)

Charact- eristics

IT acts as provider of services

Available to Internal Users only

Bidirectional Scalability

Reservationless Auto Allocation of Resources

Pay/Use – Metering Mechanism

Application & SW Agnostic

Owned and delivered by 3rd Party Application on User Side & App Services

and Resources on Cloud Srvcs Provider side Cloud Provider sells Srvcs to anyone, Data stored in Shared Facility serving multi-

tenants - impact on Perf., BW, & Availability of Apps served.

Agreed upon Cooperation between OnPremises user owned by User and Cloud Services Provider (CSP) on Cloud Side

Onsite Gateway Appliance with abstraction layer between usr location & public cloud site of CSP

Benefits

Instant Provisioning Agility Full Physical Control & Security +

User’s access to Infrastructure Customizable Srvcs & Solns to meet

User’s specific Needs Predictable & Lower Costs solutions

for large/midrange sites

Lower Costs using Shared Offsite Stg Instant Provisioning

Infinite Scalability Efficient Data Sharing by user particularly if

info shared by users dispersed in multi-locations

Storage Costs CAPEX >> OPEX

Improves Performance. by serving Hot I/Os On-Premise

Deduped Data sent to Cloud > Lowers reqd. BW & Monthly Networking Costs

Only Encrypted Data Sent > ensures security for MC Apps

Risks

• Demands extra capacity be available for unforeseen demand in provisioning and management

• Building Private Cloud Infrast. difficult, requires new expertise

• New Cos.now moving as Private Cloud Infras. Builders vs CSPs

• Requires Monitoring, DR Solns, Change Mgmt, Testing, SLA/ Charge Back mechanisms etc.

SLA managed as outsourced contract Full understanding of Performance,

Availability, Costs required per App before migration

Shared Infrastructure awareness for sensitive data despite encryption

Infrastructures Failures/Hacked SensitiveData Exposure visible in Press

Cloud Provider can abruptly exit Biz Lower HA than achievable in Pvt Cloud

Page 34: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Storage Requirements

Expect more from your storage

Hyper-efficient Storage

Storage that analyzes, adapts, and improves performance

Cloud ready storage, with features you can use today

Technology & Best Practices enable storage efficiency

Real-time Compression … up to 80% less space

Auto Tiering … 3x more performance with just 2% solid-state

Storage Virtualization … up to 30%more utilization

Thin Provisioning … up to 35%more utilization

Data Protection/HA:

Fileset level snapshots and writeable file clones

Capability for remote mirror

File replication and file level snapshots for business continuity and DR

Manageability:

Unified SAN & NAS (block and file) storage system with a tightly

integrated management console

Support for NFS/CIFS/FTP/HTTPS/SCP file protocols

Storage administration with a single user interface

Policy-based management of files with user-defined policies!

Page 35: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Storage – Issues & Solns.

Traditional

Data Growth

Sto

rage C

osts

Reduction

Capacity Requirements

Snapshots ~ 75%

Thin Provisioning ~30%

DeDuplication ~ 25-95%

Auto-Tiering 65-95%

Thin Replication ~ 95%

RAID*DP ~ 40% vs R10

Virtual Clones ~80%

CAPACITY

SAVINGS

~ xx %

Virtualization (VZ) requires Shared Storage for

- VMotion

- Storage VMotion

- HA/DRS

- Fault Tolerance

Additional Capacity Consumed for

- VZ snapshots,

- VM Kernel etc

Following techniques Reduce Storage Costs

Page 36: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Storage Requirements

Thin Provisioning & Others Simplifying the increase capacity utilization

Automatic avoidance of disk hotspots

Extent-based volume management

Efficient cache management at disk system (controller & disk arrays)

Optimized performance through automated data placement

Dynamic QoS management

Efficient Storage Back Up Automated DR Replication

Page 37: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Private Cloud - Requirements Public

Cloud

Storage

Costs

Opera

tional F

lexib

ility

App

Silos

VZ

Private

Cloud

Storage

Automation Automated Provisioning

- Moving Data & Processes Seamlessly

Allocation, Self Tuning of Resources to

meet Workload Requirements

Self-

Service Access Resources on demand to

speed deployment and delivery

Scale Resource Up/down

to optimize their usage,

Release when not needed

Service Catalog Choose pre-defined IT Services/user/dept.

Define SLA to efficiently meet services

Service

Analytics Monitor & Analyze Usage for

Charge back

Interactively auto-tune

performance

Availability with SLA

requirements

Private

Cloud

Storage

Page 38: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Service Catalog by User/Workload Service Catalog by Workloads

Class Metric

Storage Initial Capacity TB

Storage Protocol (NFS/SCSI …)

Availability Downtime/Year (Max Unplanned) Mts

Copy Retention Time Hrs/Yrs

Operational Recovery

RPO Recovery Point

RTO Recovery Time

Recovery Consistency

Recovery Assurance %

Min # Recovery Points

Disaster Recovery

RPO Recovery Point

RTO Recovery Time

Recovery Consistency

Recovery Assurance %

Min # Recovery Points

Performance I/O Response Time ms (Max/Avg)

IOPs (Min/Assured)

Bandwidth MB/sec (Min/Assured)

Security

Encryption

DLP

Copy Retention Time Hrs/Yrs

Copy Destruction Method -Quick/Secure Erase

Authencity Guarantee

Page 39: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

Cloud Storage - Requirements

Enable Policy Based Efficient File Management

Locally

- Identify files for backup or replication

- Move to right tier of storage hierarchy including cloud

- Delete expired or unwanted files

Globally

- Localize data

> Improve file access performance and reduce network costs.

- Allow multiple sites to collaborate on information exchange

- Virtualize to a single/global namespace

> Multiple users can view same files from universal locations

Collaboration in Cloud

- Local or long distance users sharing a dedicated home file

System but with individual home directory.

- Each site/system has own file set at their local cluster.

-Data Center has all Home dirs and manages BUs/hsm

- Remote sites periodically prefetch (via policy) or pull on demand

- Data Center maintains BUs & R-kive Copy of data.

- Each location maintains logs/records for that Geography

Data

Center

Locally Globally

Page 40: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

IT Infrastructure in the Cloud

SLA

Usage

Dept/User

Assets

Location • Cloud Services Hosting

• Platform – OS/Processors/#/Speed/Type

• Pooled Infrastructure Resources by

Application Metrics

• Pooled Capacity Provisioning:

Processing, Bandwidth, Storage, Repository

• Usage Profiles

• Users/Services/Workloads

• Applications (OLTP/BI/HPC/Data Streaming)

• Execution: Rules Driven, Adaptive Provisioning

• Services Abstraction, Adaptive Provisioning

Cloud

IT Services

Provider

• Business Priorities

• Cost of IT Ops/Charge Back Methods

• Response Time/Availability/Throughput, QoS

• Transactions/Sessions/Events/Analysis/Reporting

• Business Services Managed & Charged

For copy of case study on how a major financial institution implemented Cloud Infrastructure email [email protected]

Page 41: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack

41

Key Takeaways

• Cloud Storage is increasing being adopted

– IT-as-a-Service is being delivered through a virtualized, Shared IT infrastructure, automation and Service Efficiencies using Private and Hybrid Cloud Infrastructure

– Savings through Storage Efficiency using automation that is being integrated

– Due diligence in careful planning, a three year roadmap, selection and testing of vendors/products required before starting migration from existing DC operations

• IT Infrastructure to meet needs of your Business Applications/SLA – Let your business needs determine your Cloud Strategy.

– Identify and define specific SLA metrics per user/dept/division.

– Strategize for Long Term but take incremental approach to implementing Clouds

– Identify and screen promising Cloud Service Providers – their Enterprise Class, Security and Compliance adherence, Change Controls, Referenced High Availability & Performance Results.

• Be aware of technology and best practices available – Engage the expertise of consultants and system integrators to validate your strategy and

implementation plans

Page 42: Cloud Storage: Infrastructure - etouches€¦ · Agenda (1) The Need for Cloud Storage Data Centers Infrastructure IT Roadmap to Clouds Cloud Computing Ecosystem & Architectural Stack