christian.bandulet@oracle - doag.org · [email protected] the following is intended to...

[email protected]

<Insert Picture Here>

Evolution und Trends der Storage TechnologienChristian BanduletPrincipal Engineer

[email protected]

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

[email protected]


Agenda

•Data•Storage Systems•Data Preservation•Storage Grids•Storage Networks•Storage Media

[email protected]


Evolution of Data

[email protected]

IT Challenges Have Not SlowedIncreasing Demand for Storage Capacity and Performance

2006 New Digital Data(Created, Captured, Replicated)

2010 New Digital Data(¼ Created, ¾ Replicated)

6X Growthemerging economies

growing 30-40% faster than mature economies

161 Exabytes

988 Exabytes

70% created by Individuals

Enterprises responsible for85% of this new data

(Security, privacy, reliability, compliance)

95% U

nstructured

Source: IDC

[email protected]

Challenge: Storage Needs Are GrowingGreater Demand For Storage Capacity And Performance

2009 New Digital Data 2020 New Digital Data

44X Growth

800 Exabytes35,000 Exabytes

Source: IDC

[email protected]

Storage Market Segmentation

Media production,eCAD, mCAD, Office docs

Media-archive, DAM, Broadcast, medical imaging, Media-Internet

Transactional systems, ERP, CRM BI, data warehousing, scientific, transaction archive

Fixed DataDynamic Data

Stru

ctur

edU

nstru

ctur

ed

Time

[email protected]

Storage Market Segmentation

Media production,eCAD, mCAD, Office docs

Media-archive, DAM, Broadcast, medical imaging, Media-Internet

Transactional systems, ERP, CRM BI, data warehousing, scientific, transaction archive

Fixed DataDynamic Data

Stru

ctur

edU

nstru

ctur

ed

SemiStructurede.g. email

Time

[email protected]

Data

Metadata Owner, permissions, type, last mod. date, ...

Data For Clouds – File Objects

[email protected]

Data vs. Information

Data + Semantics = Information

+

[email protected]

Object ID

Data


Attributes user/application defined

policies e.g. replication

methods e.g.encryption


[email protected]

Object Attribute 1

Object Model

Object Attribute 2

Object Attribute 3

Object Attribute 4

Object Attribute n

ObjectDescriptor

OID

many:1 1:many

ObjectSub-Component 1




ObjectSub-Component n

[email protected]

Object ID

Data



Policies e.g. replication

Methods e.g.encryption


Store Retrieve

ObjectObject

Data OID OID Data

[email protected]

Object ID

Data



Policies e.g. replication

Methods e.g.encryption

Store Retrieve

Data Blocks

Inode

name OID Object

name OID Object

name OID Object

name OID Object

name OID Object


ObjectObject

Data OID OID Data

[email protected]


Evolution of Storage Systems

[email protected]

Traditional Storage Architecture

SAN

IP Clients

ServerServer CPU& Memory Complex

SCSI Port to OSiSCSITCP/IP

Adapter Driver

HW RAIDController

HW RAIDController

[email protected]

Server

Storage

The Vertical Consolidation

Storage

Server

StorageStorage

ServerServer

Storver

[email protected]

Programmable Storage

Data OID

Store

Object

Input Procedure 1

Input Procedure 2

Input Procedure x

...

Asyn

chro

nous

Pro

cedu

re C

Asyn

chro

nous

Pro

cedu

re D

Asyn

chro

nous

Pro

cedu

re A

Asyn

chro

nous

Pro

cedu

re B

OID Data

Object

Output Procedure x

Output Procedure 2

Output Procedure 1

...As

ynch

rono

us P

roce

dure

C

Asyn

chro

nous

Pro

cedu

re D

Asyn

chro

nous

Pro

cedu

re A

Asyn

chro

nous

Pro

cedu

re B

• User-defined/developed storage based apps– Synchronous:

• modify the behavior of store/retrieve/query/delete

• e.g. transcode, downsample, filter, watermark, extract metadata from file, headers, encrypt, audit log...

– Asynchronous:• process data at rest• e.g. capacity optimization, de-

dup, scrubbing, migration, sanity check...

Retrieve

[email protected]

Application-Awareness

Server

Application Context

Application

Storage Device

• Application Context:– I/O patterns, data layout, intrusion detection, data

relationships, consumer information, spatial, etc. – e.g. which blocks are stale/empty or belong to a file– e.g. relationships among data, users and applications

Data

[email protected]

Storage Device

Application Context

Application-Aware Storage

Server

Application Context

Application

• Data layout, caching/pre-fetching strategies, B&R, ILM, DR, security, power mgmt, device-resident de-mapping functions, transport optimization, resource management

• Appropriate protocol required to exchange context information

• device-resident de-mapping / devirtualization (virtualization interferes with performance tuning and problem diagnosis

Data

[email protected]

Active Storage – Distributed Data ManagementApplication Decompositioning

Storage Device

Application Context

Server

Application Context

Application

• Runs application-specific code– e.g. indexing, search, caching

• Enables adaptive-computing

Application

Data

[email protected]

The Intelligent Storage Node

Presentation Layer (Storage Personality)Block I/F File I/F Object I/F

SCSI, FCP, iSCSI, SAS, SRP, SATA, VTL ...

FTP, NFS, CIFS, WebDAV, HTTP...

API, ANSI T10 OSD, CAS, XAM...

Storage Application Layer ( Data Services)Data Management Compliance Services Scheduled Services

Backup, Classification,Data Mining, Application

Acceleration, Aggregation, Virtualization,

Sort, Views, Search,Indexing, Data Movers,

Distributed Namespace...

WORM, Audit, Retention, Secure Deletion...

De-Duplication,Hash Generation,

Verification,Snapshot, Replication...

Operating SystemFiles systems, HSM, Device Drivers, Clustering, Security....

Persistent StorageDisk, Tap, Library...

External ServicesLDAP

NIS

Active Directory

NTP

MDSGlobal Namespace

others

[email protected]


Evolution of Data Preservation

[email protected]

Information Lifecycle Management

Publish

Create

Index

Store

Retain

Cleanse

Search

Distribute

VersionCapture

Manage

Secure

Destruction

Archive

Capture data from any entry pointWeb-based submissionIngest directly from applicationsScan, Fax directly to repository

[email protected]

Green...

SATA, MAID...

[email protected]

Greener...

[email protected]

1956 first spinning hard drive (IBM RAMAC)

1956: 5MB – 2000bits/in2

2010: 2000GB ~ 400Gb/in2

200,000,000x areal density

400,000x capacitytimes improvement in 50+ years!

Disk Evolution

[email protected]

Even More Green...

Data De-DuplicationData ClassificationData Compression

[email protected]

Greenest...

Data Deletion

[email protected]


Evolution of Storage Grids

[email protected]

Storage EvolutionFrom Silos to Storage Networks and Shared Storage

Application ServerApplication Server Application Server Application Server Application Server Application Server Application Server

Storage Network

[email protected]

Storage EvolutionFrom Physical Storage to Virtualized Storage and Storage Pooling

Application Server Application Server Application Server Application Server

Storage Pool

Shared FS Shared FS Shared FS

Application Server Application Server Application Server

Storage Network

Device SharingShared Storage

Device

Shared FS Shared FS Shared FS

LUN LUN LUN LUN

VLUN VLUN VLUN

Storage Network

[email protected]

Storage EvolutionHSM: from Single-Tier Storage Pools to Multi-Tier Storage Pools


Tier 1

Storage Pool

Tier 2 Tier 3

VLUN VLUN VLUN


Storage Pool

VLUN VLUN VLUN

TAPE

Storage-Based:Dynamic policy-based migration & Stubbing

Storage Network Storage Network

Host/Network-Based:Dynamic policy-based migration & Stubbing

LUN LUN LUN LUN LUN LUN

[email protected]

Storage EvolutionFrom HSM to ILM

Tier 1

Storage Pool

Tier 2 Tier 3

LUN LUN LUN

VLUN VLUN VLUN

TAPE

Storage-BasedDynamic policy-based migration

+ backup/recovery+ regulatory compliance (WORM, retention)+ archiving (classification, indexing, search)

Tier 1

Storage Pool

Tier 2 Tier 3

LUN LUN LUN

VLUN VLUN VLUN

TAPE

Application Server Application Server Application Server Application ServerApplication Server Application Server Application Server

FCP/iSCSI Storage Network FCP/iSCSI Storage Network

Host/Network-Based:Dynamic policy-based migration & Stubbing

Storage-Based:Dynamic policy-based migration & Stubbing

Host/Network-BasedDynamic policy-based migration

+ backup/recovery+ regulatory compliance (WORM, retention)+ archiving (classification, indexing, search)

[email protected]

Storage EvolutionFrom Block Storage to Object Archive Storage (CAS)

Tier 1

CAS

Tier 2 Tier 3

LUN LUN LUN TAPE

Dynamic policy-based migration+ backup/recovery

+ regulatory,compliance (WORM, retention)+ archiving (classification, indexing, search)


Tier 1

Storage Pool

Tier 2 Tier 3

LUN LUN LUN

VLUN VLUN VLUN

TAPE

Dynamic policy-based migration+ backup/recovery

+ regulatory,compliance (WORM, retention)+ archiving (classification, indexing, search)


XAM WebDAV Object ... FTP

Object Object Object Object Object Object

FCP/iSCSI Storage Network FCP/iSCSI/NFS/CIFS

Semantics/Attributes

[email protected]

Storage EvolutionFrom RAID to RAIN (homogeneous)

Application ServerApplication Server Application Server Application Server

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

CAS

RAIN

Application ServerApplication Server Application Server Application Server

FCP/iSCSI/NFS/CIFS Storage Network

[email protected]

Storage Grid & Data AvailabilityData Placement

Storage Node Storage Node Storage Node Storage Node




Object 1

= Data

= Parity

Object 2

Object 3

Storage Grid

[email protected]

Resource Lifecycle Management

Storage Grid

Svc1 Svc1 Svc1Svc1 Svc2Svc2 Spare

Storage Grid - expand

Svc1 Svc1 Svc1Svc1

SpareNew

Storage Grid - repurposed/new services

Svc1 Svc1Svc1

Old

Svc3 SpareNew

New

Minimal application impactExpand a Storage Grid

Install an elementAuto-discover, bindProvisionRedistribute workload

Decommision elementsDrain dataDe-presentUnbindRetire and de-install

Also applies to replacements/upgrade

[email protected]

Host Host Host Host Host Host Host Host

IB FC IP

I/O RoutingI/O Redirection

Single Storage ImageISD ISD

ISDISD

RemoteIB

Global Namespace

[email protected]

NIST Definition of Cloud Computing

[email protected]


Evolution of Storage Networks

[email protected]

Today’s Infrastructure

FC-SAN Network

Ethernet Management

Ethernet Data Network

StorageServers

[email protected]

Ethernet 1, 10 Gbps

IPTCP

iSCSI

InfiniBand

SRP

IPTCPFCIP

FCP*

IPTCPiFCP

FCP*

Fibrechannel

FCP*

1, 2, 4, (8), 10 Gbps 10, 20 Gbps

iSCSI FCoE InfiniBand

iSER

iSCSI

Data Center Ethernet

FCoE

FCP*

SCSI Layer

Operating System / Applications

(1), 10 Gbps

* Includes FC Layer

Encapsulation Technologies

Source: Cisco

[email protected]


Evolution of Storage Media

[email protected]

1952 first commercial tape (IBM 726)

1952: 2MB - 1400bits/in2

2010: 1500GB* - 1.21Gb/in2

865,000x areal density

750,000x capacitytimes improvement in 50+ years!

Tape Evolution

*LTO-4 0.8TB, ~ 0.8 Gb/in2

*LTO-5 1.5TB, ~ 1.21 Gb/in2

[email protected]

• Moore’s Law has out-stripped improvements to disk drive technology by two orders of magnitude during the last decade

• Today’s servers are hopelessly unbalanced between the CPUs need for data and the HDDs ability to keep up

HDD vs. CPU Relative Performance Improvement

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Server and Storage Disconnect

[email protected]

Where to Deploy Flash Storage?External Controller

Application

OS

Memory

IO

HBA/NIC/HCA/CNA

PCIe CTR

L

StorageNetwork

DR

AMD

RAM

DR

AMD

RAM

DRAM

Fmod

SSDFmod

SSD

Fmod DRAM

Fmod

[email protected]

Where to Deploy Flash Storage?External Controller

[email protected]

Where to Deploy Flash Storage?PCI Controller

HBA/NIC/HCA/CNA

Fmod

Fmod

Fmod

Fmod

Fmod

Fmod

Fmod

Fmod

Application

OS

Memory

IO

PCIe CTR

L

StorageNetwork

DR

AMD

RAM

DR

AMD

RAM

DRAM

Fmod

SSDFmod

SSD

Fmod DRAM

Fmod

Fmod

[email protected]

Where to Deploy Flash Storage?PCI Controller

[email protected]

Where to Deploy Flash Storage?Memory Controller

CTR

L FmodFmodFmodFmod

DDR2/3

Application

OS

Memory

IO

HBA/NIC/HCA/CNA

PCIe CTR

L

StorageNetwork

DR

AMD

RAM

DR

AMD

RAM

DRAM

Fmod

SSDFmod

SSD

Fmod DRAM

Fmod

Fmod

[email protected]

Where to Deploy Flash Storage?Application-Managed Hybrid/Secondary Cache

HBA/NIC/HCA/CNA

Fmod

Fmod

Fmod

Fmod

Fmod

Fmod

Fmod

Fmod

Application

OS

Memory

IO

PCIe CTR

L

StorageNetwork

DR

AMD

RAM

DR

AMD

RAM

DRAM

Fmod

SSDFmod

SSD

Fmod DRAM

Fmod

Fmod

[email protected]

Emerging SCM Technologies

[email protected]

True Dynamic Tiering

What ?: LUN, Volume, File, Page, Block

When ?: Weeks, Day, Hours, On-The-Fly

How ?: Manual Disruptive, Manual Non- Disruptive, Policy-Based, Automatic

[email protected]

Sub-Object/File/LUN Tiering

File Page TablePhysical Address Virtual Address

X'ff1234ea X'a74efa11Tier 1

Storage Pool

Tier 2

Tier 3

[email protected]

Tiered Storage Reduces IT CostsData center best practices demand tiered storage efficiencies

Single Tier of Disk

Storage

ModernMulti-tiered

Storage

DiskMulti-tiered

Storage

100%

38%

62%

32%

6%

60%

Source: Horizon Information Strategies, 2009

Flash

PerformanceDisk

CapacityDisk

TapeDrive


Evolution und Trends der Storage TechnologienChristian BanduletPrincipal Engineer

[email protected]

christian.bandulet@oracle - doag.org · [email protected] the following is intended to...

Documents