ibm object storage and software defined solutions - cleversafe

IBM Object Storage and Software Defined SolutionsKarel van der Woude

IBM Systems

Digital Content Growth

• Continued growth in graphical content creation

• Multi-device and HD/4K/8K make it even more challenging to store and process data

• Time & location shifting: viewing on individual schedule

• Content life-cycle management provides a balance between cost & performance while maintaining customer experience

• Meta data availability and access

• Digital disruption with Over The Top (OTT) digital only competitors - new business models

• OTT viewing will grow from representing 3.4% of TV viewing hours in 2013 to 20.4% by 2017 in NA

• 63% stream on-demand media more than weekly

• Security and data protection are big issues.

• File sharing declines with legal on-demand options

• Connection speeds are increasing making new options possible

IBM Systems

Exponential Data GrowthBusiness Value in Unstructured Data

Projected Exabytes

0

20

40

60

80

100

120

2009 2010 2011 2012 2013 2014 2015 2016 2017

Unstructured content

Structured data

�Source: IDC

This creates a need for

A Cost Effective

Scalable Repository

Analytics

Collaboration

Content

Distribution

Active Archive

Unstructured data growth of

60–80%per year

creates Web-scale

storage needs

IBM Systems

Why Object Storage?

IBM Systems

File Storage vs. Object Storage

Object StorageFile Storage

Flat namespace - no hierarchy - unique object id.

Designed to serve applications.

Protocols to access objects storage are HTTP REST based, which include, for a dsNet® CS S3, OpenstackSwift and Cleversafe’s proprietary SOH.

At no point does the number of objects stored become so large that it cause performance to decline.

Designed for WAN.

Hierarchical structure - directory, directory, file, directory......

Designed to mimic human data behaviors.

Protocols to access file storage are primarily NFS or SMB/CIFS.

There is a point at which the number of files and directories in a single namespace begin to impact performance in a negative way.

Designed for LAN.

File Storage was designed for storing data in the 80’s. Object Storage is designed for data storage today and beyond.

IBM Systems

New Generation of Storage

Next Generation Storage Required

• Must scale

• Must be more efficient

• Must be available always

• Must be geographically accessible

• Must protect data integrity and security

Legacy Storage can´́́́t keep up

• Scale beyond PB

• Data protection relies on replication

•4 TB Drives takes 10 hours plus

•RAID rebuild of a 20 TB drive, 7,400 - 15,000 rpm, will take many days.

IBM Systems

Legacy Storage at Scale

RAID 6 + Replication

�Original

1.20 PB Raw

Onsite mirror

1.20 PB Raw

Remote copy

1.20 PB Raw

1 PB

3.6 PB

900

3.6x

3.6x

3 FTE

Replication/backup

Usable Storage

Raw Storage

4TB Disks

Racks Required

Floor Space

Ops Staffing

Extra Software

1 PB

1.7 PB

432

1.7x

1.7x

.5 FTE

None

Cleversafe®

�$�$�70% +�TCO Savings

567 TB Raw 567 TB Raw567 TB Raw

IBM Systems

$0

$2,000,000

$4,000,000

$6,000,000

$8,000,000

$10,000,000

480 TB 960 TB 1920 TB 3840 TB

S3

dsNet

5 Year TCO comparison by Cleversafe

Amazon S3 published prices and capacity discounts (pricing as of 10/31/14, assumes 20% reads)

$/TB comparison

Analysis by Cleversafe customer

Cost: 29–61%+ lower

$8,400

$4,210

$1,613

$1,053

Current NAS DRProtected

Current NAS Single Copy

NAS Gateway and dsNet

dsNet Object Protected

Cleversafe vs Amazon S3 Cleversafe vs NAS

Cost: 80%+ lowerEconomics

Cleversafe Platform Economics

IBM Systems

How are Customers Using Cleversafe?

IBM Systems

Archive

Backup

Content Repository

Storage as a Service

Enterprise Collaboration

Solution AlliancesEMAIL

HPC & IRODSFILE

HSM/ILMSHAREPOINT

CALL CENTER

VIDEO

HDFS HADOOP

BACKUP TO CLOUD

APPLIANCE REPLACE

TAPE REPLACEMENT

NAS REPLACEMENTHPC

DOCUMENT

MANAGEMENT

BIG DATA ANALYTICS

MUSIC/CALL CENTER

VIDEO SURVEILLANCE

MACHINE GENERATED

DATA

ARCHIVE

BACKUP

CONTENT REPOSITORY

ENTERPRISE

COLLABORATION

PUBLIC TO PRIVATE

CLOUD

CLOUD DVR

FILE SYNC & SHARE

REMOTE OFFICE ROBOCONTENT DISTRIBUTION

FILE SHARING

IBM Systems

Single, Centralized, Multi-Purpose Repository

�Exchange Email �Sharepoint �NFS File Servers �DatabaseBu

sin

ess

Ap

plic

atio

ns

�Audio/VIdeo

Data Management Software

Email Sharepoint File Database Multimedia Content

So

ftware

L

ayer

�Documentum, Filenet

Nearline performance

No backup required

No limit to scale

No mirroring

No replication

No downtime

DR built-in

80% cost reduction

Clo

ud S

tora

ge

an IBM Company

Technical

© 2015 IBM Corporation13

Innovation Delivered: Hard

‘08 Petabyte Excellence

‘09 Namespace Registry

’08 No downtime ever in production

‘10 Upgrade software with no downtime.

’10 Refresh hardware with no downtime

’12 Continue writes and reads during node outage

‘11 Continue writes and reads during site outage

Scalability

Reliability

‘06 Slice Rebuilder

‘07 Perfect Bits Data Integrity

‘07 Sixteen 9s

‘09 Strong Data Consistency Guarantees

‘10 Multi-Fault Rebuild

Confidentiality

‘07 KeyInside Threshold Encryption

‘13 Secure Multi-tenancy

‘13 LDAP

Administration

‘08 HTTP/REST Manager Interface

‘11 Zero Downtime Adds, Moves, Changes

’12 Zero Downtime Upgrades

‘12 One Admin per 25PB

Integration

‘12 OpenStack Swift

‘12 Hadoop HDFS

’12 Partner Solution Network

‘12 S3 Interface Support

Economics Security Manageability

Life-Cycle Cost

‘05 Erasure Coding

‘08 Commodity H/W

‘10 Configurable Efficiency

‘14 3.3 Watts per Usable TB

‘14 Small file support

System Scale

‘08 Petabyte Excellence

‘09 Namespace Registry

‘09 No-Database Address

‘10 Stateless Accesser

‘10 Distibuted Multi-Writer

’12 10,000 PB Support

Future: Zettabyte Scale

Availability 10 YearsOver 100 person

dedicated development and

Q&A team.


CS Manager

• Fault management• Reporting• Provisioning• Performance monitoring• Storage configuration

Accesser®

• Slices data• Disperses data• Retrieves data• Stateless

Slicestor®

• Storage for slices• Single site or multi-site• Capacity based pricing• Rebuilds slices

Supported on Cleversafe Certified industry standard platforms

Software Defined &Hardware Aware

Qualified for predictable performanceFaster time to productionChoice of drive technologySingle Pane Management of HW & SW

Cleversafe Software Platform


CONTENT TRANSFORMATION

Cleversafe software

encrypts, slices and

applies Information

Dispersal Algorithms

otherwise known as

erasure coding policies

to the data.

Data Ingest

Accesser Software

Storage Nodes

Site 1 Site 2 Site 3

Physical Distribution

Slices are distributed to separate disks and industry standard x86 hardware across geographic locations.

Data Retrieval

Storage Nodes

Reliable Retrieval

An operator defined subset of slices is needed to retrieve data bit perfectly in real time.


BENEFITS

The level of resiliency

is fully customizable

resulting in a massively

reliable and efficient

way to store data at

scale as opposed to

RAID and replication

techniques.

Slicestor Software

How the Cleversafe Platform Works

Accesser Software


�Original Data �Encrypted Data

�Writing Data to Cleversafe®

�Original object is encrypted�Original object is encrypted�1�$


�Original Data�Encrypted Data

�Writing Data to Cleversafe®

�12�11�10�9�8�7�6�5�4�3

�1�2

�Erasure Coded Slices

�7�6�5�4�3

�1�2

�Erasure coding is used to transform the data into a customizable number of slices

�Erasure coding is used to transform the data into a customizable number of slices

�2

�$


Data Security In Cleversafe DNA

Source: In-Q-Tel

Source: Computer World


S3 Compatible API Openstack Swift Compatible API

Simple Object API

Slicestor Software

Accesser Software

Security - Encryption for data at rest provides government grade security, no single disk, node or site contains enough information to constitute a data breach with built in key management.


Availability

Availability – No downtime during software upgrades, hardware refreshes, and in the face of disk, node, and site failures.


S3 Compatible API Openstack Swift Compatible API

Slicestor Software

Accesser Software

Simple Object API

Recognized Industry leader

Proven enterprise software defined storage enables efficient, secure storage at petabyte scale and beyond

�Company

• Chicago HQ and R&D

• 200+ employees

• 300+ patents

�Customers

• Commercial/government/intelligence

• Global 1000 or massively data-intense

• US, Japan, UK, France, Netherlands

Select customers

�Partners

• Density-optimized hardware

• Secure cloud storage

Select partners

Cleversafe Momentum

�Service Providers

�Media and Entertainment �Life Science, FinServices, Marketing �Federal

ibm object storage and software defined solutions - cleversafe

Technology