© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

ENT301: Integrating On-Premises Enterprise

Storage Workloads with AWS

Harry Dewedoff, NASDAQ OMX

Yinal Ozkan, Amazon Web Services

November 14, 2013

What this session is not?

• Vendor feature , technology comparison

• Vendor / product discussion

• Cloud-only workloads

• Individual / retail storage options

Agenda

• Section 1: What is new with enterprise storage?

• Section 2: On-premises storage cloud integration

• Section 3: NASDAQ OMX and cloud storage – History

– Options provided to NASDAQ OMX teams

– PoC

– NASDAQ OMX technology selection

– Managing operations

– Security

• Section 4: Evaluating a sample storage workload

WHAT IS NEW WITH ENTERPRISE

STORAGE

Section 1

Storage Services

Scalable and durable

high performance cloud storage

Compute Storage

AWS Global Infrastructure

Database

App Services

Deployment & Administration

Networking

Amazon Glacier Low-cost Archive Storage in the Cloud

Amazon Elastic Block Store

Persistent Block Storage for EC2

AWS Storage Gateway

Corporate File Sharing and Seamless Backup

of Enterprise Data to Amazon S3

Amazon S3 Redundant, High-Scale Object Store

Amazon S3 Standard Storage Is…

Designed to provide 99.999999999% durability and

99.99% availability of objects over a given year.

If you put 10,000 objects in S3 you can expect to

lose 1 object every 10,000,000 years

Common Data Storage Challenges

Primary

Block

Storage

Primary File

Storage

Archival

Storage

Disk Based

Backup

Storage

Tape

Infrastructure &

Management

Replicated

Storage for

Disaster

Recovery

Offsite

Locations

Geo-

Resilience

Traditional On-Premises Solutions

Internet Web

Services API

HTTP(S)

Block File

AWS

Cloud

Customer

Data

Center

Storage

Use

Cases Archive Backup Disaster Recovery

Next Generation Enterprise Storage

AWS Direct

Connect

Next Generation Enterprise Storage Benefits

Why Next Generation Enterprise Storage

with AWS?

Next Generation Enterprise Storage Benefits

Amazon Storage Tiers (S3 RRS Glacier)

ON-PREMISES STORAGE CLOUD

INTEGRATION OPTIONS

Section 2

Cloud Data Tiering Options

• Option 1: Software Integration

• Option 2: Plain file transfer

• Option 3: AWS Storage Gateway

• Option 4: Enterprise storage gateways

Option 1: Software Integration

1. Configure on-premises backup software to use S3

2. Backup and restore directly from software

3. Backup server communicates with cloud (S3) over Internet links

4. Use software-based encryption, compression, dedupe, backup management tools

5. Check security / integrity / functionality / performance / operations / speed

Tapeless Cloud Backup

Virtual Physical Backup Server

EU West

Region

(Ireland)

US West

Region (N.

California)

S. America

Region

(Sao Paulo)

US West

Region

(Oregon)

APAC

Region

(Singapore)

AWS

GovCloud

Region (US)

Japan

Region

(Tokyo)

US East

Region (N.

Virginia)

Option 2: Plain File Transfer

1. Store target file(s) on a file share.

2. Configure policies on target Amazon S3 buckets

3. Encrypt / compress data sets on premises

4. Transfer files via regular file transfer (Amazon S3, SFTP, SCP, FTP etc). Or use massively parallel file-transfer options

5. Retrieve encrypted file from Amazon S3 using using the same options

6. Test integrity / security / operations / performance

7. Add parallelization for performance optimization

Plain File Transfer Diagram

Encrypt Output File

Encrypted Data Is Written to on

FileShare

Customer Data Center AWS Region

Transfer / Retrieve

Encrypted File to

Amazon S3 Using

Regular File Transfer

Internet

or Direct

Connection

Create Output

1.Store Backup File on FileShare

Option 3 : Use AWS Gateway

• Integrates on-premises IT environments with cloud storage

for remote office backup and DR

• Utilizes a virtual appliance that sits in customer datacenter

• Exposes compatible iSCSI interface on front end

• Provides low-latency on-premises performance

• Asynchronously uploads data to AWS where it is stored in

Amazon S3 as Amazon EBS snapshots

Option 3 : Use AWS Gateway

• Support for Windows and RedHat iSCSI initiators

• Point-in-time snapshots accessible locally and from Amazon EBS

• Encryption via SSL and Amazon S3 server side encryption

• Snapshot scheduling

• WAN compression

• Supported in all public regions

• Bandwidth throttling

• CACHED VOLUMES / VTL SUPPORT

Backup

Corporate File

Sharing

Cacheable data like departmental file

shares, home directories

Store files in Amazon S3, while

keeping recently accessed data on

premises

iSCSI interface compatible with traditional backup applications (Netbackup, Tivoli, Backup Exec, etc.)

Store backups in Amazon S3, keep recent backups on premises

Gateway-Cached Volumes – Key Use

Cases

Gateway-Cached Volumes Architecture

VTL Gateway – Archive Your Data to Glacier

Amazon

S3 Corporate Data Center

Amazon

Glacier AWS Cloud

App/DB/SAN/NAS

Backup

Software

VTL Gateway

Corporate Data Center

Archive to AWS

versus

Traditional

Approach

SAN Disk

Backup

Offsite Tape

Storage Tier 2

Storage

Architecture of VTL Gateway

NetBackup /

CommVault /

[Backup

Software] On-premises

Host

AWS Storage

Gateway VM

Direct Attached or

Storage Area Network Disks

(for internal cache & buffer storage)

iSCSI SSL

AWS Storage

Gateway

Service

Amazon S3

Production

Systems

AWS Storage Gateway for

Virtual Tape Library

Customer Data Center

Media Changer

Tape Drive 1

Tape Drive 2

Tape Drive N

…

Amazon

Glacier

NetBackup /

CommVault /

[Backup Software]

on EC2

AWS Storage

Gateway on

EC2 AMI

AWS Storage Gateway on EC2

for Disaster Recovery or Data Mirroring

EC2

Application

VTL (1500 tapes) VTS (unlimited

tapes)

Tape Ingestion into Glacier

VTL Gateway Characteristics • Single point virtual appliance for archive use case and for customer in need for a

simple VTL Interface

• Each virtual appliance can manage up to 140TB in VTL (Virtual Tape Library) but unlimited in VTS (Virtual Tape Shelf)

• Cost of each appliance could be around $125

• Ease of mgmt. when data grows in multi PBs per year

• Current ingest rate is about 3-5 TBs per day per gateway (option to use multiple GW in a cluster environment)

• Data passed through VTL gateway is not deduped (ease of restore and reuse) – suited for long-term archive

Bottom line: Archive, fixed content, entertainment, scientific, social networks, compliance and

unstructured data requirements generate much of today’s tier 3 storage demand and have become the

primary drivers for tape storage demand. With Amazon Glacier and VTL gateway, AWS is very well

poised to help customers leverage the benefits of cloud storage!

Option 4

Enterprise Storage Options on AWS

How Does It Work?

• Enterprise storage gateway presents itself as – CIFS/NFS file share

– iSCSI endpoint

– File archive via file tiering policies from filers

– Policy-based routing from FC switch

• Gateway cache data locally, tiers data back to Amazon S3-based on policies after dedupe, encryption, compression

• Data is accessible to all other gateways

Design Considerations

• Ingest / restore / access rates

• Deduplication / compression rates

• Throughput rates

• High availability / integrity

• Restore in the cloud option

• Data transfer costs

• Security integration

Option 4: Enterprise Storage Gateway

Corporate Data Center AWS Region

Enterprise Storage

Gateway Dedupes,

Compresses and

Encrypts Data and

Then Moves Data

to AWS

AWS Direct

Connect

Enterprise Storage Gateway

Block File

AWS

Cloud

Customer

Data

Center

Storage

Use Cases Archive Backup

Disaster

Recovery

Internet Web

Services API

HTTP(S)

AWS Direct

Connect

Amazon Glacier

Gateway Appliance/ AWS Storage Gateway

Amazon S3

• Block storage: – Data organized as an array of unrelated blocks

– Host file system places data on disk: Microsoft NTFS or Unix ZFS

– Structured data is predicted to grow at 18.7% CAGR until 2018

• File storage: – Unrelated data blocks managed by a file (serving) system

– Native file system places data on disk: EMC UxFS or NTAP WAFL

– Unstructured data is predicted to grow at 47.3% CAGR until 2018

• What is object storage?: – A new data access, data storage, and data management model

• API access to data vs. traditional block or file system access

• Metadata driven, policy-based, self-managing storage

• No host overhead for storage functions

– A system that stores virtual containers that encapsulate the data, data attributes, metadata, and Object IDs

Block vs. File vs. Object Storage

Internet Web

Services API

HTTP(S)

S3

SDK: Application developers can

leverage the Amazon S3 SDK for

custom application integration

Plug & Play: IT can bridge on-

premises environments with familiar

storage interfaces and methodologies

via cloud storage gateways

Cloud Storage: SDK or Plug & Play?

S3

AWS Storage Gateway

Example Deployments for

Enterprise Storage Gateways

NFS / CIFS Archive

• Offload stale data to low-cost cloud storage

• Scale instantly as needed

• Integrate seamlessly with standard archiving, tiering solutions

• “Cloud drive” just another disk target, accessible anywhere

Global Deduplication

Encryption

Multiple Gbps

Case Data

Analysis Data

Administrative Data

Global Online Access

Cloud Storage

NFS / CIFS Backup

• 100s TB raw local cache

• Eliminate tape from infrastructure

• Slash time and manpower for data protection

• Global deduplication

• Military-grade encryption

• Seamless integration with major backup tools

• Restore anywhere, virtual or physical

Global Deduplication

Encryption

Multiple Gbps

Global Online Access

Cloud Storage

Panzura + Amazon S3

• Eleven 9s durability

• Four 9s vailability

• Highly-secure sites

• Unlimited scale

• Commodity pricing

• Glacier option

• Multiple-geos

• Largest public cloud

▪ Global file system

▪ Military-grade encryption

▪ Global deduplication

▪ CIFS/NFS

▪ Global file locking

▪ Local caching/pinning

▪ AD integration/ACLs

▪ Snapshots

NextGen Enterprise Storage

S3 Glacier

Local file system interface to the SOAP / REST

API used by the Amazon storage cloud platform

Virtual namespace to seamlessly integrate local

and Amazon cloud storage cloud for users

1

SOAP / REST

CIFS / NFS

2

Automatically identify inactive and other

appropriate files to store in the Amazon storage

cloud

3 4

Migrate files to Amazon storage cloud platform

without disrupting user access or causing

downtime 5

Encrypt every file stored in the Amazon storage

cloud for data security

File Archive with S3

Local file system interface to the SOAP / REST

API used by the Amazon storage cloud platform

Virtual namespace to seamlessly integrate local

and Amazon cloud storage cloud for users

Automatically identify inactive and other

appropriate files to store in the Amazon storage

cloud

Migrate files to Amazon storage cloud platform

without disrupting user access or causing

downtime

Local Storage

Unified storage

• Files, databases, & VMs

• NAS & SAN

Unified Storage with Amazon S3

• Thin snapshots

• File & Bare Metal

CTERA Appliance

Mobile

Devices

LOCAL

BACKUP

NAS

• NFS, CIFS,

AFP, FTP, rsync

• AD Integration

CLOUD

STORAGE

• Pay as you go

SECURE

REPLICATION

• AES-256 + SSL

REMOTE

MANAGEMENT

• De-duplicated

• Bandwidth controlled

• RAID 0/1/5/6 • Administration

• Central logging • Automated

• Firmware updates

• Secure & redundant

Customer Location AWS Cloud

• Exchange, SQL,

AD Recovery

• Incremental • Incremental

• Thin snapshots

Roaming Laptops

Workstations

Servers

• Flexible backend

options

• Partner dashboard

• Compression

NextGen Enterprise Storage

Fast File Transfer into AWS

Supported Access

• NFS

• CIFS

• WebDav

• FTP

• Eliminate the need for a cloud storage gateway

Maintain all ECM capabilities

• Automatic version control

• Rules & workflow

• Full-text search

• Policy enforcement

NextGen Enterprise Storage

NASDAQ OMX AND CLOUD

STORAGE

Section 3

NASDAQ OMX and AWS

• History

• Options provided to NASDAQ OMX teams

• Evaluation of architectural options and

NASDAQ OMX technology selection

• Managing operations for cloud backup

• Security

History of Relationship and FinQloud

Sample FinQloud Workflow – How Does It Work?

Customer data sets are ingested at Nasdaq hosted inbox via

secure file transfer

Nasdaq preprocesses data (e.g., trade data) at Nasdaq

Facilities

Split trade data records into chunks (about 1M records per

chunk)

Each file is encrypted with AES-256 / FIPS complaint system

Custom encryption is applied (e.g. per client/per day, random

initialization)

Sample FinQloud Workflow – How Does It Work?

A custom metadata header file is attached to encrypted chunk.

Metadata is signed via SHA-256

File chunks are uploaded to Amazon S3/R3 – each FinQloud

customer gets a new AWS account, a new bucket

WORM or regular Amazon S3 buckets can be utilized

Search and retrieve functionality is performed by Amazon Elastic

MapReduce (AWS-managed Hadoop) for performance

Each customer gets an assigned Amazon Virtual Private Cloud (VPC)

Amazon EMR request key files from Nasdaq hosted host security files

Sample FinQloud Workflow – How Does It Work?

For cloud-based processing (e.g. reporting) trade data chunks,

and files are decrypted in the memory with Amazon EMR

Data is never in clear-text in transit or at rest

Once the jobs are completed, data sets are re-encrypted again

and either written to Amazon S3/R3 or shipped back to

Nasdaq

Results data-sets can be decrypted at Nasdaq facilities via

HSM hosted keys or customer can integrate their PGP keys

for asymmetric encryption and download the results

Selection of Enterprise Storage Workload

Type for First Cloud Project

• Cloud first strategy

• Tier 3 storage vs backup workloads

• Selection of backup as the first use case

– Greater control of the implementation / outcome

– Less risk as it was backup data vs production data

• Backup technology options

Selection of Architecture and Technology

• Why Riverbed Whitewater was chosen – Ease of deployment

– Strong vendor support model

– Good integration and compatibility

– On-premises cache repository for backup platform

– Inline dedupe, compression

– Data encryption at appliance before data transfer

– Listed company

NASDAQ OMX Cloud Backup Architecture

How NASDAQ OMX Runs Storage

Operations for the Cloud

• No major changes since Netbackup is integrated

with Riverbed Whitewater

• Riverbed appliance looks like a standard disk

pool to the media server

• NBU policies altered to make us of RB disk pool,

which in turn sends data into Amazon S3

How Does Security and Isolation Work?

AWS standard security +

– Data is always encrypted at transit and at rest

– Keys are stored at Nasdaq facilities

– Nasdaq InfoSec department performed security

review and provided sign off of security

measures.

Integration Points

• All AWS services+

– End-to-end isolation

– End-to-end encryption

– Separation of duties by hybrid security

– Patented worm: R3

– Dynamic key management

PROOF OF CONCEPT EXAMPLE

WITH TIER 3 STORAGE

Section 4

• NASDAQ OMX is planning to leverage scale, durability

and the cost advantages of cloud-based storage solutions

• In addition to backup and archive workloads, testing Tier-

3 storage on the cloud makes sense for NASDAQ OMX

due to the ratio of spend on Tier-3 storage (compared

with backup/archive workloads)

• There is a management initiative to leverage cloud

technologies at NASDAQ OMX

Why Tier-3 Cloud Storage Solution?

AWS Storage Gateway is a service that connects an on-

premises software appliance with cloud-based storage to

provide seamless and secure integration between on-premises

IT environment and AWS' storage. AWS Storage Gateway is:

– Native AWS offering

– Scalable

– Cost effective

– Controllable from AWS Management Console

– Promising roadmap

Why AWS Storage Gateway?

• The objective of this Proof-of-Concept (PoC) is to provide a

high-level analysis and checklist of all elements and attributes

necessary to successfully implement a Tier-3 Cloud Storage

Gateway.

• The PoC is the initial step prior to undertaking a detailed system

design and implementation and is intended to function as a

prototype system. It is meant to demonstrate key technologies,

as well as provide an environment for experimentation and

evaluation. The design and implementation of a POC, while very

detailed and organized, does not serve as a replacement for a

complete system analysis and design.

Objective of Proof-of-Concept

AS-IS Architecture

PoC Architecture

VMware:

• VMware ESXi Hypervisor (v 4.1 or v 5)

• 4 virtual processors assigned to the VM

• 7.5 GB of RAM assigned to the VM

• 75 GB of disk space for .ova installation and system data

External Connectivity:

• Ports 80 and 443 are used by the vSphere client to communicate to the ESXi host.

• Port 80 is used when you activate your gateway from the AWS Storage Gateway console.

• Port 3260 is the default port that your application server uses to connect to iSCSI targets.

PoC AWS Storage Gateway Requirements

VMware:

• VMware ESXi Hypervisor server x 2 (existing servers can be used)

Ethernet NICs:

• Existing NICs can be leveraged; dual NIC tests are recommended

Ethernet Switch:

• Existing network switches can be leveraged; isolation and

bandwidth allocation recommended

Connectivity:

• Existing AWS connectivity or Internet connections can be leveraged

PoC On Premises Hardware Requirements

Provision 2 VMware physical servers (hosts)

Download the AWS Storage Gateway software at

http://console.aws.amazon.com/storagegateway

Allocate on-premises storage for active data

Activate gateway and select an AWS region

Create and mount iSCSI volumes

Provision Ethernet Cards and Network Infrastructure

Test Primary Storage Access over iSCSI on new volumes

Configure volume management to copy data sets from

existing volumes to new volumes

Estimate the approximate data plan to write on a daily basis.

It is recommended to allocate at least 150 GBs.

Sizing On-Prem Storage Upload Buffer

Backup use cases: at least the size of upload buffer

File share use cases: 20% of current storage

Cache Storage Sizing On-Prem Storage

• Solaris iSCSI initiators

• Cache storage is durable store

• Allocate separate disks for cache storage and

upload buffer

• Quick format vs. full format of drives

• Virus scanning

Additional Configuration Considerations

• By January 31 this PoC should be completed.

• This POC is not an open-ended project. It is a limited

implementation for a fixed period of time. The project

duration will be a direct result of the project objectives.

• This duration includes project time necessary to plan,

design, and implement the POC system.

• Project management teams should actively work to

control scope.

PoC Schedule

Amazon Web Services

• High-level design

• AWS Storage Gateway installation and configuration

• AWS cloud components configuration

• Assistance in iSCSI and VMware configurations

• Provide test plans

• Assistance is test execution

• Delivery of final analysis

NASDAQ OMX

• On-premises hardware installation (NICs, hosts)

• VMware installation and configuration

• On-premises network configuration (Switches, VLANs etc.)

• Providing test targets, assistance in test plans

• Execute tests

• Assistance in final analysis

PoC Resources and Responsibilities

• Virtualization hardware (VMware physical host)

and network systems are not for production

• Hardware and network limitations that are not

critical for success will not be addressed

• Compromises will be made to accommodate the

smaller scale of the implementation

Limitations of PoC

• Access tier-3 volumes in cloud over iSCSI links

• Failover / failback / redundancy tests

• Reliability tests

• Performance tests

• Security controls

• Manageability tests

• Cost analysis

PoC Test Plans

• Meet test targets 80% and over

PoC Success Criteria

• Nasdaq interviews

• Issue review

• Implementation review

• Reliability review

• Scalability review

• Performance review

• Security review

• Manageability review

• Cost review

• Final project analysis

Results Analysis

Please give us your feedback on this

presentation

As a thank you, we will select prize

winners daily for completed surveys!

ENT301