cloud data migration strategies - aws may 2016 webinar series

51
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Business Development May 2016 Cloud Data Migration: Eight Strategies for Getting Data into AWS

Upload: amazon-web-services

Post on 07-Jan-2017

1.201 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

AWS Business Development

May 2016

Cloud Data Migration: Eight Strategies for Getting Data into AWS

Page 2: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Storage is the Gravity for Cloud Applications

Page 3: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon EFS

File

Amazon EBS Amazon EC2Instance Store

Block

Amazon S3 Amazon Glacier

Object

Data Transfer

AWS Direct Connect

ISV Connectors

Amazon Kinesis

Firehose

Storage Gateway

S3 Transfer Acceleration

AWS Storage is a Platform

AWS Snowball

AmazonCloudFront

Internet/VPN

Page 4: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Internet / VPN Ingest

Page 5: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

What is Internet/VPN…

Globally Available

Default method of ingesting content into Amazon S3

Simple standards based (HTTP) connection

Use your existing internet connection

Available within a VPC for VPN connectivity

Acceleration via Multipart Upload

Data Transfer Into AWS is free

VPN Connections using VPC Virtual Private Gateway• $0.05 per VPN Connection-hour• $0.048 per VPN Connection-hour for connections to the Tokyo region

Page 6: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

How does Internet / VPN ingest work?Accelerate Data Transfer using

Multipart UploadIngest Data Directly Into S3 Buckets with

existing internet connectivity

S3 BucketAWS Region

and

Via Management Console or API

customer gateway

endpoints

VPN connection

Internet Internet via VPN + VPC

Page 7: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon S3 Transfer Acceleration

Page 8: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

What is Amazon S3 Transfer Acceleration…

Network and Protocol Based Data Transfer Service

Acceleration of Data Ingress / Egress with S3 Buckets

Typically 50% to 400% faster

Feature of S3 Enabled at the Bucket Level

Available at All S3 Regions Worldwide

No Client / Server Software Required

No Code Changes to Your Application

No Firewall Exceptions

Simple Pricing Model

Page 9: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Ingest & Egress with S3 transfer acceleration

S3 BucketAWS EdgeLocation

Uploader

OptimizedThroughput!

Uses AWS 55 global edge locations

AWS determines best edge location

Data transfer optimized between edge and customer, and edge and S3

Data is not stored on the edge cache

Page 10: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon Route 53

Resolve b1.s3-accelerate.amazonaws.com

HTTPS PUT/POST upload_files.zip

HTTP/S PUT/POST “upload_files.zip”

Service traffic flowClient to S3 Bucket example

S3 Bucket

b1.s3-accelerate.amazonaws.com

EC2 Proxy

AWS Region

AWS Edge Location

Customer Client

1

2

3

4

Data is not cached on the AWS Edge location

Fully Managed File Transfer Acceleration using all AWS Edge

Locations

Page 11: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Using the Service is as easy as 1, 2, 3…

Update Application to Point to new S3 URL• Update“bucket.s3.amazonaws.com” to

“<bucket-name>.s3-accelerate.amazonaws.com”• Original bucket location and contents are the same, only

namespace changes

Or Use Permissions via API s3:PutAccelerateConfiguration

Enable the Service in the Management Console

Start Uploading Data to Amazon S3

1

2

3

Page 12: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

How fast is S3 transfer acceleration?

Rio De Janeiro Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los Angeles Seattle Tokyo Singapore

500 GB upload from these edge locations to a bucket in Singapore

Tim

e [h

rs]

Public InternetS3 Transfer Acceleration

Page 13: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

How much will it help me?

Page 14: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Speed Checker Demo

Page 15: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Pricing*

Dimension Price / GB

Data Transfer In from Internet** $0.04 (Edge location in US, EU, JP)$0.08 (Edge location in rest of the world)

Data Transfer Out to Internet $0.04

Data Transfer Out to Another AWS Region $0.04

Amazon S3 Charges Standard data transfer charges apply

*Plus standard Amazon S3 data transfer charges apply**Accelerated performance or there is no bandwidth charge

Page 16: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon CloudFront

Page 17: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Global Content Delivery Network

55 Edge Locations Worldwide

Supports Ingest via PUT/POST methods

Works with S3 Multi-part upload

Supports SSL SNI and TLS connections

Integrated with ACM and AWS WAF for additional security

Proxy ingest to S3, EC2 and even your own origins

Tiered and Custom Pricing Models

What is Amazon CloudFront…

Page 18: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Using CloudFront to Ingest Data into AWS

AWS Region

Customer Client

HTTP/S PUT/POST “upload_files.zip”

Amazon EC2

S3 Bucket

ELB

Custom Origin

CloudFront Edge Location

Ingest content into S3, EC2, ELB or your own custom origin with Amazon CloudFront

Use cache behaviors to direct to the correct origin based on PATH pattern matching

Restrict Access via Geo Restriction or AWS WAF Web ACL

Page 19: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon CloudFront Pricing

Data Transfer out of Amazon CloudFront to your origin server billed at the “Regional Data Transfer Out to Origin” rates listed in the Regional Data Transfer Out to Origin (per GB) table.

Data Transfer out of Amazon CloudFront to Internet will be charged at rates listed in “Regional Data Transfer Out to Internet (per GB)” table.

Amazon CloudFront offers additional pricing options via a CloudFront Reserved Capacity (CFRC) contract. Contact sales for additional details and pricing.

Page 20: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

AWS Direct Connect

Page 21: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Dedicated, 1 or 10 GE private pipes into AWS

Create private (VPC) or public virtual interfaces to AWS

Reduced data-out rates (data-in still free)

Consistent network performance

At least 1 location to each AWS region

Option for redundant connections

Uses BGP to exchange routing information over a VLAN

What is AWS Direct Connect…

Page 22: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Physical Connection

• Cross Connect at the location

• Single Mode Fiber - 1000Base-LX or 10GBASE-LR

• Potential onward Delivery via Direct Connect Partner

• Customer Router

Page 23: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

At the Direct Connect Location

CORP

AWS Direct

Connect Routers

Customer

Router

Colocation

DX Location

CustomerNetwork`

AWS BackboneNetwork

Cross Connect

Customer Router

Customers Network

Demarcation

Page 24: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Dedicated Port via Direct Connect Partner

AWS Direct

Connect Routers

Colocation

DX Location

Partner Network

AWS BackboneNetwork

Cross Connect

Customer Router

PartnerNetwork

AccessCircuit

Demarcation

PartnerEquipment

CORP

Page 25: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Direct Connect - LocationsAWS Region AWS Direct Connect LocationAsia Pacific (Singapore) Equinix SG2, GPX, MumbaiAsia Pacific (Seoul) KINX, SeoulAsia Pacific (Sydney) Equinix SY3, Global SwitchAsia Pacific (Tokyo) Equinix OS1, Equinix TY2China (Beijing) Sinnet JiuXianqiao IDC, CIDS Jiachuang IDCEU (Frankfurt) Equinix FR5, Interxion FrankfurtEU (Ireland) TelecityGroup, London Docklands’, Eircom Clonshaugh

Equinix LD4 - LD6, LondonSouth America (Sao Paulo) Terremark NAP do Brasil, TivitUS East (Virginia) CoreSite NY1 & NY2, Equinix DC1 - DC6 & DC10US West (Northern California)

CoreSite One Wilshire & 900 North Alameda, CA, Equinix SV1 & SV5

US West (Oregon) Equinix SE2 & SE3, Switch SUPERNAP, Las VegasAWS GovCloud (US) Equinix SV1 & SV5

Page 26: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon Kinesis Firehose

Page 27: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon Kinesis PlatformAmazon Kinesis streaming data on the AWS cloud• Amazon Kinesis Streams• Amazon Kinesis Firehose • Amazon Kinesis Analytics

Page 28: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon Kinesis FirehoseLoad massive volumes of streaming data into Amazon S3 and Amazon Redshift

Zero administration: Capture and deliver streaming data into S3, Redshift, and other destinations without writing an application or managing infrastructure.

Direct-to-data store integration: Batch, compress, and encrypt streaming data for delivery into data destinations in as little as 60 secs using simple configurations.

Seamless elasticity: Seamlessly scales to match data throughput w/o intervention

Capture and submit streaming data to Firehose

Firehose loads streaming data continuously into S3 and Redshift

Analyze streaming data using your favorite BI tools

Page 29: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Vertical/Use Case Accelerated Ingest-Load to final destination for Analytics

Ad Tech/ Marketing Analytics

Advertising data aggregation

Consumer Online/Gaming

Online customer engagement data aggregation

Financial Services Market/ Financial Transaction order data collection

IoT / Sensor Data Fitness device , vehicle Sensor, telemetry data ingestion

Amazon Kinesis Firehose Use Cases

Page 30: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

AWS Storage Gateway

Page 31: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

What is AWS Storage Gateway?

Works with your existing applications

Secure and durable storage in AWS

Low-latency for frequently used data

Scalable and cost-effective on-premises storage - $125 per gateway per month + S3/Glacier storage fees

Service connecting an on-premises software appliance with cloud-based storage

Page 32: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Common uses for AWS Storage Gateway

Backup and archive

Disaster recovery

Data migration

Page 33: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

How does AWS Storage Gateway work?

Amazon EBS snapshots

Amazon S3Amazon Glacier

AWSStorage Gateway

appliance

Applicationserver

AWSStorage Gateway

backend

Customer premises

S3 Transfer

Acceleration

AWSDirect

Connect

Internet

Page 34: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

AWS Storage Gateway configurations

iSCSI block storage

Gateway-stored volumes

iSCSI virtual tape storage

Low-latency for all your data with point-in-time backups to AWS

Replacement for on-premises physical tape infrastructure for backup and archive

Gateway-cached volumes

Gateway-virtual tape library (VTL)

Low-latency for frequently used data with all data stored in AWS

Page 35: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Gateway-virtual tape library (VTL)• Replace or augment your aging tape infrastructure with durable object

storage• Virtual tapes stored in AWS. Frequently accessed data cached on-premises• Up to 1,500 tapes, up to 2.5 TB each, for up to 150 TB per gateway-VTL• Unlimited number of tapes in virtual tape shelf (VTS)

Customer data center

VTS storage backed by Amazon Glacier

AWS Storage Gateway VM

BackupServer IN

ITIA

TOR

AWSStorage Gateway

service

MED

IA

CHAN

GER

UploadBuffer

CacheStorage

Gateway-VTLstorage backedby Amazon S3

VTS

TAPE

DR

IVE

Page 36: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

AWS Snowball

Page 37: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

What is AWS Snowball? Petabyte-scale data transport

E-ink shipping label

Ruggedized case“8.5G impact”

All data encrypted end-to-end

Rain- and dust-resistant

Tamper-resistant case and

electronics

80 TB10 GE network

Page 38: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

How it works

Page 39: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

• Less than 1 day to transfer 200TB via 3x10G connections with 3 Snowballs, less than 1 week including shipping

• Number of days to transfer 200TB via the Internet at typical utilizations

How fast is Snowball?

Internet Connection SpeedUtilization 1Gbps 500Mbps 300Mbps 150Mbps

25% 71 141 236 471

50% 36 71 118 236

75% 24 47 225 157

Page 40: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Use cases: AWS Snowball

Cloud Migration

Disaster Recovery

Data CenterDecommission

ContentDistribution

Page 41: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Pricing

Dimension Price

Usage Charge per Job $200.00 (50 TB)$250.00 (80 TB)

Extra Day Charge (First 10 days* are free) $15.00

Data Transfer In $0.00/GB

Data Transfer Out $0.03/GB

Shipping** Varies

Amazon S3 Charges Standard storage and request fees apply

* Starts one day after the appliance is delivered to you. The first day the appliance is received at your site and the last day the appliance is shipped out are also free and not included in the 10-day free usage time. ** Shipping charges are based on your shipment destination and the shipping option (e.g., overnight, 2-day) you choose.

Page 42: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

AWS Technology Partnerships

Page 43: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Amazon Storage Partner Ecosystem

Gateway/NAS DataManagement Sync and ShareBackup/DR Content and

Acceleration

Archive

File System

Page 44: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Example of Data Transfer with Partner Solution: Attunity Cloudbeam for AWS

S3 EMR

Hourly Model, BYOL, and Free Trial Availablehttp://www.attunity.com/cloud-solutions-for-amazon-web-services

Page 45: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Backup to AWS Approaches

Amazon S3

Amazon GlacierAWS

DirectConnect

InternetAmazon S3-IA

Applicationservers

Cloud Gateway

Local disk

MediaServer

Cloud Gateway

HTTPS/API

Applicationservers

Backup SW cloud connector

Local diskMedia

Server with cloud

connector

HTTPS/API

Page 46: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

CommVault Ties Together On Premise and Cloud Data StrategiesCommvault Orchestrates the Enterprise

• Back up in the Cloud: Keep backups of cloud workloads internal to the cloud

• Back up to the Cloud: Allow on premise workloads the ability to leverage AWS

• Disaster Recovery to the Cloud: Automate disaster recovery to the cloud on a scheduled basis

• Workload Portability: Rest assured that virtual servers can be moved from on-premise to the cloud and back, keep your data available wherever you need it

• Archiving to the Cloud: Moving legacy data to tier 2 storage in the cloud for long term archive

AWS and Commvault together combine to minimize networking, storage and infrastructure

costs, while providing the business a sound data protection and disaster recovery strategy.

Page 47: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Backup to AWS Approaches

Amazon S3

Amazon GlacierAWS

DirectConnect

InternetAmazon S3-IA

Applicationservers

Cloud Gateway

Local disk

MediaServer

Cloud Gateway

HTTPS/API

Applicationservers

Backup SW cloud connector

Local diskMedia

Server with cloud

connector

HTTPS/API

Page 48: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

NetApp AltaVault Backup from On-premises to S3/Glacier

Common backup applications integrated with AltaVaultSolve backup & archive headaches with cloud-integrated storage

90% reduction in time, cost, and data volumes Shrink recovery times from days to minutes 85% of backup & software providers supported

Glacier

On Premises

AWS

Cloud-integrated storage appliance

NetApp AltaVault

FAS

E-SeriesNon-NetApp

Storage

NetApp SnapProtect Arcserve CommVault Simpana EMC NetWorker HP Data Protector IBM Tivoli

Storage Mgr

Symantec Backup Exec

Symantec NetBackup

Veeam Microsoft SQL

Server Oracle RMAN

S3

AltaVault also available on marketplace to protect cloud-native workloads

Seamlessly integrates into existing storage and backup software environment

Caches recent backups locally, vaults older copies to the cloud

Store data in the public or private cloud of choice

Deduplicates, compresses, and encrypts

Page 49: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

unified file services that extend from endpoints, to remote

offices, to the cloud.

snapshots, file versioning and file sync runs across all access points via the cloud

data is secured and optimized at

the source

all stored in your AWS VPC, data is stored

on AWS S3-IA

Integrated with trusted enterprise security and

management tools

ROBO NAS Gateways

Endpoint Apps

Cloud Server Agents

Data ProtectionEngine

File SyncEngine

with centralized automation, management and multi-tenancy

Identitymanagement

datagovernance

cloudorchestration

S3 Infrequent Access

CTERA GlobalDeduplication

Ctera: Enterprise File Services Platform

Page 50: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Summary – When to Use each Service

IF YOU NEED: CONSIDER:An optimized or replacement Internet connection to:

connect directly into an AWS regional datacenter Direct Connect

migrate TB or PB of data to the cloud Snowball

Accelerate data transfer S3 Transfer Acceleration, CloudFront, AWS Partner

A friendly interface into S3 to:cache data locally in a hybrid model (for performance reasons) Storage Gateway, AWS Partner

redirect backups or archives with minimal disruption Storage Gateway, AWS Partner

aggregate data streams from multiple devices Kinesis Firehose

Page 51: Cloud Data Migration Strategies - AWS May 2016 Webinar Series

Thank you!