migrating to snowflake blast out of the starting gate€¦ · © 2019 snowflake inc. all rights...

© 2019 Snowflake Inc. All Rights Reserved

MIGRATING TO SNOWFLAKE

BLAST OUT OF THE STARTING GATE

JOSHUA SKARPHOL

CONSULTING MANAGER

SNOWFLAKE


AGENDA

2

My Migration Story

Migration Basics

Migration Strategy

How Snowflake Helps Our Customers Migrate


MY MIGRATION STORY


MY MIGRATION STORY BEGAN AS A TERADATA ARCHITECT AT A FORTUNE 100 COMPANY

4


CLOUD FIRST INITIATIVE

5


CAPACITY CHALLENGES

6

Added 2 more

unplanned for

production nodes -

Summer 2015

Teradata on

AWS POC -

Summer 2016

Teradata floor

sweep -

Summer 2014

Teradata offload

to Hadoop POC -

Winter 2015


• The floor sweep in Summer 2014 was

projected to meet the Teradata capacity

needs for the next 3 years

• In Summer 2015, the new CIO signed

for an unplanned Teradata capacity

purchase on his second day on the job

• The CIO delivered the message that he

wasn’t going to sign for any more

unplanned Teradata capacity purchases

7

COST


SNOWFLAKE EVALUATION PROCESS

8

POC

End-to-end data processing

from source to target for 3

Dev/QA/Prod environments of

the Teradata tables used for an

existing analytics application

Scope

Conducted a bake off

between 4 GSIs to see

their migration

capabilities in action

ApproachValidated Teradata

benchmark of 99

concurrent queries could

run on Snowflake in

less than 2 minutes

Pilot


• Snowflake was selected as the going

forward Enterprise Data Warehouse

• The pilot proved doing a full Teradata to

Snowflake migration was achievable

• POS Migration - Point of Sale workload

that consumed 30% of the production

Teradata server was successfully

offloaded to Snowflake

9

EVALUATION OUTCOMES

30%Teradata Production


BENEFITS OF MIGRATION

10

Support teams of all sizes and provide a

chargeback model for system usage

Data consolidated across the company

onto a single platform

Business units scale to meet their own needs

without impacting or being impacted by

other business units

Provide capacity on demand instead of

being capacity constrained


• Establish security model and resource

monitors at the beginning of the

migration

• Bring in a Snowflake Technical Account

Manager to help get teams up and

running more quickly and share

Snowflake knowledge across the

company

• Communicate best practices and wins

more proactively

11

LESSONS LEARNED


MIGRATION BASICS

© 2019 Snowflake Inc. All Rights Reserved 13

How much does it

cost?

TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE


How much does it

cost?

How long will it take?



Have other customers

done what we’re

trying to do?

How much does it

cost?





done what we’re

trying to do?

How much does it

cost?

Costs depend on the size and

scope of what is migrated from the

existing solution to Snowflake




Duration depends on the size and




done what we’re

trying to do?

How much does it

cost?











done what we’re

trying to do?

How much does it

cost?





Customers have successfully migrated

from other data warehouse platforms to

Snowflake, but each customer is different




19

The answer is…it depends!


scope of what is migrated from

the existing solution to Snowflake

Have other customers done

what we’re trying to do?

How much does it cost?





Customers have successfully migrated

from other data warehouse platforms to

Snowflake, but each customer is different


MIGRATION STRATEGY


THE NEED FOR MIGRATION STRATEGY

21

Migration PlanningA successful migration begins with planning that is led by Snowflake PS or a SI partner to accurately scope the migration.

Code ConversionDDL, procedures, and scripts must be translated to be valid in Snowflake. Code Conversion effort is typically the largest unknown in scoping migrations.

Data MigrationData migration is the movement of an initial dataset from the legacy system to Snowflake. Effort in this phase varies based upon the number of tables and the method for moving the data.

Data IngestionData ingestion is the movement of data changes from the source system to Snowflake. Effort in this phase varies based upon the number of data sources and the method for moving the data.

Data ValidationSide by side comparison of the legacy data results and Snowflake results. Verifies data was moved successfully and that any translated logic is giving the intended results.

Reporting & Analytics Tools

Integrations for reporting and analytics tools must be modified and reconnected. This can be very involved for some systems where objects must be re-architected.

Security Implementation

Evaluate legacy security requirements, architect and implement the new security plan including SSO, roles, and data level security.


3 QUESTIONS THAT SHOULD BE ASKED ABOUT MIGRATION TO SNOWFLAKE

22

What support is available to

help with the migration to

Snowflake?

What really needs to

be migrated to

Snowflake?

What are the common

characteristics of successful

migrations to Snowflake?


• List databases and database objects to

migrate

• List processes and tools that populate

and pull data from the existing data

warehouse

• List security roles, users and

permissions

• Documentation of the existing data

warehouse solution into an as-is

architecture diagram

23

WHAT NEEDS TO BE MIGRATED –

DOCUMENT THE EXISTING SOLUTION


• Categorize:

• processes to migrate as-is

• processes that need re-engineering

• processes that need fixing

• processes that are no longer needed

• Draft migration deliverables

• Create to-be architecture diagram

24


ESTABLISH A MIGRATION APPROACH



MODERNIZATION CONSIDERATIONS

• Migrating to Snowflake provides an

opportunity to reevaluate the design of

the data warehouse solution

• Consider the tradeoffs between

refactoring parts of the previous data

warehouse solution and the

downstream impacts of that refactoring

• Scalability limitations that existed in the

previous data warehouse solution are

mitigated in Snowflake which requires a

new approach to capacity utilization


• List data sets to migrate first

• Create method for identifying process

dependencies for data sets

• Document process dependencies for

data sets

26


PRIORITIZE DATA SETS FOR MIGRATION


• Completing a migration requires time,

money and energy, so you need a

compelling reason to migrate

• Having a compelling reason to migrate

keeps you moving forward if challenges

arise with the migration

• Completing the migration brings a

sense of accomplishment that

something important has been achieved

27

CHARACTERISTICS OF SUCCESSFUL MIGRATIONS –

MOTIVATION


• Determine the commitment of decision

makers to providing the necessary

funding

• Document budget allocated for the

migration project

• Work with the Snowflake sales team to

estimate the costs to run Snowflake and

the Snowflake Professional Services

team to estimate the cost of the

migration

28


FINANCIAL COMMITMENT


• Need automated repeatable processes

to minimize the amount of time required

to migrate to Snowflake

• Snowflake has solution partners with

migration tooling to help our customers

with their migrations

29


MIGRATION TOOLING


SUPPORT FOR MIGRATING TO SNOWFLAKE

30

Our Snowflake Solutions Partners

have experience migrating

customers from other Data

Warehouse platforms to Snowflake

Professional Services has a scalable,

repeatable methodology and tools to

help our customers with their migration

to Snowflake

Solutions PartnersDocumentation

Migration Reference Manuals

available for:

Teradata, Oracle, and Netezza https://resources.snowflake.com/

Snowflake Migration Team

https://resources.snowflake.com/


HOW SNOWFLAKE HELPS OUR

CUSTOMERS MIGRATE


IS THERE A MIGRATION EASY BUTTON?

32


THERE IS A MIGRATION EASIER BUTTON


SNOWFLAKE CONVERSION SERVICES

34

We are looking for customer sponsors to develop additional sources!

Tables

Views

DML Statements

User Defined Functions

Stored Procedures

BTEQ

Tables

Views

DML Statements

User Defined Functions

Stored Procedures


MIGRATION READINESS ENGAGEMENT

Delivery of a migration plan to a customer

35

• 1 or 2 week, depending on

code conversion

approach, onsite paid

Professional Services

engagement to provide a

detailed migration plan for

customers across all

phases of the migration

• Cost - $20K (1 week) or

Custom SOW (2 weeks)

What it is:

• Completed migration

questionnaire

• Customer DDL and DML

scripts

• Customer subject matter

experts for the 1 or 2 week

on-site engagement

What is Required:

• Customer receives a full migration plan including cost

estimation, migration timeline, resource plan, and

detailed list of migration steps for code conversion,

data migration, data ingestion, data validation, and

reporting & analytics tools

• Customer determines if the migration plan is based

on a SI, Snowflake Professional Services, or a

combination of both executing the migration

Output & Value


SUMMARY

36

• The cost and timeline for migrating to Snowflake

varies based on each customer’s requirements

• Migrating to Snowflake is most successful when

using a planned-out migration strategy that

leverages as much automation as possible

• Snowflake has migration services to help our

customers migrate successfully


Thank You

migrating to snowflake blast out of the starting gate€¦ · © 2019 snowflake inc. all rights...

Documents