applying systems thinking to aws enterprise application migration

© 2014 Cloud Technology Partners, Inc. / Confidential

1

April 7, 2014

Adventures in Enterprise Application Migration to AWS

Dreams to Realities to Best Practices


2

• Me?

– Kacy Clarke

– Vice President, Principal Architect at Cloud Technology Partners

– Cloud architecture, AWS, big data, application migration, cloud data

management, DevOps/ProdOps

[email protected]

– www.linkedin.com/in/kacyclarke/

– www.google.com/+KacyClarke

– @kacyclarke

updraft-downdraft.blogspot.com

• Us?

– Cloud Technology Partners, Inc. www.cloudtp.com

– A cloud consulting and products company focused on application

migrations and new application development

• This Session?

– A discussion of enterprise application migration: the dreams, the

realities, and best practices

Setting the Stage

mailto:[email protected]

http://www.linkedin.com/in/kacyclarke/

http://www.linkedin.com/in/kacyclarke/

http://www.google.com/+KacyClarke

http://twitter.com/kacyclarke

http://twitter.com/kacyclarke

http://updraft-downdraft.blogspot.com/



http://www.cloudtp.com/


3

The Dream: Enterprise Application Migration


4

A 2012 survey by IDC reports an average 72% savings and a seven month

payback for moving applications to Amazon Web Services

The Dream: The Pragmatists Are Starting to Move

Geoffrey Moore: Crossing the Chasm


5

AWS is a very different environment from the

typical enterprise data center • Applications are configured for pre-allocated capacity

• They assume infrastructure availability

• Infrastructure teams use ITIL processes for changes

• Operations teams monitor infrastructure, not apps

• Security teams want to encrypt and firewall everything

The Reality: Enterprise Applications Don’t Migrate That Easily

network

latency


6

The Reality: The Cloud is Very Different

Traditional Architectures

• Scale Up • Monolithic • Stateful • Infra Dependent • Fixed Capacity • LAN Located

dependencies • Latency intolerant • Tightly coupled • Consolidated /

clustered DB • Rich / chatty client • Commercial licenses • Infra Supported

Availability • Semi-automated

build/deploy • Manual fault

recovery • Active/Passive/DR • Perimeter Security • Allocated costs

The “Old World”

Cloud Aligned Architectures

• Scale Out • Distributed • Stateless • Infra Agnostic • Elastic capacity • WAN, Location

transparency • Latency tolerant • Loosely coupled • Sharded / replicated

/ distributed DB • Mobile/thin client • PaaS / Open Source • App Supported

Availability • Continuous

Integration/Delivery • Self healing, fault

tolerant • Active/Active • Defense in depth • Pay as you go

The “New World”

The Targets

Ref

acto

r

Au

tom

ate


7

Cloud is harder and has a broader impact than enterprises expected

The Reality: Enterprises are Concerned About the Effort and Risk

Gartner Inc., 2013


8

The Reality Check for Enterprise Application Migration

• The Lift and Shift Dream

• The Forklift Dream


9

The Lift and Shift Dream


10

The Dream: Creating a Data Center Annex

Enterprise DC Annex

Enterprise Data Center

Used for:

• Dev/Test

• Non-critical workloads

• BC/DR

• Batch analytics

• Storage

Cloud Provider

Built on:

• IaaS

• BYOL

• Extension of current

Ops/Security


11

Application

Data

App Infrastructure

• Webservers

• App Servers

• Database Servers

OS, File System, Network Topology

Security Framework

Operational Tools

Virtualization

Compute, Storage, Network

Cloud Provider Data Center

The Dream: Lift and Shift Application Migration

Internal Deployment Cloud Deployment

Application

Data

App Infrastructure

• Webservers

• App Servers



Security Framework

Operational Tools

Virtualization


Internal Data Center


12

You don’t have to change your applications to migrate to the cloud

Baseline Migration Will Enable Rapid Lift and Shift Migration

• Simple discovery tools

• Automated application migration

• Same operations environment and security

framework

• Example Cloud Migration Tool Vendors:

– Racemi

– RiverMeadow

– CloudVelocity

– Ravello Systems

– Bolt VTA by Yuruware

– Rackware

– Appcara

– CliQr

– AppZero


13

The Reality: Enterprise Applications Don’t Always Fit the Cloud


14

• Missed SLA’s

• Long Mean Time to Recovery

• Gaps in monitoring and operational tools

• Sporadic performance problems

• Data synchronization and placement issues

• SDLC sprawl and inconsistencies

• Lack of transparency on what’s happening with

the application or the system

• Dev and operational skills and knowledge

• Integration latency

• Manual or frozen configurations

• Compliance or security findings

• Cost savings objectives not met

The Reality: Lift and Shift Can Result in Cloud Migration Debt


15

systems thinking: a holistic approach to analysis

that focuses on the way that a system's

constituent parts interrelate and how systems

work and change over time and within the context

of larger systems. The systems thinking approach

contrasts with traditional analysis, which studies

systems by breaking them down into their

separate elements

• Cloud Migration Impacts:

– Application Architecture

– Infrastructure Architecture

– Data Lifecycle Management

– Security and Compliance

– SDLC

– Operations and Monitoring

– Support

– Cost Management and Planning

Application migration requires the holistic analysis of systems thinking

The Reality: It’s Not an Application, It’s a System


16

1. Rehost/Lift and Shift

2. Replatform and Tune

3. Fully Refactoring

Best Practices: Initial Application Migration Strategies


17

Application

Data

App Infrastructure

• Webservers

• App Servers



Security Framework

Operational Tools

Virtualization



Best Practices: Enhanced Lift and Shift


Application

Data

App Infrastructure

• Webservers

• App Servers



Security Framework

Operational Tools

Virtualization




18

1. Bootstrap or fully bake AMIs

2. Automate provisioning and processes

3. Replace existing storage options with AWS Storage (EBS, S3, RDS, etc..)

4. Test mounted file systems carefully (GlusterFS, Ceph) and replace if necessary

5. Leverage AWS security features (IAM, security groups, VPC, key pairs, MFA, VPN

connectivity, etc..)

6. Evaluate networking options (Direct Connect, WAN optimization, )

7. Analyze data movement performance and cost (AWS Import/Export, Storage

Gateway, data compression, etc..)

8. Upgrade your system and application monitoring to gather more metrics

Even with Lift and Shift, some changes are needed to deploy on AWS

Best Practices: Enhanced Lift and Shift


19

Application

Data

App Infrastructure

• Webservers

• App Servers



Security Framework

Operational Tools

Virtualization



Best Practices: Replatform and Tune


Application

Data

App Infrastructure

• Webservers

• App Servers



Security Framework

Operational Tools

Virtualization




20

• Resiliency and Self-Healing

– Alerting and monitoring for failures, automated reconnect/restart/recover

– Application managed availability

– Increased redundancy, spread across more availability zones

• Data Management

– Cloud integrated data lifecycle and master data management, optimized data movement

• Replatform to leverage AWS Native App Infrastructure (Elasticache, RDS, DynamoDB,…)

• Transparency and Metrics

– New application monitoring tools and activity logging

– User Experience and performance testing/monitoring

• Elasticity and Right Sizing

– Cost monitoring and instance type optimization

– Fully automated provisioning, pre-warmed instances for planned scaling

• Manageability

– Auto-response for common conditions

– Integrated app and systems monitoring with joint DevOps/SysOps team

Targeted optimization of the application, infrastructure, tools and operations

Best Practices: Second Generation AWS Modifications


21

The Forklift Dream


22

The Dream: Large Scale App Portfolio Migration to the Cloud

• “We want to move 40-50% of our 9000

applications to the cloud in 2 years”

• “Our objective is to shutdown our data

center in 18 months”

• “We’re constrained by power and cooling

and need to expand our capacity outside of

our four walls.”

• “Our applications all run on VMware so

they should move easily.”

• “We’ll just move the easy applications first,

and then we’ll know enough to be able to

move the mission critical applications”

• “This is an infrastructure (or application) led

project. The other teams don’t need to be

involved until we figure the strategy out.”


23

• F1000 application portfolios can contain

1000’s

• Most application portfolio or configuration

management databases (CMDB) are <

50% complete and < 50% accurate

• Cloud strategies are being decided in

boardrooms and executive suites

• Enterprises don’t have the knowledge nor

the resources to refactor every application

• Missing automated builds or test suites to

rapidly validate releases

• Spaghetti integrations between apps

• Unknown dependencies

• Low hanging fruit sometimes have worms

The Reality: Migrating Apps is Shaped by What You Don’t See


24

Use a fact based, decision support methodology and tool-set to guide placement of applications

within a range of infrastructure endpoints from physical servers to private cloud to public cloud

Best Practices: Application Portfolio Analysis at Scale

Stay where it is

Target Endpoints

Target Cloud Provider

Refactor / Replatform /

Replace

Options for Application

Rationalization

Dep

th A

naly

sis

Replace

Reuse

Refactor

Replatform

Rehost

Retain

Retire

App Strategy

Sensitive Data

COTS vs.

Custom

Commodity vs.

Core

Workload

Logical

Architecture

Eco-system

Criticality

Programming

Language Physical

Architecture

Availability

Application

Decision

Framework -

ADF ™

Breadth Analysis


25

• Determining the right migration

strategy for your app depends on

its level of cloud alignment, cloud

readiness, potential benefits

achieved from migrating, and risks

• Not everything will migrate

• Migration strategy selection also

depends on the target cloud

endpoint

• Public and other external cloud

endpoints may require 2x-10x

effort

• Rehost and replatform strategies

minimize change for applications

that are either cloud ready, or

have higher risks or lower cloud

benefits

• Refactor and reuse strategies

accept the cost of change in

return for business benefits

• There are patterns, processes,

teams and tools to support each

strategy and each cloud endpoint

Best Practices: Application Migration Is Not One Size Fits All

• Replace application with SaaS service • Build cloud native application with similar / improved

characteristics and features

• Develop and productize common business and technical services

• Consolidate similar applications and services

• Targeted changes to the application to address issues leverage the cloud

• Revise applications to leverage common services

• Move to more cloud-aligned technology and platform services

• Integration with cloud operations and monitoring

• Lift and shift, minimizing changes, virtualization replatforming

• Leave application as-is; do not move to cloud

• Application end of life

Replace

Reuse

Refactor

Replatform

Rehost

Retain

Retire


26

Best Practices: Applying Systems Thinking to Migration@Scale

Fast Track Application Assessment

Code Depth Analysis and Initial POCs

Infrastructure

Applications

Operations

SDLC/Testing

Governance

Integration

Security

Planning

Data

Execution


27

PaaSLane Accelerates Portfolio Analysis

Application Decision Framework

Thousands of Applications

Narrow Candidate Applications

Define Target Applications

Days

Minutes


28

Depth: PaaSLane™ for Cloud Migration & Development

PaaSLane™ analyzes application source code,

discovering issues and recommending

solutions to ensure cloud readiness and

accelerate application migrations.

1

2

Rapidly Profile Java and

.NET Application Code

Assess Application Readiness

for Multiple Cloud Platforms

3 Deliver Code Remediations

Required for Cloud Platforms

4 Estimate Development Effort

Required to Implement

Remediations

3 Key Use Cases

1. Cloud Health Check

Pre-migration review and

estimation.

2. Cloud Migration

Track progress of cloud

readiness effort.

3. Continuous Optimization

Keep applications optimized for

the cloud.

Assess Cloud Readiness In Minutes, Not Weeks


29

Amazon Rules

• 30+ rules to help migrate applications to AWS

• Recommend AWS services such as S3, RDS, SES, SMS, etc.

Default Rule Categories

– General: Best practices for creating cloud-native applications.

– Availability: Potential stability issues in cloud platforms.

– Performance: Improving application performance and reducing overprovisioning.

– Quality: Improving overall code quality and adherence to standards.

– Scalability: Enhancing elasticity and reducing bottlenecks in the cloud.

– Security: Implementing secure coding practices and strong cryptography.

– Standardization: Enforcing platform standardization and identifying third-party dependencies.

PaaSLane™ AWS Edition: Data Driven Refactoring

• May fail to run or be a high security risk Blockers

• Likely to impact performance or functionality Important

• May be impactful: further research required Warnings

• Build a more cloud-native application Optimizations

applying systems thinking to aws enterprise application migration

Technology

cloud consulting

cloud baseline migration

cloud migration debt

cloud data management

old world cloud

application migrations

shift dream

enterprise applications