using aws, eucalyptus and chef for the optimal hybrid cloud

Using AWS, Eucalyptus

and Chef for the

Optimal Hybrid Cloud

Daniel Bozeman, Software Engineer

My Background

• Software Development

• Traditional Systems Administration and datacenter design

• AWS

• DevOps / Infrastructure Engineering

• My opinions are my own and do not necessarily represent

those of my employer, Mosaik Solutions

About Mosaik

• Founded in 1988, creating roaming guides

• Transitioned into making “coverage map” art assets for carrier

retail locations and television

• Launched our first wireless industry intelligence web SaaS

application, WiSARD, in 1999

• We’ve built highly skilled GIS, Research, Creative, and

Development teams

• Launched MapELEMENTS in 2011, putting powerful GIS tools

into the hands of decision makers through the web browser

• Clients include Fortune 100 companies, Federal agencies, and

regional and international mobile operators

On with the show!

Problem: “The cloud” gets

expensive (fast)

“The cloud” gets expensive (fast)

Requirements

We need to run an application in the cloud. It costs $5,000

per month. We also need a replica of the application stack to

stage new releases for in-house and customer testing. We

also need a replica of the application stack for development.

Problem

We just tripled the cost of our application!

Solution

Run your non-production stacks in-house on commodity

hardware.

Problem: Developers are Rock

Stars

Developers are Rock Stars

Requirements

Developer autonomy can often lead to new products,

increases in efficiency, and overall developer happiness.

Problem

Despite having your best interests in mind, one of your

developers accidentally left five very expensive test servers

running for two weeks.

Solution

Give your developers a sandbox that minimizes your

organizations operational expenditures.

Problem: Time

Time

Developer:

“Hi sysadmin! I hope all is well with you today! Um, yeah, I

need a server with 4 cores and 16GB of memory, networking,

DNS, accounts for my entire team with public keys uploaded,

and the list of software packages listed below installed and

patched. It also needs to ship logging information to our log

collector cluster. I need five of them. I need them for a

customer demo in 30 minutes, so if you could get those

together in the next 10, that’d be great. Mkaay?”

Time

Systems Administrator:

“Sure, no problem! I just need to go show the sales team

how to restart their Outlook client and I’ll get right on that.

Oh, and I need to write a script to sanitize some of the data

in our CRM for marketing’s latest e-mail campaign. Uh oh, I

smell burning electronics. Also, even if I did know what you

were talking about, this is gonna take me 12 hours.”

Time

Solution:

Provide your developers with a self-serve cloud

infrastructure. Sometimes developers need to deploy in-

house tooling to keep the software closer to a big dataset or

for security purposes. Empowering your developers to use

the same provisioning and infrastructure engineering tools

they utilize in the public cloud on-site will give your team a

serious efficiency edge.

But is a private cloud for me?

Money Money Money Money

• Time to put those Excel skills to use

• Get an idea of your overall cloud spending

• Don’t forget to investigate EC2 reserved instances, etc. as part

of your overall cloud savings plan

• Can any of my workloads by brought on-site?

• What are my capital expenditures going to be if I implement a

private cloud?

• What is the cost in man-hours of implementing my private cloud?

• Why haven’t you made a pretty graph yet of projected potential

savings over the next 36 months?

• Will you save enough money year over year for an additional

hire?

• Yes? What are you waiting for!?

Back to Time

• If you’re heavily entrenched in a large-scale public cloud

deployment, hopefully your team has implemented (or is in the

progress of implementing) some infrastructure management and

server provisioning tooling using Chef or Puppet.

• Pick a private cloud provider that best “synergizes” with your

public cloud in order to minimize or eliminate re-tooling of your

DevOps codebase and processes.

• Not only will this provide you with additional man-hour savings

with your initial cloud deployment, but will allow you to migrate

stateless services between the public and private cloud with little

to no re-tooling. Some call this a “hybrid cloud”

Our Use Cases

Development and Staging

• We host a full development and staging stack of our

MapELEMENTS cloud-based GIS software solution on-site.

• We can play with new technologies and experiment on new Chef

cookbooks without having to spin up public cloud instances.

• Anything developed, improved or updated on-site can easily be

pushed into production in the public cloud thanks to Chef

managing our servers and watching our software repositories for

new commits to “production” branches.

• All internal tooling is now developed on and deployed to our

internal infrastructure with Chef and Eucalyptus.

Another Problem: Solving

Complex Geospatial Problems

Solution

Off-the-shelf desktop GIS software

That was easy!

Yet Another Problem: Solving Extremely

Complex Geospatial Problems Very

Quickly

Distributed Geoprocessor

• Break the problem up into small chunks

• Fire up “worker” instances with Chef in Eucalyptus

• Workers ask a “master instance” for the next task in a work

queue

• Workers complete their jobs and ask for a new task

• This process continues until the work queue is empty

• The job is done!

• This scales linearly. Should we need the job to complete faster to

meet a client deadline, we can simply fire up more worker

instances in Eucalyptus.

• Should we exceed our on-site resources, we can continue to fire

up more worker instances in EC2.

Any Chef insights?

Chef

Have an audit trail. You need to know when things work as

expected. More importantly, you need to know when things go

horribly wrong!

Also, prevent people from making stupid mistakes where possible.

How you do that?

Chef

One of our developers created knife-santoku, a replacement for

knife which supports callbacks and notifiers. Use it to send

messages to IRC, HipChat, etc. concerning who did what and

when. Also, have your chef clients phone home to your chat clients

as well concerning their current health, state of mind, and personal

feelings

https://github.com/knuckolls/knife-santoku/

I’ll share a link to these slides on my Twitter:

@dboze

© 2013 Eucalyptus Systems, Inc.

Supporting Private and Hybrid Cloud Infrastructure

David FederleinCustomer Success Team


Hello

40


Cloud vs. Traditional Enterprise

• Bottom Up vs. Top Down

• “It's just an extension of virtualized infrastructure.”

• Silo'ed Teams vs. Collaborative DEVOPS

41


Changes In Measurements of Success

• Useful Deployments vs. Simple Milestone Goals

• Engagement of Cross-functional Teams

• Not All Metrics are the Same Weight When Measuring Success

42


Support vs. Customer Success

• Metrics vs. Goal-oriented Success

• Cross-team Documentation and Communication

• “Knowing is half the battle.” (Go Joe!)

• Documentation of Infrastructure

• Documentation of Use Case

43


“So, what's your point, man?”

• Seamless support from on-boarding to end of life of platform built.

• Documentation is the key to reducing remedial engagement times regardless of communication method.

• Traditional sales/support model is inferior to cross-functional and success-oriented collaborative work.

44


Questions?

45


Learn More

46

Read the Mosaik Solutions Case Studyhttp://www.eucalyptus.com/customers/case-studies

Download Eucalyptus FastStarthttp://www.eucalyptus.com/faststart

Do More

http://www.eucalyptus.com/customers/case-studies



http://www.eucalyptus.com/faststart



© 2012 Eucalyptus Systems, Inc.© 2012 Eucalyptus Systems, Inc.

Thank you.

David Federlein

http://engage.eucalyptus.com

using aws, eucalyptus and chef for the optimal hybrid cloud

Technology