building private-clouds-qconsf

Implementing Private Clouds

Andrew Clay ShaferQCon SF 2010

Andrew Clay Shafer

@littleidea

Andrew Clay Shafer

@littleidea

Andrew Clay Shafer

@littleidea

Andrew Clay Shafer

@littleidea

Andrew Clay Shafer

@littleidea

Andrew Clay Shafer

@littleidea

Andrew Clay Shafer

@littleidea

real artists ship

great artists steal

pri·vate

http://www.flickr.com/photos/athomeinscottsdale/3279949186/

pri·vate• confined to or intended only for the persons immediately concerned; confidential

http://www.flickr.com/photos/athomeinscottsdale/3279949186/

pri·vate• confined to or intended only for the persons immediately concerned; confidential

• not open or accessible to the general public

http://www.flickr.com/photos/athomeinscottsdale/3279949186/

pri·vate• confined to or intended only for the persons immediately concerned; confidential

• not open or accessible to the general public

• undertaken individually or personally

http://www.flickr.com/photos/athomeinscottsdale/3279949186/

cloud

cloud- noun• a dim or obscure area in something otherwise clear or transparent.

cloud- noun• a dim or obscure area in something otherwise clear or transparent.

- verb• to make obscure or indistinct; confuse

Chance of Tangential Ranting and Meandering Editorialization

Warning

is_a?(Cloud)

CloudTaxonomies

CloudTaxonomies

CloudTaxonomies

Too Much *aaS

CloudStandards

CloudStandards

CloudStandards

I have no standards.

If someone tells you they have a cloud solution you can install in a rack...

If someone tells you they have a cloud solution you can install in a rack...

ask them for the brochure with the architecture...

If someone tells you they have a cloud solution you can install in a rack...

ask them for the brochure with the architecture...

and how much it costs...

If someone tells you they have a cloud solution you can install in a rack...

ask them for the brochure with the architecture...

and how much it costs...

and if they will throw in a free bridge.

Where did cloud arise?

Where did cloud arise?

Where did cloud arise?

Where did cloud arise?

Where did cloud arise?

Where did cloud arise?

Industrial Revolution

Revolutionized

Scale Changes Everything

So, you are sold, you are ready to build your own robotics IT factory, what’s next?

Shall we build a compute cloud?

NO

We will build web applications that brokers access to compute resources.

NO

Automation

AutomationAbstraction

AutomationAbstractionReliability

AutomationAbstractionReliabilityInnovation

AutomationAbstractionReliabilityInnovationDisruption

To hypervise or not to hypervise, that is the question?

To hypervise or not to hypervise, that is the question?

So many choices

“It works at my desk”

quacks_like_a?(Cloud)

Reliability...

6 Laws Of ReliabilityIsolation

Concurrency

Failure Detection

Fault Identification

Live Upgrade

Stable Storage

6 Laws Of ReliabilityIsolation

Concurrency

Failure Detection

Fault Identification

Live Upgrade

Stable Storage

Joe Armstrong

if p( fail | one unit) = 10-3 then

p( fail | n units) = 10-3xn

Isolation

if p( fail | one unit) = 10-3 then

p( fail | n units) = 10-3xn

Isolation

So 10 nines should be easy, we just need 4 units, amirite?

n > 1

Concurrent

Problem Solved?

Cloudin’ it up

Cloudin’ it up

Sorta...

Simple Services: Single Responsibility

Some people, when confronted with a problem, think “I know, I'll use a message bus.”

Some people, when confronted with a problem, think “I know, I'll use a message bus.”

Now they have queue problems.

The plot thickens... Users? Images?

Kernels?

Metering?

Performance?

Auditing? Upgrades?

Deployment?Security?

Authentication?

Server Size?

Complexity Much?Users? Images?

Kernels?

Metering?

Performance?

Auditing? Upgrades?

Deployment?Security?

Authentication?

Server Size?

But wait, the cloud IS an application?

How do we build and deploy that?

automate the automation

automate the automation

If it’s not meta-circular, you are doin’ it wrong.

Configuration Management

Configuration Management

solve this problem

Developers, Developers, Developers...

Developers, Developers, Developers...

It’s the applications stupid...

How do you deliver value?

Continuous Delivery/Deployment

Black’s Theorem:the wrong abstraction is worse than no abstraction.

Live Upgrade

Live Upgrade

systems that never stop

Speaking of frameworks...

Communities of Practice

Communities of Practice

6 Laws Of Reliability

6 Laws Of ReliabilityIsolation

Concurrency

Failure Detection

Fault Identification

Live Upgrade

Stable Storage

6 Laws Of ReliabilityIsolation

Concurrency

Failure Detection

Fault Identification

Live Upgrade

Stable Storage

unmentionables

6 Laws Of ReliabilityIsolation

Concurrency

Failure Detection

Fault Identification

Live Upgrade

Stable Storage

unmentionables

WWJAD

If a cloud falls in a forest and no one is there to hear it, do we still have 100% uptime?

UndeliberateDiscovery

oh.

UndeliberateDiscovery

oh.

Oh!

UndeliberateDiscovery

oh.

Oh!

WTF?!??

UndeliberateDiscovery

oh.

Oh!

WTF?!??

OMG?!??

UndeliberateDiscovery

oh.

Oh!

WTF?!??

OMG?!??

UndeliberateDiscovery

*crying*

Untested Vs. Unmonitored

Untested Vs. Unmonitored

a well behaved cloud should inform you of it's health

Cloud Hobbyists

TCO and ROI crossing the chasm

cause cloudin’ ain’t easy

cause cloudin’ ain’t easy

scale

cause cloudin’ ain’t easy

web ops

scale

cause cloudin’ ain’t easy

web ops

networkingscale

cause cloudin’ ain’t easy

web ops

networking

storage

scale

cause cloudin’ ain’t easy

web ops

networking

storage

scale

power

cause cloudin’ ain’t easy

web ops

networking

storage

scale

coolingpower

cause cloudin’ ain’t easy

web ops

networking

storage

scale

coolingpower people

cause cloudin’ ain’t easy

web ops

networking

storage

scale

coolingpower

support

people

cause cloudin’ ain’t easy

web ops

networking

storage

scale

coolingpower

trainingsupport

people

SO YOU THINK YOU CAN

CLOUD

SO YOU THINK YOU CAN

CLOUD now holding auditions across the world, looking for the top clouders...

finandrew@cloudscaling.com

@littleidea

•IRC and mailing lists

•http://cloudscaling.com/blog

•http://perspectives.mvdirona.com/

•http://www.kitchensoap.com/

•http://www.planetdevops.net/

•http://www.jedi.be/blog/

•http://highscalability.com/

•Velocity, Surge, LISA