cloud computing for developers and architects - qcon 2008 tutorial

San Francisco 2008

Cloud Computingfor Developers & Architects

Stuart CharltonChief Software Architect, Elastra

San Francisco 2008

Tutorial Objectives

Provide an overview of the emerging cloud industry, the jargon, the trends, and a model to help sort through the mess

Dig into a couple of specific examples on how to provision and operate a cloud environment, conveying practical insight

Explore cloud computing architectures, looking at whether they change traditional system architectures

2

San Francisco 2008

About Your PresenterStuart Charlton• Canadian,

now in San FranciscoChief Architect, Elastra• Responsible for technical

direction & long-termproduct strategy

In prior lives... • BEA Systems,

Rogers Communications, Financial Services,global training & consulting

Stu Says Stuffhttp://stucharlton.com/blog

3

http://stucharlton.com/blog

http://stucharlton.com/blog

San Francisco 2008

Agenda - Part 1

A Look at the Clouds• (Good Luck) Defining Cloud Computing• Qualities of a Cloud• The Cloud Computing Industry - Late 2008• A Cloud Reference Model

Amazon Web Services Tutorial• Simple Storage Service (S3)• Elastic Compute Cloud (EC2)• Elastic Block Storage (EBS)• Covering APIs, Tools, and Experiences

4

San Francisco 2008

Agenda - Part 2

Managing & Operating Cloud Systems• Whither IT Service Management?• The Hope for Cloud Standards• The Puppet Administrative System• A Preview of Elastra Cloud ServicesCloud Architecture• Common Patterns• Integrating applications, networks, and data• Scalability and MonitoringQ&A and Open Discussion

5

San Francisco 2008

CaveatsThe technology is a (very) moving target• Expect this to increase as the industry tries to

drive a new round of retooling & spending• Lots to cover; we’ll try to scratch a reasonable

amount of surfaceMuch cloud technology is quite proprietary• Too early to dive into committee-land• Even if it’s open source, only one distribution

may eventually be problematicThe “definition game” is only fun for so long• Fondly recall the crisp and concise industry

definitions such as SOA, OO, Components, etc...

6

San Francisco 2008

A Look at the Clouds

San Francisco 2008 8

San Francisco 2008

(Good Luck) Defining Cloud Computing

Software-as-a-Service • “My customer resource management (CRM) system is

out on the Internet!”Grids vs. Clouds• Shared Virtual Resources• Batch Jobs vs. Online Applications• Different Approaches to State ManagementNetwork Diagrams• A service is “on a cloud somewhere”Virtualization Platforms & APIs• Hardware can be manipulated with software

9

San Francisco 2008

Qualities of a CloudOn-Demand• Lowered requirement to call-ahead forecasts• Demand trends are predicted by the provider

Usage-metered (i.e. an operating expense)• Pay-by-the-drink or over time, not up front

Self-service• Resources directly/indirectly reserved with a GUI or API

Elastic Scalability• Grow or shrink resources as required

Mandatory Network• The network is essential to consume the service

10

San Francisco 2008

A Subset of the Cloud Landscape

11

Large Providers

Mid-Size Providers

Software Vendors

San Francisco 2008

The Cloud Provider Continuum

12

Closer to theDeveloper/User

Closer to theSysAdmin/Ops

Platform-as-a-Service Infrastructure-as-a-Service

“Supplier Ecosystem”“Retail Ecosystem”


Software & Hardware Infrastructure

A Cloud Technology Reference Model

Testing,Monitoring,Diagnostics,

andVerification

Facilities &Logistics

Begin with the Basic Data Center





andVerification


Add easy software access to:Elements - HW/SW/Network/Storage Settings, Installations, and ConfigurationsResources - Reservations from a pool of excess capacity in storage, computing, and network

ElementManagement

ResourceManagement



Add some visibility:A Web of Metadata(What uses or contains what other things?)Lifecycle (when and how can things change?)



andVerificationFacilities &

Logistics

ElementManagement

ResourceManagement

Web of MetadataCategories, Capabilities, Configurations & Dependencies

Lifecycle(Birth, Growth, Failure, Recovery, Death)


A Cloud Technology Reference ModelAdd some real-world context:

Governance(Who has authority / responsibility to change, and how?)Architecture Views (How are my concerns addressed?)


andVerification



Architectural Views(e.g. scalability, availability, recovery,

data quality, security)Governance




Architectural Views

ElementManagement


ResourceManagement

Your Application



andVerification

Governance



Infrastructure Clouds Start Here:


ResourceManagement

Basic Monitoring




Architectural Views

Your Application


andVerification

Governance

TheirProblem

YourProblem

Operating System Images

Element Management

ElementManagement

(Split Responsibility)

San Francisco 200819

“Cloud Servers” Try to Extend Infra:


ResourceManagement

Basic Monitoring




Architectural Views

Your Application


andVerification

Governance

CloudInfra

(private orpublic)

Cloudservers

Yourproblem

San Francisco 2008

Cloud Platforms, As Perceived Today

20

Your Application(Insert Code Here)

Application-Level

Monitoring

lol,Governance

DON’T WORRY YOUR PRETTY HEAD,WE HAVE THE REST UNDER CONTROL


Application Lifecycle(Birth, Growth, Failure, Recovery, Death)

Scalability, Integration,Backup & Recovery, Security Views

Your Application

How Cloud Platforms Likely Will Evolve

App-LevelTesting,

Monitoring,Diagnostics,

andVerification

Governance

BLACK BOX OF INTRIGUE

San Francisco 2008

Amazon Web Services Tutorial

San Francisco 2008

AWS Registration and Security

Create an AWS account• aws.amazon.com• Attachable to your existing Amazon.com account

Creating an Access Key ID and Secret Key

23

San Francisco 2008

Simple Storage Service (S3)Web-Based Media Storage • Scalable, Redundant, Reliable, and Fast• XML-Based Metadata over RESTful Web Interface• Available over HTTP, HTTPS, and BitTorrent

Official 99.9% availability SLA (per month)• 10% service credit when between 99% and 99.9%• 25% service credit when less than 99%

Available in United States and EuropePricing (U.S.) - November 2008• Storage Rates: starting at $0.15 per GB monthly• Usage Rates: $0.10 inbound, $0.17 outbound• Request Rates: $0.01 per 10k GET, 1k POST, PUT, etc.• Rates are reduced as volume increases (multi-TB)

24

San Francisco 2008

S3 Conceptual Model

25

S3 Bucket

S3 Objects

/2008-11-08/QCon.html S3 Key

https://QConPages.s3.amazonaws.com/2008-11-08/QCon.html

QConPages

Mapped into:

https://s3.amazonaws.com/QConPages/2008-11-08/QCon.html

Protectedby ACL





San Francisco 2008

S3 RESTful Interactions

26

Creating Buckets as Resources

PUT /qconpages HTTP/1.1Host: s3.amazonaws.comDate: Mon, 17 Nov 2008 09:15:00 PSTAuthorization: AWS <AccessKeyID:signature>Content-Length: 0

ResponseHTTP/1.1 200 OKLocation: /qconpagesDate: Mon, 17 Nov 2008 09:15:01 PSTContent-Length: 0

San Francisco 2008


27

Writing objects in bucketsPUT /qconpages/QCon.html HTTP/1.1Host: s3.amazonaws.comDate: Mon, 17 Nov 2008 09:15:16 PSTAuthorization: AWS <AccessKeyID:signature>Content-Length: 104Content-Type: text/html

<html><head><title>QCon San Francisco 2008</title></head><body><p>Welcome!</p></body></html>

San Francisco 2008


28

Retrieving Objects

GET /HugeFile HTTP/1.1Host: qconpages.s3.amazonaws.comDate: Mon, 17 Nov 2008 09:15:16 PSTAccept: */*Range: bytes=0-1048579

(Range is an optional, standard HTTP, way to retrieve subsets and/or to resume broken transfers)

San Francisco 2008

Transfer Considerations

HTML Form Uploads• Content type is multipart/form-data• Hidden form fields can pass other parameters

Object Key, Authorization Signature, etc.

BitTorrent Access• Request /bucket/key?torrent for .torrent file• Object needs to be available by anonymous users• Other downloaders will contribute to the Torrent,

S3 will act as a seeder

29

San Francisco 2008

AWS Authorization Format

Ensures that requests were not tampered with and was authorized by the AWS account holder• An HMAC-SHA1 Algorithm applied to several

canonicalized HTTP headers and and content

Passed as an Authorization headerOptionally can be passed as URI parameters for pre-signed, expiry-based signatures

30

San Francisco 2008

Elastic Compute Cloud (EC2)Resizable Compute Capacity in the CloudCPU, Memory, Storage, and Network• Storage is “ephemeral” ; is lost on termination

Supports Linux, OpenSolaris, and Windows Server 2003Free data transfer • Between S3 and EC2• Among EC2 instances

In/Outbound data transfer similar price to S3Baseline CPU Speed is 1.0-1.2 Ghz AMD Opteron•aka. Elastic Compute Unit (ECU)

31

San Francisco 2008

EC2 Sizes

32

Size Cores / Speed Storage Memory Cost

Small1 Core, 1 ECU(32-bit) 160 GB 1.7 GB

$0.10/hr (*NIX)$0.125/hr (Windows)

Large2 Core, 2 ECU(64-bit) 850 GB 7.5 GB


X-Large4 Core, 2 ECU(64-bit) 1690 GB 15 GB


High CPU Medium

2 Core, 2.5 ECU(32-bit)

350 GB 1.7 GB $0.20/hr (*NIX)$0.30/hr (Windows)

High CPUX-Large

8 Core, 2.5 ECU(64-bit)

1690 GB 7 GB $0.80/hr (*NIX)$1.20/hr (Windows)

San Francisco 2008

The Lazy Developer’s Tool: Elasticfox

33

San Francisco 2008

EC2 Authorization Keypairs

Amazon EC2 uses an x.509 Certificate and Private Key pair to enable authorizationOn Linux & UNIX: • Passwordless-SSH

On Windows:• Keypair is used to access administrator password

Generate your own (e.g. Elasticfox), or use Amazon’s web interface

34

San Francisco 2008

Image Management

Amazon Machine Images (AMIs)• A copy of the OS filesystem, minus the kernel• Chunked up into smaller pieces, uploaded to S3• After uploading, can be registered with EC2

Library of AMIs available through EC2 API• Amazon-provided AMIs

e.g. Fedora 8, Windows Server 2003

• Publically-available 3rd Party AMIse.g. OpenSolaris, various Linux distros

• Paid-AMIs• Private (your own) AMIs

35

San Francisco 2008

Instance Management

Launching an AMI• Select the min/max

number of instances desired• Choose security groups• Choose instance size• Ensure OS fits the size

(i.e. 32 vs 64-bit)• Choose the registered

keypair for authentication

36

San Francisco 2008

Availability Zones

A grouping of the data centre infrastructure that’s isolated from other infrastructure• Could be in the same data centre, just redundant

power, HVAC, etc.Generally, failures in one zone will not impact the other zones (except for catastrophic failure)In future, regions will also be available for planned disaster recovery.

37

San Francisco 2008

EC2 Query API

Intuitive Functions• Describe*

AvailabilityZonesImagesInstancesKeyPairsSecurityGroups

• RunInstances• TerminateInstances

Constructed via URI (not RESTful, though)• https://ec2.amazonaws.com/?Action=RunInstances&ImageId=ami-60a54009..

38

https://ec2.amazonaws.com

https://ec2.amazonaws.com

San Francisco 2008

Image Bundling

Bundling Images on Linux & UNIX• ec2-bundle-vol utility run on the instance• ec2-upload-bundle utility to send to S3

Bundling Images on Windows• ec2-bundle-instance API wrapper cmd

39

San Francisco 2008

Example of Launching an InstanceUsing the Typica Toolkit (Java Wrapper)http://code.google.com/p/typica/

List<String> params = new ArrayList<String>();List<ImageDescription> images = ec2.describeImages(params);for (ImageDescription img : images) { if (img.getImageId().equals(“ami-2a5fba43”)) ReservationDescription = ec2.runInstances(img.getImageId(), 1 /*min*/, 1 /*max*/, securityGroups, “”, “mykeypair”);}

40

http://code.google.com/p/typica/

http://code.google.com/p/typica/

San Francisco 2008

EC2 Security Groups

41

Database

LoadBalancer

AppServer

Web SecurityGroup

Data Security Group

Virtual Group-Based Firewalls in the EC2 Data Center

CIDR-based group firewall forexternal clients (e.g. 0.0.0.0/0)

San Francisco 2008

EC2 NetworkingEach instance is given a Public Dynamic Host:• e.g. ec2-33-131-3-227.compute-1.amazonaws.com

And a Private Host for within EC2:• e.g. domU-10-21-18-00-69-D5.compute-1.internal

Cross-Instance Traffic should almost always use the Private HostNo UDP Broadcast or IP Multicast is allowedElastic IP• Static public IP address, allocated within 24 hours• Attaching an Elastic IP may take ~15 minutes• Note that it asynchronously replaces your public

dynamic host name & IP address without warning

42

San Francisco 2008

Elastic Block Storage

Persistent, highly-available, block storage• (Similar experience to a SAN)

Released August 2008Volumes between 1GB to 1TB• Multiple volumes allowed

RAID striping allowed (bandwidth constrained at ~100+ MB/sec)Supports snapshots to S3 for later restore• Snapshots are asynchronous and take a long time• Restores, on the other hand, are relatively quick

43

San Francisco 2008

Elastic Block Storage APICreate/DeleteVolume

Attach/DetachVolume

Create/DeleteSnapshotDescribeVolumes

EBS Storage is normally provisioned very quickly (seconds)Initial writes will be slow, as with ephemeral stores; All EBS volumes must be formatted with a file system prior to use

44

San Francisco 2008

End of Part 1

San Francisco 2008

Agenda - Part 2

Managing & Operating Cloud Systems• Whither IT Service Management?• The Hope for Cloud Standards• Tutorial - The Puppet Administrative System• A Preview of Elastra Cloud ServicesCloud Architecture Topics• Common Patterns• Integrating applications, networks, and data• Security (Identity, Privacy, etc.)• Scalability and MonitoringQ&A and Open Discussion

46

San Francisco 2008

Managing & Operating Cloud Systems

San Francisco 2008

How have we managed our IT?

Developer-led• Concurrent Versioning, Unit Testing, Maven, Ant, Capistrano• Focused on code-promotion ; sometimes database transform

Manager-led• One extreme: firefighting• The other extreme: bureaucracy

Architect-led• Round-trip modeling tools (e.g. Rational UML, Together, etc.)• Gated reviews (i.e. “The technology cops”)

Operations-led• Management suites (OpenView, Tivoli, etc.)• Runbook Automation (e.g. HP/OpsWare, BMC/BladeLogic, Opalis)

48

San Francisco 2008

IT Infrastructure Library (ITIL) v3:The Current Best Practice?

49

San Francisco 2008

Dependency Management vs. UniformityThe “Google Secret Sauce” Theory:• Always available, scalable, fast• Computing as fungible commodity• Reliability is enabled by architecture• But you have to rewrite your software

Does a seemingly magical architecture reduce or eliminate the need for configuration & dependency management?Does this architecture match classic enterprise requirements?

50

If I spill this on aserver, who

is affected, and by how much?

San Francisco 2008

EC2 is great, but...

That’s a lot of images!That’s a heckuva lot of instances!How do I change many machines at once?• Scripts that wrap SSH?

Do I need to re-image every time I add/update software?How do I detect configuration drift?

51

San Francisco 2008

The Puppet Administrative System

An Open Source Runtime System andDomain-Specific Language (DSL) for managing Linux, BSD, & UNIX servers• Maintained by Reductive Labs since 2005• Founded by Luke Kanies, ex-BladeLogic

Encapsulates cross-package installation, configuration setting, permissions, etc. in a transactional runtime

52

San Francisco 2008

Puppet Architecture

Puppetmasterd• Maintains central configuration repostiory

Puppetd• Agent on each client, polls the puppetmaster

every 30 minutes (adjustable)

53

San Francisco 2008

Puppet Manifest Example

class mysql::server { $mountpath = $mydc::constants::ebs_mount $datadir = "${mountpath}/mysql"

package { "mysql-server": ensure => installed, } include amazon::ebs file { $datadir: ensure => directory, require => Exec["Mount Device"] }}

54

San Francisco 2008

Puppet Sites & Nodes

node “web.qconsf.com” {

include apachewebserver

}

node “mysql1”, “mysql2” {

include mysql::server

}

55

Declaratively Adds Infrastructure to Nodes

San Francisco 2008

Security

Puppetmasterd provides a form of PKI for deployments• Clients are authenticated via keypairs• Can act as a Self-Signed Certificate Authority or

use a registered certificateCurrent encrypted XML-RPC being transitioned to RESTful HTTP in a future release

56

San Francisco 2008

Inventory and Drift Control

Puppet includes Facter, a system inventory tool• Returns facts about nodes

e.g. hostnames, kernel, IP addresses, etc.

• Facts can then be used in Puppet configurations• Detects changes and updates information

57

San Francisco 2008

Elastra Cloud Services

Today• Load Balanced, Clustered & Recoverable MySQL,

PgCluster, and Apache Tomcat 5.5• Turn-Key Deployment on Amazon EC2• Private beta support for VMWare or Eucalyptus

In Early 2009•Elastra Cloud Suite v2.0

Enterprise cloud server for IT services management •Open Cloud Services

Resource Provisioning APIConfiguration Management APIAdministrative Tools & Utilities

58


IterativeDesired-State

Design

Markup For Each Role,

GUI managementinterface

Wire FundsWeb App

Msg Bus

WireProcess

AcctSvc

DB

ApplicationDesign(ECML)

TomcatV 5.5

Mule ESB 1.6

WLS10.1

MySQL

DeploymentDesign(EDML)

Lombardi6

Elastra Design & Deploy Lifecycle

59

San Francisco 2008

IT

Private BankingApplication

MortgageApplication

ECMLECMLECMLEDMLEDMLEDML

Application Architect

BusinessUnit B

Application Architect

BusinessUnit A

StandardInfrastructure

Images

App AParameters

ConfiguredInfrastructureInstance

App BParameters

ConfiguredInfrastructure

Instance

Reuse Mechanisms,Standards,

Best PracticesSystems

ArchitectsSystemAdmins

Business UnitArchitects Focus

On Business Logic

Helping Drive a Collaborative IT Process

60

San Francisco 2008

Lifecycle-Managed Architectures

61

PgCluster Data Component

PgCluster Data Component

PgCluster Load Balancer

Load Balancing Connector

PgCluster Replication Component

ReplicationConnector

Scalability Policy

ResourceAllocationStrategy

MonitoringPolicy

San Francisco 2008

Cloud Architecture Topics

San Francisco 2008

Recurring Topics and Patterns

Some design decisions and tradeoffs are continually associated with the cloudSome designs are due to fundamentals• e.g. CAP Tradeoffs

(Consistency, Availability, Partitioning)Others are due to out-of-date software• Assuming a single machine• ...On a local area network• ...With reliable nodes

63

San Francisco 2008

Availability > Consistency

Increasingly common way of handling higher loadsLocks & distributed transactions reduce availability• If my data is locked, it’s not available!

A variety of techniques enable this• Caching everywhere (e.g. Memcached, Gigaspaces)• Distributed Replication (e.g. MySQL slaves)• Compensating transactions

64

San Francisco 2008

Stateless Web / Application ServersWhat?Servers do not maintain state between requests (pushed to database or client)Why?• Scalability - smaller working set to manage;

session replication becomes hard at scale• Reliability - easier to recover when there is no

conversational state• Support - EC2 doesn’t support multicast for

session replicationDanger: Most enterprise web application development still makes heavy use of sessions

65

San Francisco 2008

Partitioned DatabasesWhat?• Partitioning, also known as Sharding, or (loosely)

Shared-Nothing, spreads the load across multiple instances by having each manage a subset of data

Why?• Scale-up breaks down fairly quickly when dealing with

spikes; scale out becomes the viable option• Shared-disk databases tend to be commercial and require

high-end SANsDanger:• Cross-partition communication is very slow - must have

good data locality or heavily denormalize• Doesn’t help scale “hot” write-intensive data!• Quite unfamiliar to enterprises used to large-SMP Oracle

databases

66

San Francisco 2008

Stateless Workers

The most common case for elastic scalability• e.g. Animoto’s 50 -> 3600 -> 100 servers

Appropriate for computationally intensive processingThough much of Enterprise IT’s processing needs are I/O-bound, not CPU-bound

67

San Francisco 2008

Federated IdentityFrom Lookup to AssertionsSAML, WS-Federation, OAuth

Major Feature of Windows Azure

68

Private Cloud

Public Cloud

OtherCloud

IdentityDelegatedIdentity

San Francisco 2008

Auto-Scale, Monitoring and Diagnosis

The Journey of Monitoring• From Log Management & Search• ... to Aggregation and Statistics• ... to Event Correlation• ... to Complex Event Analysis

How in-depth is necessary depends on how predictable or unique your application design is!

69


ApplicationDesign

DeploymentDesign

Cloud Deployment

Virtualization Layer

Configured Software Infrastructure

3. Correlating Events for Diagnosis

Out of MemoryErrors

(cause)

Failed WireTransfers(effect)

1. AggregatingMonitoring

Data

2. Log Mining

Monitoring Service

Monitoring Service

Auto-Scale, Monitoring and Diagnosis

70

San Francisco 2008

ConclusionCloud Computing comes in many shapes and sizes• From Infrastructure • ...to Middleware• ...to Entire Platforms

Reduces Lead Time to Deploy Systems• With varying degrees of visibility

The full impact on IT Management & Operations is still unknown• Chances are it won’t eliminate what we do today

Cloud architectures promote what were secondary problems to a higher status (e.g. integration, security)

71

San Francisco 2008

Thank You

Stuart CharltonChief Software Architect, Elastra