how to build scalable websites in the cloud

1

How to Build

Scalable Web Applications

in the Cloud

Reference Architectures and Best Practices

December 14, 2010

2

Your Panel Today

Presenting:

• Michael Crandell, CEO

• Brian Adler, Sr. Professional Services Architect

Q&A:

• Jarryd Hensel, Account Manager

Please use the questions window to ask questions any time!

3

Agenda

• What’s, why’s, when’s, and how’s of scalable web apps

• Comparison of traditional and cloud resource models

• Reference architecture for a scalable web application

• Best practices for each tier of the reference architecture

• Summary & conclusions

Please use the questions window to ask questions any time!

4

Scalable Web Application

• What?

• An application built on an architecture that can adapt to changing

conditions

5

Scalable Web Application

• What?

• An application layered on an architecture that can adapt to changing

conditions

• Why?

• Traffic and load patterns are unpredictable• Viral or flash-mob events can result in very dynamic conditions

• Availability and Reliability• Application must be distributed to increase likelihood of end-user accessibility

• Overprovision• Under-utilized resources == wasted $$$

• Underprovision• Missed opportunities – users unable to access your site/product

• Don’t be a victim of your own success

6

Pre-Cloud Provisioning

7

Cloud-enabled Provisioning

8

Reference Architecture

9

Load Balancing Tier

10

Load Balancing

• HAProxy + Apache

• Can handle SSL termination on the load balancer

• Connection statistics available via socket connection and status web page

• Each instance can handle a specific amount of traffic with no ramp-up time

• Each instance can only handle a specific amount of traffic

• Addition of load balancers is possible, but requires DNS modifications

11

Load Balancing

• HAProxy + Apache

• Can handle SSL termination on the load balancer

• Connection statistics available via socket connection and status web page

• Each instance can handle a specific amount of traffic with no ramp-up time

• Each instance can only handle a specific amount of traffic

• Addition of load balancers is possible, but requires DNS modifications

• Elastic Load Balancer (ELB)

• SSL termination is now supported

• Can scale to handle large amounts of traffic, but can be slow to ramp-up• Options do exist for “pre-warming” the ELB

• Only need one – it will scale to accommodate traffic load

• Essentially a load balancing appliance• No visibility into inner-workings and/or connection rates, statistics, failures, etc.

• (RightScale has a technical white paper on load balancing solutions that is

available at www.RightScale.com/whitepapers)

12

Load Balancing

• Load Balancer + Application server

• Possible, and good for test and dev

• Not a best practice for a production environment• Traffic spikes can cause instance to perform both load balancing and application

functions…poorly

13

Load Balancing

• Load Balancer + Application server

• Possible, and good for test and dev

• Not a best practice for a production environment• Traffic spikes can cause instance to perform both load balancing and application

functions…poorly

• Recommendation: Minimum of two load balancers

• Each load balancer should be in a different availability zone (AZ) to

increase reliability and availability

• RightScale testing has shown that m1.large is a good choice for load

balancers• Due to 100K-120K packet-per-second limit, larger instances do not provide much gain in

throughput

• Roughly 5K responses/second can be handled by m1.large

• With the 5K threshold in mind, select the number of load balancers required to handle your

peak traffic

14

Application Server Tier

• Puts the “scalable” in a scalable application

• True autoscaling a must in any dynamic/unpredictable

environment

15


• Autoscaling

• Fully automated server launch based on autoscaling triggers

• No manual intervention (can be challenging in certain environments, i.e.

Windows)

• Download and install application code from common repository to ensure

identical configuration of all servers in the tier

16


• Autoscaling

• Fully automated server launch based on autoscaling triggers

• No manual intervention (can be challenging in certain environments, i.e.

Windows)

• Download and install application code from common repository to ensure

identical configuration of all servers in the tier

• Triggers

• Common• CPU idle

• Free memory

• System load

• Custom• Web server connections

• Application-specific metrics

17


• When to scale?

• Conservatively. Both up and down

18


• When to scale?


• Up

• Allow adequate lead time for new servers to become operational

• Before system is negatively impacted

• Look for trends in activity and react early

• Worst that can happen: Charged for an extra instance hour

19


• When to scale?


• Up

• Allow adequate lead time for new servers to become operational

• Before system is negatively impacted

• Look for trends in activity and react early

• Worst that can happen: Charged for an extra instance hour

• Down

• When system has been underutilized for a consistent, consecutive period

of time

• Scale down fewer servers than in a scale-up event

• Again, only downside is an extra hour of instance charge

• Better safe than sorry

20


• Array considerations

21



• Weight the array across all availability zones (not regions)• Increases reliability of application

• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte

charge

• Traffic between regions is charged at public Internet rates

22





charge


• Set minimums and maximums appropriately• Minimum can assist in cost savings in times of low usage

• Maximum can limit overall cost exposure

23





charge




• Instance size• m1.large is typically a good choice for array-based servers in a production environment

• m1.smalls (and even micro instances) can be used in test and development environments

• Every application is different, so run load tests and benchmarks to find the optimal solution for

your environment

24





charge




• Instance size• m1.large is typically a good choice for array-based servers in a production environment

• m1.smalls (and even micro instances) can be used in test and development environments

• Every application is different, so run load tests and benchmarks to find the optimal solution for

your environment

• Code Deployment• Updated code can be pushed to all servers in an array via a single click of a button

25

Caching Tier

• Caching can dramatically decrease the load on the database

• Particularly in read-intensive applications

• Costs of caching

• Application complexity/modification

• Additional instance hours to support the cache

26

Caching Tier

• Best practice is to have a separate, dedicated caching tier

• Caching can be implemented on each application server• Prevents the use of a distributed cache

• Local cache should only be used by the co-resident application server

• Application complexities

• Database performance degradation

27

Caching Tier






• Instance Size and Count

• Determine memory caching footprint• Select instance size and count that spreads the load over several servers

• Prevents loss of entire cache if a single instance fails

• Distribute caching servers across AZs for reliability

• Overprovision if possible

• Provide capacity for system to grow to fully utilize cache (budget permitting)

28

Caching Tier






• Instance Size and Count

• Determine memory caching footprint• Select instance size and count that spreads the load over several servers

• Prevents loss of entire cache if a single instance fails

• Distribute caching servers across AZs for reliability

• Overprovision if possible

• Provide capacity for system to grow to fully utilize cache (budget permitting)

• Manually scaling caching servers is possible but non-trivial

• Involves application and database performance degradation

• Time To Lives (TTLs)• Always set to expire

29

Caching Tier

• Write-intensive applications

• Don’t see as large a performance gain as read-intensive apps

30

Caching Tier

• Write-intensive applications

• Don’t see as large a performance gain as read-intensive apps

• Third-party providers

• Vendor solutions exist that allow dynamic memcached scaling

31

Database Tier

• Numerous database architecture options exist

• No “one size fits all” solution, so testing and benchmarking are

critical to determine proper configuration

32

Database Tier

• Masters and Slave(s)

• Multiple Slaves if budget permits

• Distribute Master and Slave(s) across AZs

• Always use same instance size for Master and Slaves

33

Database Tier





• Data Storage

• EBS volumes for data store

• Never use ephemeral storage for persistent data

• Back up Master and Slaves frequently

• Upload snapshots to S3 or some other persistent, redundant storage

34

Database Tier





• Data Storage

• EBS volumes for data store

• Never use ephemeral storage for persistent data

• Back up Master and Slaves frequently

• Upload snapshots to S3 or some other persistent, redundant storage

• Instance Size

• Varies greatly based on the nature of the application and site traffic

• Load testing and benchmarking can assist in identifying a reasonable

initial size

35

Database Tier

• Relational Database Service (RDS)

• Database Appliance• No access to instance (no visibility into CPU utilization, memory usage, slow-query logs, etc.)

• Requires scheduled downtime• Announcement of multi-AZ functionality in May 2010 allows 24/7 operation

• Read replicas announced in October 2010

36

Database Scaling

37

Database Scaling

• Vertical

• “Grow” or “shrink” a database server from one instance size to another

38

Database Scaling

• Vertical

• “Grow” or “shrink” a database server from one instance size to another

• Horizontal

• Add additional servers to spread the database load

39

Database Vertical/Horizontal Scaling

RightScale Makes Vertical or Horizontal Scaling Easier

SmallType A

SmallType B

SmallType B

SmallType B

SmallType A

SmallType A

LargeType A

LargeType B

HorizontalScaling

VerticalScaling

More servers of same types

Same quantity of larger servers

40

Horizontal Database Scaling

• Addition of read Slaves

• Effective for read-intensive applications

• Only writes need to access the master

• Replication lag to slaves must be considered

41


• Addition of read Slaves

• Effective for read-intensive applications

• Only writes need to access the master

• Replication lag to slaves must be considered

• Effective mechanism is to use MySQL Proxy

42


• Sharding

• Concept is to partition the database into distinct, non-overlapping pieces

• “Horizontal slicing” of the database tables into groups of rows

• Forethought required in setting up shards since cross-shard joins are

resource intensive

43


Before Sharding

44


After Sharding

45


• Master-Master

• Two (or more) Master DBs

• Any Master can modify any database object

• Replication lag can result in database inconsistencies

• Poorly-designed applications can cause data object collisions and leave

databases in indeterminate state

• Not a recommended best practice, nor supported by RightScale

46


• NoSQL solutions

• Many options exist –

SimpleDB, Cassandra, Membase, CouchDB, MongoDB, Riak, etc.

• Basically a Key/Value store

• No complex operations between data objects (no relational operations)

• Multiple nodes can be used to implement a distributed data store• Coordinated backup and recovery can be challenging

• Some RightScale customers are beginning to use some NoSQL solutions

in specific use cases.

47

So What’s Best?

48

So What’s Best?

• As is typical in many technology discussions…

49

So What’s Best?


“It depends”

50

So What’s Best?


• Many scalable environments share some common underlying

architecture concepts

“It depends”

51

So What’s Best?




• Every application is different. As such, there is no “one size fits

all”

“It depends”

52

So What’s Best?




• Every application is different. As such, there is no “one size fits

all”

• But…

“It depends”

53

Mix-n-Match Pre-Built Components

Operating Systems

Windows 2003/2008

CentOS

Ubuntu

SUSE

Language/Framework

PHP

Ruby/Rails

Java/Tomcat

.NET

Python/Django

Database

MySQL

SQL Server

MySQL Proxy

dbShards

Membase

Web Server

IIS

Apache

Nginx

Lighttpd

Load Balancing

HAProxy

Elastic Load Balancer

Zeus

aiCache

Automation

Chef

Ruby

Bash

PowerShell

54

Have a project, but need some help?

Contact us: [email protected] or (866) 720-0208

Ready to get started?

Sign up for our Free Edition: www.RightScale.com/Free

Call us for a VIP trial of our paid editions

Need to learn more?

Scalable Web Apps white paper – Coming Soon!

TCO calculator: www.RightScale.com/tco-calculator

Webinar archive: www.RightScale.com/webinars

White papers: www.RightScale.com/whitepapers

Q&A / Getting Started

mailto:[email protected]

http://www.RightScale.com/Free

55

Thank You!

how to build scalable websites in the cloud

Technology