aws webcast - accelerating application performance using in-memory caching in aws

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

Accelerating Application Performance

Using In-Memory Caching in AWS

Omer Zaki ([email protected])

Shakil Langha ([email protected])

mailto:[email protected]

mailto:[email protected]


Agenda

Caching Overview

Memcached

Amazon ElastiCache

Use Cases & Design Patterns

Demo


A Typical Web Application

Web

Server

DB

Server

Users

Web Server

Database Server



Web

Server

DB

Server

Step 1: Users hit your web app



Web

Server

DB

Server


Step 2: The Web App queries the Database



Web

Server

DB

Server



Step 3: The Database returns the Result



Web

Server

DB

Server



Step 3: The Database returns the Result

Step 4: The Web App returns the Page



Web

Server

DB

Server

Step 5: You scale Web and Database Servers

Web

Server

DB

Server

Web

Server

DB

Server



Web

Server

DB

Server


Step 6: Now What?

Web

Server

DB

Server

Web

Server

DB

Server



Web

Server

DB

Server


Step 6: Now What?

Step 7: Use a Cache

Web

Server

Web

Server

DB

Server

Cache


What is a Cache?

Component that stores frequently accessed data in memory

Requests for data in the cache are returned faster

Benefits:

• Faster responses for cached data

• Reduces load on your database


Why Use a Cache?

For a majority of web applications, workloads are read heavy • Often as high as 80-90% reads vs. writes

Memory is orders of magnitude faster than disk • Latency can be reduced from milliseconds to microseconds


Caching: An Example

Example: Get John Smith’s Phone Number

First Time:

Cache = {}

Step 1: Look in the cache. It’s empty; ‘cache miss’

Step 2: Get the value from the DB; update the cache


Caching: An Example

Example: Get John Smith’s Phone Number

Second Time:

Cache = {‘John Smith’ : ‘206-555-1212’}

Step 1: Look in the cache. Return the result; ‘cache hit’

Step 2: Until John’s number changes, we don’t hit the DB


Caching: An Example

Example: John gets a new phone number – ‘415-555-1212’

Cache Invalidation:

Cache = {‘John Smith’ : ‘206-555-1212’} (out of date)

Step 1: Update Database with John’s new number

Step 2: Update the cache with John’s new number

Cache = {‘John Smith’ : ‘415-555-1212’}


Caching: Pros & Cons

Pros • Speed up applications, especially if they are read-intensive

• Reduce load on your database servers

Cons • Requires a change in application architecture

• Application needs to ensure data in the cache is current


Agenda

Caching Overview

Memcached

Amazon ElastiCache


Demo


Memcached: Overview

Memcached (read: mem-cache-dee)

Free, open-source, high-performance, in-memory key value store

Developed for LiveJournal in 2003

Used by many of the worlds top websites:

• YouTube, Facebook, Twitter, Pinterest, Tumblr, …


Memcached: Architecture

Servers with key-value associative

arrays

Client software populates the array

and queries it

Clients know about all the servers

Servers don’t communicate with

each other

Least Recently Used

No persistence

Source: http://architects.dzone.com/news/notes-memcached


Memcached: Storing Data

Stores arbitrary data • HTML snippets

• Database query results

• Computed results

Keys are limited to 250 bytes

Values are limited to 1Mb


Agenda

Caching Overview

Memcached

Amazon ElastiCache


Demo


Amazon ElastiCache Web service that lets you easily create and use cache clusters in the cloud

100% Memcached compatible

Managed, scalable, secure

Pay-as-you-go, and flexible, so you can add capacity when you need it


Amazon ElastiCache: Benefits

Easy to Deploy

Deploy multi-node cache

clusters with a few button

clicks or API calls

Easy to Migrate

100% Memcached compatible

Existing code will work when

you update server lists

Easy to Administer

Automatically replaces failed

nodes and patches software

CloudWatch enables you to monitor cache performance

metrics

Easy to Scale

Add or remove cache nodes

with a few button clicks, or API calls

Easy to Secure

Works well with EC2 and DB Security

groups which means the cache is secure


Agenda

Caching Overview

Memcached

Amazon ElastiCache


Demo


Common Use Cases

Social networks

Gaming

Media and News Sites

Q&A Portals

E-Commerce

Recommendation Engines

Mobile app back-ends


Sample Deployment: Gaming

Auto-scaling front end

Amazon ElastiCache

Amazon RDS

Amazon S3

Amazon CloudFront


Common Design Patterns

Database offloading

• Allows web apps to scale independent of backend data tier

Session management for transient data

• Increment/decrement high scores for gaming environments

In memory storage for difficult / time consuming tasks

• Picture ids for all pictures tagged with a particular keyword


Anti-Patterns Build application logic assuming cache

hit latency

Put objects into cache without Time-

To-Live set (TTL)

Sequentially sending single operations

to cache (avoiding usage of batched

operations)

Configuring Memcached client to use

Redistribute Failure mode for

ElastiCache cache nodes

Resolving DNS only once.

Failing application request for cache

miss.

Using Memcached as a durable store.

Not designing application error paths

regarding cache operations

• Connection timeouts, operation

timeouts, etc.


Best Practices

Share Memcached Client objects in application

Use TTLs (short TTLs)

Consider Memory for Connections Overhead

Use CloudWatch Alarms / SNS Alerts

Use Consistent Hashing

Use Auto Discovery


Agenda

Caching Overview

Memcached

Amazon ElastiCache


Demo


Adding Caching to a Web App

Existing Web App running in EC2

• Using RDS MySQL database

Steps

1. Create ElastiCache Cluster with 2 cache nodes

2. Modify the Web App to use Caching


Create Cache Cluster: Cluster Specification

https://console.aws.amazon.com/elasticache/home?region=us-east-1


Create Cache Cluster: Security / Parameters


Create Cache Cluster: Cluster Created


Modify Web App to use Caching

Initialization import memcache

CACHE_NODES = ['hostname1:port1',

'hostname2:port2']

mc = memcache.Client(CACHE_NODES)

Key Lookup KEY = "demo.key"

KEY_EXPIRY_SECONDS = 60

value = mc.get(key)

if (not value):

value = getDataFromExistingLogic()

#retrieve from existing app logic -

database, filesystem, other processing

mc.set(key, value, time =

KEY_EXPIRY_SECONDS)

return value


Web App with Caching


Performance Improvement

Performance run using Multi-Mechanize

• Runtime: 600 seconds

• Time-series interval: 5 seconds

• ElastiCache usage configured to start at 200 seconds mark


Performance Improvement

http://ec2-54-234-198-96.compute-1.amazonaws.com/testruns/dbmcrandtest/results/results_2013.05.30_05.20.40/results.html


Questions

aws webcast - accelerating application performance using in-memory caching in aws

Technology

web appstep

express consent of

database serversstep

memory caching

awsomer zaki