aws black belt techシリーズ amazon elasticache

Amazon ElastiCache

AWS Black Belt Tech Webinar 2015 ()

Agenda Amazon ElastiCache ? ()

ElastiCache MemcachedRedis

(201310)Webinar Multi-AZ Redis : Multi-AZ

3. 4.

" Amazon ElastiCache ?

EBS, S3, Glacier, Storage Gateway

CloudFront

VPC, Route 53, Direct Connect

IAM, CloudTrail,

CloudHSM

Cloud Watch WebManagement

Console

Elastic Beanstalk,Cloud Formation,

OpsWorks

CLI & SDKs

Java, PHP, .NET,

Python, Ruby

AZ Region

EC2, Auto Scaling, Elastic Load Balancing,

RDS, DynamoDB, Redshift, ElastiCache

Elastic MapReduce, Kinesis, Data Pipeline

AppStream, Cloud Search, SWF, SQS, SES, Elastic Transcoder

Workspaces, Zocalo

Mobile Analytics, Cognito, SNS

30AWS

Amazon ElastiCache

EC2RDS

2(memcached, redis)

VPC

Memcached/Redis on EC2

OS memecached/ Redis Management Console/CLI/API

Firewall

AWS

DB

(DB) () APP)

Web+DB

App RDBMS

1. 2. AppDB3. DB4. App

Web+DB

App RDBMS

5. App,DB6. DBRDB

App RDBMS

Web+DB

DB DB

App RDBMS

App

Web+DB

App RDBMS

DB

DB

Point

Web+DB

App DBInsert / Update

RDBMS

Point

Web()

App

App

App

ID

Memcached

key-value 2003Danga Interactive(BSD) Live Journal (YouTube, Wikipedia, mixi, etc. )

KVS

Key-value Telnet

LRU3

Amazon ElastiCache for memcached

1.4.51.4.14 (2015.1.14) memcached Cache ClusterCache Node Cluster Group Congration EndpointCache NodeNode Endpoint (Snapshot)

CacheCluster A CacheCluster B

Configration Endpoint

Configration Endpoint

Node Endpoint

Node Endpoint

Node Endpoint

Node Endpoint

Auto Discovery for memcached

Cache Cluster Auto Discovery(Java, PHP,.Net

Cache ClusterConguration Endpoint Conguration EndpointCache Cluster ( Proxy) Endpoint

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.html

CacheCluster

Configuration Endpoint xxx.cfg.apne1.cache.amazonaws.com

xxx.0001.apne1.cache.amazonaws.com

xxx.0002.apne1.cache.amazonaws.com

App

App

Auto Discovery

Memchached 1.4.14>Cong get cluster Memchached 1.4.14>get AmazonElastiCache:cluster

Memcached Client Library

Auto Discovery

https://code.google.com/p/memcached/wiki/Clients

Auto DiscoveryClient LibrayClient Library

AWS Management ConsolePHPJava.NET

Memcached

PHPJava.NetCC++RubyPythonPerl

CloudWatchmemcached

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/CacheMetrics.Memcached.html

CPUUtilization (CPU) Evictions ( ) SwapUsage CurrConnections http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/CacheMetrics.WhichShouldIMonitor.html

Redis

In-memory Key-Value Store

List, Set, Sorted Set, Hash

Snapshot, Append only File

Replication Pub/Sub Lua scripting

http://redis.io/

ElastiCache for Redis

: 2.6.132.8.6 (2015.1.14) Cluster Group Replication Group S3(RDB)ElastiCache Multi-AZ Snapshot Redis

Lua Scripting Pub/Sub Append Only File

CONFIG, SLAVEOF ()

(Replication)

() Read

Replication Group 5 Replica of Replica

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Replication.html http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/ManagingReplication.html

Availability Zone - a Availability Zone - b

CacheCluster CacheCluster CacheCluster

Replication Group

AZ AZ

Replication

Availability Zone - a Availability Zone - b

SET

GET

SET

GET

App App

endpoint

unlink

SYNC

DNS change

RDB RedisElastiCache

RedisRDBS3 S3RDB

RDB S3ElastiCache RDB()

Redis

Redis on EC2

S3

RDB copy

ElastiCache for Redis

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/ManagingCacheClusters.html#ManagingCacheClusters.SeedingRedis

S3 S3(: mybacket/path/data.rdb)

slaveof Redis

EC2 S3

RDB copy

EC2Redisslave EC2Redis slaveRDBS3 S3

:

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/ManagingCacheClusters.html#ManagingCacheClusters.RedisSnapshots

http://redis.io/topics/persistence

restore

Append-Only Files(AOF)

AOF Redis()AOF reboot

reboot

o appendonly yes

AOF

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/CacheParameterGroups.Redis.html#CacheParameterGroups.Redis.AOF

CloudWatchRedis

CPUUtilization (CPU) Evictions ( ) CurrConnections Replica Lag ( )

CPU Redis 1 cache.m3.xlarge(4) 25% (100% / 4) Alert http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/

CacheMetrics.WhichShouldIMonitor.html

" Webiner(201310)

2014 3 - 2014 3 - Redis 2.8.6 2014 4 - Redis 2014 7 - M3R3 2014 7 - AZ 2014 9 - T2 (memcached) 201411 - RedisMulti-AZ/

cache.t2.micro

cache.t2.small

cache.t2.medium

cache.m3.medium

cache.m3.large

cache.m3.xlarge

cache.m3.2xlarge

cache.r3.large

cache.r3.xlarge

cache.r3.2xlarge

cache.r3.4xlarge

cache.r3.8xlarge

Redis 2.8.6Redis 2.8.6 repl-backlog-size min-slaves-max-lag

T2M3R3 EC2RDS Redis

Redis AOFT2

cache.m1.small

cache.m1.medium

cache.m1.large

cache.m1.xlarge

cache.m2.xlarge

cache.m2.2xlarge

cache.m2.4xlarge

cache.t1.micro

CPU

cache.c1.xlarge

AZ AZ

Availability Zone A Availability Zone B

CacheCluster

Memcached Redis

NodeAvailability Zone(s) Specify ZonesZone

Read ReplicaAZPrimaryRead ReplicaAZ

ElastiCache Redis ElastiCache Redis Snapshot SnapshotRedisBGSAVERead Replica

Redis Master

Redis Read

Redis Read

Snapshot

Master

Read ReplicaCluster

Replication Group A

MasterSnapshot

Snapshot

Replication Group B

Redis Master

Redis Read

Replication Group

ElastiCache Redis

Cache ClustersSnapshotClustersCreate

Modify

Yes

Read Replica3511

Snapshot

ElastiCache Redis Multi-AZ ElastiCache Redis Master Node Replication GroupRead

ReplicaMaster

Redis Master

Redis Read

Redis Read

Redis Read

Redis Read

Redis Read

Availability A

Availability B

Replication Group =Cache Subnet Group

Redis Read

Redis Read

Redis Read

Redis Master

Redis Read

Availability A

Availability B

Replication Group =Cache Subnet Group

GroupAZReadReplicaMaster

ElastiCache Redis Multi-AZ

Replication Multi-AZ

AZVPC Subnet Cache Subnet Group

Primary AZRead Replica

1. SNS2. 3. 4. DNS

1. SNS2. 3. DNS

SNS /retry,

DB

SNS

memcached / redis

redis (atomic counter) (bitmaps) (lists) (pub/sub) , (sorted sets)

Client 1

subscribe

deliver msg

Client 2

subscribe

deliver msg

Channels / Patterns

MySQL

Client

20151ElastiCache

20151 VPCVPC

- Endpoint Public Facing

Action

Action

FlankfultRegion

Web9:1

DB

SPOF

DB ()

App RDBMS

App

(Consistent Hashing) Consistent Hashing

AppConsistent Hashing

CH

APP

PHPlibmemcacheddistribution

Twemproxy TwitterMemcached/Redisproxy (consistent hashing)

Consistent hashing - Wikipedia, the free encyclopedia

http://en.wikipedia.org/wiki/Consistent_hashing memcached4memcachedgihyo.jp

http://gihyo.jp/dev/feature/01/memcached/0004?page=3 Partitioning: how to split data among multiple Redis instances.

http://redis.io/topics/partitioning

Twemproxy

TwemproxySPOFTwemproxytwemproxy

App +

Twemproxy

App +

Twemproxy

memcached

RG1

App +

Twemproxy

App +

Twemproxy

RG2

Redis

MemcachedRedis

http://aws.amazon.com/jp/elasticache/pricing/ http://aws.amazon.com/jp/elasticache/reserved-cache-nodes/

(

70%) EC2

2015114

Standard Cache Nodes - Current Generation

cache.t2.micro $0.026

cache.t2.small $0.052

cache.t2.medium $0.104

cache.m3.medium $0.120

cache.m3.large $0.240

cache.m3.xlarge $0.485

cache.m3.2xlarge $0.965

Memory Optimized Cache Nodes - Current Generation

cache.r3.large $0.273

cache.r3.xlarge $0.546

cache.r3.2xlarge $1.092



Redis 1Snapshot 2Snapshot 0.085 USD/GB

AZ ElastiCache 2ElastiCacheAZ

0.01 USD/GB

Amazon ElastiCache Documenthttp://aws.amazon.com/jp/documentation/elasticache/

Amazon ElastiCache FAQ

http://aws.amazon.com/jp/elasticache/faqs/

Amazon ElastiCache Pricinghttp://aws.amazon.com/jp/elasticache/pricing/

Memcached http://memcached.org/ Redis http://redis.io/

Webinar

AWS http://aws.amazon.com/jp/aws-jp-introduction/

Twitter/FacebookAWS

@awscloud_jp

http://on.fb.me/1vR8yWm

aws black belt techシリーズ amazon elasticache

Technology