this is redis - feature and usecase

This is REDIS- Feature and USECASE

About Me

Kris jeong(정경석 )

Author a Book of REDIS ‘This is REDIS’ in korean

Software development Over 14 years. Web service dev.

Messaging server dev.

Recommendation dev.

Interested about NoSQLhttp://www.aladin.co.kr/shop/wproduct.aspx?ISBN=8968480591

What is

REDIS is In-Memory Database

Variety data types

Support persistence

Key value based NoSQL

IMDB(In-Memory DB)

Query Engine Query Engine

Storage Man-ager

Memory(Cache)

Disk Disk

Memory

In-Memory data storeDisk based data store

Speed

Durability Disk is optional

Redis is cache server?

Not a cache server.

But, It can be.

Turn off Persistence.

Running platform

https://github.com/MSOpenTech/redis

Install and test

$ wget http://download.redis.io/redis-stable.tar.gz$ tar xvf redis-stable.tar.gz$ cd redis-stable$ make……done.

$ make test

※ CentOS 6.4 64bit

Feature

Super fast (TPS)

$ ./src/redis-benchmark --csv -d 1024"PING_INLINE","34602.07""PING_BULK","35971.22""SET","35211.27""GET","31347.96"…"LPUSH (needed to benchmark LRANGE)","32467.53""LRANGE_100 (first 100 elements)","4714.76""LRANGE_300 (first 300 elements)","1605.91""LRANGE_500 (first 450 elements)","1030.93""LRANGE_600 (first 600 elements)","735.13""MSET (10 keys)","22471.91"

※ CentOS 6.4 64bit, Core i5 3337U@1.8Gz

Feature - expire

Duration cache expire user:212:cart 259200

It will be expired 3 days later.

Specified time cache expireat user:212:item:sward 1388534400

It will be expired 2014/01/01

seconds

Unix timestamp

Support data types

String

user:1:cart_info

{ "product": { "id": "2951", "name": "testing 01", "options": { "color": "red" }, "quantity": 4 }}

※ Up to 512MB

key value

Support data types (cont.)

Hash

item:1:info

name bastard sword

min dam-age

13

max dam-age

20

durability 32/50

price 3500

durability 31/50

add a new value

field valueKey

Support data types (cont.)

Hash

item:1:info

name bastard sword

min dam-age

13

max dam-age

20

durability 32/50

price 3500

name bastard sword

min dam-age

13

max dam-age

20

durability 31/50

price 3500

field valueKey

Support data types (cont.)

List

user:1:message

how r you.hi~

bye.

add a new value

Key elements

Support data types (cont.)

List

user:1:message

how r you.hi~ bye.

Key elements

Support data types (cont.)

Set

user:1:friend

kris

mike

frank

martin

tom

chris

add a new value

kris

mike

frank

martin

tom

chris

Support data types (cont.)

Set

user:1:friend

kris

mike

frank

martin

tom

1 kris

91 mike

200 frank

250 martin

251 tom

Support data types (cont.)

Sorted Set

user:ranking

220 chris

add a new value

score valueKey

1 kris

91 mike

200 frank

250 martin

251 tom

1 kris

91 mike

200 frank

220 chris

250 martin

251 tom

Support data types (cont.)

Sorted Set

user:ranking

score valueKey

Feature – RDB(Snapshot)

Redis data

Redis instance

Dump file(Redis-.rdb)

When snapshot event raised

Data dumpto disk

Redis data size vs Physical memory size ratio

vm.overcommit_memory setting

Disk I/O rate for swap

Check point of Redis snapshot

Feature – AOF(Append Only File)

Redis data

Redis instance

appendon-ly.aof

Command logging

Redis client

Data change command

Disk space

Restart time is slower than RDB

Check point of Redis aof

Replication – Single slave

Redis cluster

Slave NodeMaster Node

Redis client

set key1 ‘test value’

set key1 ‘test value’

Replication

key1 test value key1 test value

Replication – Multiple slave

Master Node

Redis cluster

Redis client

Write

Replication

Read

Slave Node 2

Slave Node 1

Slave Node 3

Replication – Multiple slave (cont.)

Master Node

Redis cluster

Slave Node 2

Redis client

Write

Slave Node 1

Slave Node 3 Slave Node 4

1st Replication

2nd Replication

Redis client

Read

Sharding – range

Shard 1

Master Node

User Key 1 ~ 100

User Key 1~50 User Key 51~100

Shard 2

Master Node

Slave Node Slave NodeSlave Node Slave Node

● Data range based data split

Sharding - vertical

Shard 1

Master Node

User info Content

Shard 2

Master Node

Slave Node Slave NodeSlave Node Slave Node

● Schema based data split

Key % Number of Shard(Key % 2)

Sharding - consistent hashing

Shard 1

Master Node

User Key 1 ~ 100

Shard 2

Master Node

Slave Node Slave NodeSlave Node Slave Node

● Key based data split

Sharding Calc

…

Sharding – Redis client

Redis client

Shard 1 Shard N

Read/Write Read/Write

Redis does not support server-side sharding

Most Redis client support hash based sharding

Re sharding issue

Check point of Redis Sharding

Useage

Daily logged in user count

Weekly logged in user count

List of Logged-in account at least once of week

PV/UV Calculation 1Billion user

Top 10 of news by click count

News list by popular by comment count

Most popular News list - Real time

Score based Top 10 ranker.

Recent user list

Real time gamers ranking

Demo