pycon india, 2011 (work in progress) sunil arora
DESCRIPTION
PyCon India, 2011 (Work in Progress) Sunil Arora. Redis And Python. Raising Hands. How many of you have used Redis before ?. How many of you have got a laptop here ?. Who. Sunil Arora / @_sunil_ Work for ShopSocially ( http://shopsocially.com ) Interests: - PowerPoint PPT PresentationTRANSCRIPT
Redis And Python
PyCon India, 2011
(Work in Progress)Sunil Arora
Raising Hands...
How many of you have used Redis before ?
How many of you have got a laptop here ?
Who
Sunil Arora / @_sunil_ Work for ShopSocially (http://shopsocially.com) Interests:
Backend, Frontend, scaling, internet technologies in general, startups ... all kind of shit
Who
Sunil Arora / @_sunil_ Work for ShopSocially (http://shopsocially.com) Interests:
Backend, Frontend, scaling, internet technologies in general, startups...
Today's talk
What is Redis
How it works and what you can do with it
Real life use-cases
What is Redis ?
Its between lot of stuff, so difficult to categorize it precisely
What is Redis ?
I see Redis definitely more as a flexible tool that as a
solution specialized to solve a specific problem: his mixed soul of cache,
store, and messaging server shows this very
well
-Salvatore Sanfilippo
Picture by herzogbrhttp://www.flickr.com/photos/herzogbr/2274372747/sizes/z/in/photostream/
Brief History of Redis
Released in March 2009 by Salvator Sanfilippo (@antirez)
Open source, BSD licensed
VMWare sponsored its development in March, 2010
A few fundamentals
Written in C (25K LOC) Uses memory as main storage Single Threaded Uses disk for persistence Screamingly fast performance 50K read/write operations per seconds 200K read/write ops per second on a regular
EC2 instance
Installation
$ git clone http://github.com/antirez/redis
$ cd redis
$ make
$ ./src/redis-server
..........
..........
redis-py
The most popular python client library Andy McCurdy ([email protected]) Github: http://github.com/andymccurdy/redis-py easy_install redis OR pip install redis Optional: easy_install hiredis or pip install
hiredis
Lets get started...
$ cd redis
$ ./src/redis-server
.........
>>> from redis import Redis
>>> redis_client = Redis()
>>> redis_client.keys()
>>> help(redis_client)
Redis Keys
Not binary safe. Should not contain space or newline character A few rules about keys:
Too long keys are not a good idea Too short keys is also not a good idea “object-type:id:field” can be a nice idea, i.e.
“user:1001:name”
Operations on Keys
KEYS EXISTS DEL EXPIRE OBJECT PERSIST
RANDOMKEY RENAME TYPE TTL EXPIREAT MOVE
Lets play with keys
>>>redis_client.keys()
>>>redis_client.exists('key')
>>>redis_client.delete('key')
>>>redis_client.type('key')
>>>......
>>>......
Data Structures
Strings Lists Sets Sorted Sets Hashes
Strings
SET GET MSET MGET SETEX
INCR INCRBY DECR DECRBY
Strings – with redis client
>>> redis_client.set('key', 'value')
>>> redis_client.get('key')
>>> redis_client.delete('key')
Fetch multiple keys at once
mget/mset redis_client.mset({'key1': 'val1', 'key2': 'val2'}) redis_client.mget('key1', 'key2',......)
Expiration
Set a value with expire
>>>redis_client.setex('key', 'value', 2) #key to expire in 2 secs
>>>redis_client.expire('key', 2)
>>>redis_client.get('key')
>>>None
Uses
To store transient states in your web application
Uses
Who is online?
Uses
Redis as LRU cache (http://antirez.com/post/redis-as-LRU-cache.html)
Atomic Increments
>>>help(redis_client.incr)
>>>help(redis_client.decr)
>>>
>>> redis_client.incr('counter', 1)
>>>1
>>> redis_client.incr('counter')
>>>2
>>> redis_client.incr('counter')
>>>3
Uses
High Speed counters (views/clicks/votes/likes..)
Uses
API Rate Limiting
Uses
Generating unique IDs
Lists
Ordered list of binarysafe strings Doubly linked list Memory footprint optimized for smaller list O(1) insertion/deletion at both ends
Lists - operations
LPUSH RPUSH LSET LRANGE LPOP BLPOP BRPOP BRPOPLPUSH
LINSERT RPOP RPOPLPUSH LPUSHX RPUSHX
Uses
Web apps are full of lists :)
Uses
Capped List
Uses
Normal QueueReal time message Queue
Uses
Social Activity Streams
Sets
An unordered collection of distinct byte strings
Nothing different from the data type in python
Sets - Operations
SADD SCARD SDIFF SDIFFSTORE SINTER SINTERSTORE SISMEMBER SMEMBERS
SMOVE SPOP SRANDMEMBER SREM SUNION SUNIONSTORE
Uses
Picking random items from a set using SRANDMEMBER
Uses
Primitive to construct filtering mechanism on items
Which of my friend's are online right now?
Uses
Sorted Sets
ZADD ZCARD ZCOUNT ZINCRBY ZINTERSTORE ZRANGE ZRANGEBYSCORE ZRANK
ZREM ZREMRANGEBYRA
NK ZREMRANGEBYSC
ORE ZREVRANGE ZREVRANGEBYSCO
RE ZSCORE ZUNIONSTORE
SORT
SORT KEY
SORT key BY pattern (e.g. object-type:*:age)
SORT key LIMIT 0 10
SORT key GET user:*:username
SORT key BY pattern STORE dstkey
Uses
Generating sorted transient views
Publish/Subscribe
A simple and efficient implementation of publish/subscribe messaging paradigm
Client can subscribe/psubscribe to receive messages on channels (key patterns)
Publish/Subscribe
PSUBSCRIBE PUBLISH PUNSUBSCRIBE SUBSCRIBE UNSUBSCRIBE
Uses
Many to Many message passing
Uses
Web Chat
Questions