Download - Cassandra vs. Redis
![Page 1: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/1.jpg)
Cassandra vs. Redis
NoSQL Frankfurt2010-09-28
![Page 2: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/2.jpg)
Who?
• Tim Lossen / @tlossen
• Ruby developer
• Berlin
![Page 3: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/3.jpg)
![Page 4: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/4.jpg)
Challenge
![Page 5: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/5.jpg)
Requirements
• backend for facebook game
![Page 6: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/6.jpg)
Requirements
• backend for facebook game
• 1 mio. daily users
• 10 mio. total users
• 100 KB data per user
![Page 7: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/7.jpg)
Requirements
• peak traffic:
• 10.000 concurrent users
• 200.000 requests per minute
![Page 8: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/8.jpg)
Requirements
• peak traffic:
• 10.000 concurrent users
• 200.000 requests per minute
• write-heavy workload
![Page 9: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/9.jpg)
Sneak Preview
![Page 10: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/10.jpg)
![Page 11: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/11.jpg)
![Page 12: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/12.jpg)
![Page 13: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/13.jpg)
Cassandra
![Page 14: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/14.jpg)
Overview
• written in Java
• 55.000 lines of code
![Page 15: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/15.jpg)
Overview
• written in Java
• 55.000 lines of code
• Thrift API
• clients for Java, Python, Ruby
![Page 16: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/16.jpg)
History
• originally developed by facebook
• in production for “inbox search”
![Page 17: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/17.jpg)
History
• originally developed by facebook
• in production for “inbox search”
• later open sourced
• top-level apache project
![Page 18: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/18.jpg)
Features
• high availability
• no single point of failure
![Page 19: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/19.jpg)
Features
• high availability
• no single point of failure
• incremental scalability
![Page 20: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/20.jpg)
Features
• high availability
• no single point of failure
• incremental scalability
• eventual consistency
![Page 21: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/21.jpg)
2006
![Page 22: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/22.jpg)
2007
![Page 23: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/23.jpg)
Architecture
• Dynamo-like ring
• partitioning + replication
• all nodes are equal
![Page 24: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/24.jpg)
Architecture
• Dynamo-like ring
• partitioning + replication
• all nodes are equal
• Bigtable data model
• column families
![Page 25: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/25.jpg)
![Page 26: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/26.jpg)
Cluster structure
![Page 27: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/27.jpg)
“Cassandra aims to run on top of an in"astructure
of hundreds of nodes.”
![Page 28: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/28.jpg)
![Page 29: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/29.jpg)
Redis
![Page 30: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/30.jpg)
Overview
• written in C
• 13.000 lines of code
![Page 31: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/31.jpg)
Overview
• written in C
• 13.000 lines of code
• socket API
• redis-cli• client libs for all major languages
![Page 32: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/32.jpg)
Features
• high (write) throughput
• 50 - 100 K ops / second
![Page 33: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/33.jpg)
Features
• high (write) throughput
• 50 - 100 K ops / second
• interesting data structures
• lists, hashes, (sorted) sets
• atomic operations
![Page 34: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/34.jpg)
Features
• high (write) throughput
• 50 - 100 K ops / second
• interesting data structures
• lists, hashes, (sorted) sets
• atomic operations
• full consistency
![Page 35: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/35.jpg)
Architecture
• single instance (not clustered)
• master-slave replication
![Page 36: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/36.jpg)
Architecture
• single instance (not clustered)
• master-slave replication
• in-memory database
• append-only log on disk
• virtual memory
![Page 37: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/37.jpg)
“Memory is the new disk,disk is the new tape.”
⎯ Jim Gray
![Page 38: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/38.jpg)
![Page 39: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/39.jpg)
Solution
![Page 40: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/40.jpg)
Key Decisions
• keep operations simple
![Page 41: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/41.jpg)
Key Decisions
• keep operations simple
• use as few machines as possible
• ideally, only one
![Page 42: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/42.jpg)
![Page 43: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/43.jpg)
Architecture
• single Redis master
• with virtual memory
• handles all reads / writes
![Page 44: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/44.jpg)
Architecture
• single Redis master
• with virtual memory
• handles all reads / writes
• single Redis slave
• as hot standby (for failover)
![Page 45: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/45.jpg)
Throughput
• redis-benchmark
• 60 K ops / s = 3.6 mio ops / m
![Page 46: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/46.jpg)
Throughput
• redis-benchmark
• 60 K ops / s = 3.6 mio ops / m
• monitoring (rpm, scout)
• ca. 10 ops per request
![Page 47: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/47.jpg)
Throughput
• redis-benchmark
• 60 K ops / s = 3.6 mio ops / m
• monitoring (rpm, scout)
• ca. 10 ops per request
• 200 K rpm = 2.0 mio ops / m ✔
![Page 48: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/48.jpg)
Capacity 1
• 100 KB / user (on disk)
• 10.000 concurrent users (peak)
![Page 49: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/49.jpg)
Capacity 1
• 100 KB / user (on disk)
• 10.000 concurrent users (peak)
• 1 GB memory
• (plus Redis overhead) ✔
![Page 50: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/50.jpg)
Capacity 2
• Redis keeps all keys in memory
• 10 mio. total users
• 20 GB / 100 mio. integer keys
![Page 51: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/51.jpg)
Capacity 2
• Redis keeps all keys in memory
• 10 mio. total users
• 20 GB / 100 mio. integer keys
• 2 GB memory for keys ✔
![Page 52: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/52.jpg)
Data model
• one Redis hash per user
• key: facebook id
![Page 53: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/53.jpg)
Data model
• one Redis hash per user
• key: facebook id
• store data as serialized JSON
• booleans, strings, numbers, timestamps ...
![Page 54: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/54.jpg)
Advantages
• efficient to swap user data in / out
![Page 55: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/55.jpg)
Advantages
• efficient to swap user data in / out
• turns Redis into “document db”
• atomic ops on parts
![Page 56: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/56.jpg)
Advantages
• efficient to swap user data in / out
• turns Redis into “document db”
• atomic ops on parts
• easy to dump / restore user data
![Page 57: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/57.jpg)
Lessons
![Page 58: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/58.jpg)
“You are not facebook.” ⎯ me
![Page 59: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/59.jpg)
Advice
• use the right tool for the job
![Page 60: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/60.jpg)
Advice
• use the right tool for the job
• avoid scaling out / sharding, if possible
• do the numbers!
![Page 61: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/61.jpg)
Advice
• use the right tool for the job
• avoid scaling out / sharding, if possible
• do the numbers!
• keep it simple
![Page 62: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/62.jpg)
Q & A
![Page 63: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/63.jpg)
Links
•cassandra.apache.org
•redis.io
•tim.lossen.de
![Page 64: Cassandra vs. Redis](https://reader037.vdocuments.net/reader037/viewer/2022103113/5549345db4c905054d8b458e/html5/thumbnails/64.jpg)