why your mongodb needs redis
TRANSCRIPT
Why Your MongoDBNeeds RedisAsk a Redis Expert™ Webinar
September 3rd, 2015
@itamarhaber
A Redis Geek and Cheif Developer AdvocateatHave you signed for newsletter?[1] http://bit.ly/RedisWatch
The Menu: What MongoDB? What Redis?
A document-oriented disk-based
database
If you’re here,you’re probably
already using it
An in-memory and optionally persistable data structuresengine
Redis isBlazing-Fast™
Why broth?“Both projects are about there is something wrong if we use an RDBMS for all the kind of works… in response to the same non-nail problems, these two tools have taken different paths”
[2] MongoDB and Redis: a different interpretation of what's wrong with Relational DBs, June 3rd 2009, @
antirez
● “… an [4] open source (BSD licensed), in-memorydata structure store, used as database, cache and message broker."
● 5 data structures, 180+ commands, in-memory, persistable to disk, atomic operations, Lua scripting, replication, high availibility, clustering and an active & vibrant community
● Nee circa 2009, by [5] antirez(a.k.a Salvatore Sanfilippo)
● Sponsored by Redis Labs
[3] Redis (REmote Dictionary Server)
Why is Redis so Glazing Fast?Is not unlike asking “Why is a Ferrari so fast?”
Answer: Performance dictates the Design
Redis is designed for performance• [6] The Redis Manifesto: We’re against complexity• RAM is fast (
[7] Latency Numbers Every Programmer Should Know)• C is fast ([8] Programming Languages Benchmark)• (mostly) single-threaded event loops are fast• Data structures are optimized for performance• Transparent complexity, time-space tradeoff knobs, …• Shameless Plug:
[9] webinar – How To Achieve 1.5 Million ops/second with Redis
A talk about MongoDB performance
[10] WiredTiger iiBench Results
NOT!I'm hardly an expert, but with MongoDB v3 storage engines and future work this could very well be a moot point
Hors d'oeuvre: La Quiche• You can make an excellent
quiche cache with Redis• Configurable eviction, key expiration and
optional data persistence <- blend and mix them to your taste
• Manage the cache in app; wrap the logic around your primary database driver; or grab something ready from GitHub
Souped up Intelligent Cache
Intelligent cache “understands”
the data it manages,
whereas for the dumb ones data
is opaque
[11] An introduction to Redis data types
A talk about Redis’ Data Types
Primary1. String2. Hash3. List4. Set 5. Zorted Set
Secondary1. Integer2. Float3. String bitmap4. HyperLogLog5. Coming: Geo hash,
Bloom filter
NOT!
A talk about Redis’ 180+ Commands
[12] Redis Command Reference
[13] Red is Beautiful: A Visualization of
Redis Commands
NOT!
Entremets: Counting with RedisBefore moving to the main course, consider the common need for counting things. Arguably, any database can do that, but do you want to load yours with that?
Redis is great for counting stuff and does it reallyfast…
- Main Course -Stop Big Data Indigestion
Before It Starts
You probably haven't seen anything like this before
Volume
Velocity Variety
MongoDB truly excels when is comes to volume and variety of data……but data coming in at extreme velocity posesa digestive challenge forfor any disk-based database
Data ingestion at high velocityMobile, online and IoT appsproduce more and more datawith every day that passes.
Simply storing the data as itcomes in doesn't cut it anymore – real time processing is a must in order distill information from the data as it rushes in.
A talk about more performance
By doing LESSyou can do MORE(with MongoDB)
Put differently, "chew" your data with Redis to prevent data ingestion indigestion
Use case A: Google Analytics• A real time analytics platform provider• Strongly focuses on users' behavior• Primary data storage is MongoDB• Activity is collected immediately or in bulks• Raw data fed to Hadoop for offline crunching• Real time metrics and initial information from
the stream is obtained with Redis
NOT!
The tidal flow
Sessions events
Real time analysis
Offline analysis
Deep dive topic: sessionizing data• Stream of events• A session is a document• Each has 10s-1000s events• Events from different users
arrive in order but interleaved• The result: many small updates
to each session's document• Peak load: 1.1M ops/sec (Q1 2015)
You say potato, I say potatoHash data type:HSET session:1 event:1 dataHSET session:1 event:2 data ...
HINCRBY session:1 seq 1
JSON:{ session: 1, events: [ { id: 1, data: data }, { id: 2, data: data }, ...
Swallowing in Pythonimport redisimport pymongo
r = redis.Redis()session = r.hgetall('session:1')# {'event:1': 'data', 'event:2': 'data', 'seq': '2'}...m = pymongo.MongoClient()db = m.rtasessionid = db.sessions.insert_one(session)
Keeping track of sessions• Sessions end after a logout or a timeout• Logout events are trivial to detect• Timeouts, e.g. 30 minutes of inactivity, are
trickier to manage considering there could be 10,000s of active sessions
• This is where Redis' key expiry and keyspace notifications come in very handy
Once you see it, it can't be unseenUsing Redis as a buffer in front of MongoDB for write-intensive, hot Big Data is a useful pattern that makes it easy to get information in real time as well as distribute the load more efficiently.
Ceci n’est pasune Quiche
Use case B: Waze• An international navigation app/service• Strongly focuses on public transit• 10s of millions of users during peak hours• Primary data storage is MongoDB• Base data is created in advance• Real time updates (traffic, vehicles and
passengers) pour into Redis for scheduling adjustments and notifications
NOT!
Use case C: Tinder• A dating app/service• Strongly focuses on spatially-related groups• Primary data storage is MongoDB• Data includes user profiles & preferences• An influx of positional and preferential
("swipes") events is first munched by Redis
NOT!
Use case D: Clash of Clans• A massive real time game• Strongly focuses on matched team play• 1000s of teams with 100s of members• Primary data storage is MongoDB• Match progress is sieved through Redis for
real time resources status, leaderboards and scoring
NOT!
Use case E: Weather.com• IoT startup• Focuses on environmental monitoring• Pilot: real time fire fighting• Primary data storage is MongoDB• Sensor data (temperature, humidity, …) is
aggregated in Redis, providing warnings and alarms in real time
NOT!
Getting started with Redis• Try it online at [14] http://try.redis.io/• Build it from the source
• [15] Try Redis Labs Enterprise Cluster• Run it in a container
• [16] Connect to it from any language
git clone https://github.com/antirez/rediscd redisgit checkout 3.0.1make; make test; make install
docker run -d --name redis -p 6379:6379 redis
Questions or feedback? Contact me!Itamar Haber
Chief Developer Advocate
📧 [email protected]@itamarhaber
Follow us on Twitter@redislabs