confessions of a relational addict
TRANSCRIPT
Confessions of a relational addict
Chandra Patni
Twitter @cpatni
Architecture
http://www.seas.upenn.edu/~zives/03f/cis550/codd.pdf
Volume 13 / Number 6 / June 1970
Promise of ACID
Atomicity
Consistency
Isolation
Durability
READ COMMITTED may block
ANSI/ISO SQL...
READ UNCOMMITTED to allow for non-blocking reads
But allowed dirty reads...
Isolation. Really?
Locking
Locking
Lock Lock
Escalation
Escalation
Read Lock
Read Lock
Write Lock
Write Lock
Table Lock
Table Lock
MVCCFor transactional memory
Readers don’t block readers
Readers don’t block writers
Writers don’t block writers
Writers block writers (serialized)
Living in a bliss
Until...
need redundancy
scale reads
scale writes
horizontally
Solutions...
replication to scale reads
cache through writes
expensive solution for redundancy
mongodb
Distributed database from ground up
replication in a minute (HA)
built-in cache through writes
replicate if you care about your data
sharding (scale out)
Effective Design
Domain objects rarely fit in a table row
join to assemble domain objects
mongo documents as pre-assembled domain objects
in place updates ($set, $push)
New hotness
Multikeys
Map reduce
map = function () { for (var i = 0; i < this.tags.length; i++) { emit(this.tags[i], this.state == "published" ? 1 : 0); } }reduce = function (key, values) { var total = 0; for (var i = 0; i < values.length; i++) { total += values[i]; } return total; }
Capped collections
Geospatial
GridFS
Rubber meets the road
Realtime Dashboard
Realtime Dashboard
Realtime stats for 1 sec, 5 secs, 15 secs
4K updates per second
pywebsocket
IGN Media: Editorial Generated Content
Content in MongodbMaster/slave over filer for instant backups
100 ops
Ruby on Rails backend
MongoMapper
Integrate Solr Search with MongoMapper: http://github.com/tsxn26/customized-mongomapper-search
XML/JSON serialization: http://github.com/rubyorchard/mongomapper
RESTful API
Game Schema DesignRapid Schema Revision
Optimize canonical use-case
IGN Social: User Generated Content
Manish Pandit
Activity Streams
3M documents @10-15 qps, @5 updates/sec
Moving to activity propagation: 4 documents for every user, i.e. a total of 2M documents with arrays of 500 activities each
Move to replica sets on 1.7
Meta+Level Games
Shadelight
Data in MongoDB and Redis
Replicated in real-time to slave servers
Master and slave are in different colo
Ben Myles wrote awesome library called MongoMatic http://mongomatic.com/
Mongomatic
Ruby datamapper for MongoDB
Simple and embraces MongoDB
Shadelight and Mongo Machine (http://mongomachine.com/) use it
We’re hiringhttp://corp.ign.com/http://labs.ign.com/