mongodb, development and you
DESCRIPTION
My talk at MongoTorino about how the use of MongoDB empowers your development teams, as well as tips and tricks to get around common pitfalls (and not so common challenges).TRANSCRIPT
![Page 1: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/1.jpg)
MongoTorino 2013
MongoDB, Development and You
Mitch Pirtle!CTO Sounday Music!www.soundaymusic.com
![Page 2: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/2.jpg)
![Page 3: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/3.jpg)
45 MINUTES?! Let’s get this thing going then.
![Page 4: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/4.jpg)
Speaker Intro
![Page 5: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/5.jpg)
About Me❖ American, living in Turin!❖ CTO at Sounday Music!❖ Mongo Master!❖ Joomla! Founder!❖ Contributor to many FOSS!❖ Family man!❖ Coach (Giaguari Torino)!❖ Musician!❖ Longboarder
![Page 6: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/6.jpg)
Sleeper Check!
![Page 7: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/7.jpg)
Sounday Intro
![Page 8: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/8.jpg)
Sounday in 2013❖ Growing community of music
professionals!
❖ Integrated with many social media platforms!
❖ Very large binary files (audio)!
❖ 100% UGC!
❖ e.commerce B2C, C2C!
❖ Based in Torino!
❖ Engineering based in Cagliari
![Page 9: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/9.jpg)
The Team
![Page 10: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/10.jpg)
![Page 11: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/11.jpg)
Ok, not really.
![Page 12: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/12.jpg)
![Page 13: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/13.jpg)
The Challenge
![Page 14: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/14.jpg)
The Challenge❖ “We’re gunning for a grant”!
❖ (no money yet)!
❖ “It needs to be live soon” !
❖ (zero time)!
❖ “We’re disrupting the industry”!
❖ (no requirements)!
❖ “Wanna move to Cagliari?”
![Page 15: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/15.jpg)
![Page 16: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/16.jpg)
Ok, it’s not really that bad.
![Page 17: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/17.jpg)
Well maybe.
![Page 18: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/18.jpg)
Classic Startup Challenges
❖ Limited budget!
❖ Limited staff!
❖ Limited time!
❖ Zero chance for best practices during development
![Page 19: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/19.jpg)
How I felt when accepting the challenge
![Page 20: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/20.jpg)
How it actually looked
![Page 21: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/21.jpg)
The Solution
![Page 22: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/22.jpg)
The Solution
❖ Single, RESTful base platform (web, mobile, partners)!
❖ Document database!
❖ Store media in database for simplicity, portability!
❖ Ability to break out separate services when scale needed!
❖ Tethered cloud for initial deployment
![Page 23: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/23.jpg)
Solution: The application
Lithium
• Super lightweight PHP framework!• RAD!• MVC!• Promiscuously opinionated!• Post-relational!• Fully leverages PHP 5.3+!• Cherry-picker methodology (mix of
functional, object, aspect, and more)
![Page 24: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/24.jpg)
Solution: The database
MongoDB
• Document database!• GridFS for large binary files!• Scales vertically!• Scales horizontally!• Reduces male pattern baldness
![Page 25: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/25.jpg)
Solution: The approach
Agile, Xtreme
• Quick, iterative steps!• Constant refactoring!• Captured features during
implementation!• Pair programming!• Local, development and production
environments!• Development and production
environments mirrored!• Dedication to DRY
![Page 26: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/26.jpg)
Sleeper Check!
![Page 27: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/27.jpg)
The Good
![Page 28: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/28.jpg)
![Page 29: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/29.jpg)
Lithium saved our bacon
❖ Extremely quick to get bootstrapped and running a base platform!
❖ Many libraries available for specific needs!
❖ Natively talks to non-relational databases!
❖ Intelligent code layout, encourages best practices!
❖ Adapters, filters, this stuff is the shizzle
![Page 30: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/30.jpg)
MongoDB saved our bacon, with eggs
❖ Simple document structure made for extremely simple models!
❖ Better matched with the objects used in our apps!
❖ GridFS made storing media dead simple!
❖ Being schemaless allowed us to iterate rapidly!
❖ After launch, we completely forgot about MongoDB from an operational standpoint
![Page 31: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/31.jpg)
The Bad
![Page 32: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/32.jpg)
![Page 33: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/33.jpg)
The Bad
❖ Hey remember how easy it is to change your document schemas?!
❖ Loosely-typed language and database makes for fun!!
❖ Hope you love the command line. (limited tools)
![Page 34: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/34.jpg)
And lest we not forget…
![Page 35: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/35.jpg)
The Ugly
![Page 36: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/36.jpg)
![Page 37: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/37.jpg)
The Ugly
❖ What do you mean, ‘no need for constraints’?!
❖ Hey this GridFS database is freaking ginormous.!
❖ I’ve never seen that happen before.!
❖ What, there’s no more SQL? Then who’s to blame when something breaks?
![Page 38: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/38.jpg)
What We Learned
❖ Our code was significantly lighter!
❖ This slim codebase is extremely fast!
❖ Smaller codebase == fewer bugs!
❖ GridFS combined with metadata is really powerful!
❖ Atomic documents + working memory = TO INFINITY AND BEYOND!
![Page 39: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/39.jpg)
![Page 40: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/40.jpg)
What Else We Learned
❖ Shift responsibilities from tools to developers!
❖ Testing and documentation are NOT OPTIONAL!
❖ Define models, even when iterating rapidly in a schemaless environment!
❖ Consider backup and operational requirements before you get started
![Page 41: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/41.jpg)
Example Time
![Page 42: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/42.jpg)
User schema, when we started
class Users extends Model { ! //protected $_schema = array() !}
![Page 43: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/43.jpg)
User schema, today protected $_schema = array( '_id' => array('type' => 'id'), 'name' => array('type' => 'string'), 'username' => array('type' => 'string'), 'email' => array('type' => 'string'), 'password' => array('type' => 'string'), 'salt' => array('type' => 'string'), 'facebook_id' => array('type' => 'string'), 'date_created' => array('type' => 'date'), 'registration_source' => array('type' => 'string'), 'active' => array('type' => 'boolean', 'default' => false), // to make sure the account is live 'status' => array('type' => 'string'), 'mvadmin' => array('type' => 'boolean', 'default' => false), 'pufadmin' => array('type' => 'boolean', 'default' => false), 'resetadmin' => array('type' => 'boolean', 'default' => false), 'profile' => array('type' => 'string'), 'avatar' => array('type' => 'string'), 'enabled' => array('type' => 'string'), 'address' => array('type' => 'array'), 'vat' => array('type' => 'string'), 'url' => array('type' => 'string'), 'services' => array('type' => 'array'), 'likes' => array('type' => 'array'), // Polymorphic associations 'moderated' => array('type' => 'boolean'), 'follows' => array('type' => 'array'), 'last_login' => array('type' => 'date'), 'edit_time' => array('type' => 'date'), 'last_ip' => array('type' => 'string'), 'login_counter' => array('type' => 'integer'), 'is_legacy' => array('type' => 'boolean'), 'legacy_id' => array('type' => 'integer'), 'partners' => array('type' => 'array'), );
![Page 44: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/44.jpg)
Database structureactivities_log!admin_bmu_calendars!admin_landing_pages!admin_notifications!artists!banlists!blacklists!blogs!communications!db_logs!engineers!events!fs.chunks!fs.files!galleries!job_applications!job_songs!jobs!
labels!media!messages!news!payments!playlists!sequence!services!sounday_contest_songs!sounday_contest_users!sounday_contest_videos!studios!system.indexes!temp_lookups!tokens!users!venues!videos
![Page 45: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/45.jpg)
One last thing
![Page 46: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/46.jpg)
Behold, the mighty firehose
![Page 47: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/47.jpg)
![Page 48: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/48.jpg)
Whups, not that one
![Page 49: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/49.jpg)
rs0:PRIMARY> db.firehose.stats() { "ns" : "sounday_analytics_prod.firehose", "count" : 3146600, "size" : 10086355648, "avgObjSize" : 3205.4775465581897, "storageSize" : 10830245856, "numExtents" : 26, "nindexes" : 1, "lastExtentSize" : 2146426864, "paddingFactor" : 1, "systemFlags" : 1, "userFlags" : 0, "totalIndexSize" : 113204896, "indexSizes" : { "_id_" : 113204896 }, "ok" : 1 }
![Page 50: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/50.jpg)
Yeah, I said firehose. Got a problem wit dat?
Console time
If you wanted to see this part of the talk, you should have attended the session.!!! ! ! ! :-)
![Page 51: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/51.jpg)
Sleeper Check!
![Page 52: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/52.jpg)
Stump the Speaker
![Page 53: MongoDB, Development and You](https://reader036.vdocuments.net/reader036/viewer/2022070301/5453d146af795907578b50c7/html5/thumbnails/53.jpg)
Mitch Pirtle [email protected] http://about.me/mitchitized http://twitter.com/mitchitized http://github.com/spacemonkey
Thank You!