1.

• Practical MongoDB and Drupal

• Doug Green (douggreen)
• Karoly Negyesi (chx)

2. #drupalchi, #drupalchimongo for questions

3. http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal 4.

Design for Speed and Scalability

5.

Examiner.com

I work at Examiner.com, aQuantified top 100 North American website, probably the highest traffic Drupal 7 website in the world.Most minutes on most days we delivercustomized non-cached content to over a thousand people, every page built to the individual based on location or preference.In the time it takes you to count the number of people near you, we'll probably touch more people. well maybe not in this room today,but still you'll have to count fast! 6.

Examiner.com

... our daily traffic has increased and our monthly availability has gone from 99% uptime to 99.97% uptime hours of downtime are now measured in minutes.Just today we hit another record day, breaking the record from just yesterday, breaking the record from last week.That's all significant! - Doug Green, Drupal Watchdog, March 2011 7. Design for Speed and Scalability

Understand databases, queries, and indexes, and know the pros and cons and tradeoffs

Cache, memcache entities, Drupal render cache, mongodb block cache

8. Never call a web page from your web page 9. CSS and JS aggregation 10. CDN's 11.

Change your frame

RDBMS traditional SQL solutions NoSQL solutions but let's change the name ... Document Object DB NO sql 12.

What do we call this?

Document-oriented Database ? DODB ? Distributed Key-Value Pair Graph Database ? We really can't agree, I think NoSQL is the wrong Frame, let's just call this a Document-oriented Database Management System or DODBMS 13.

RDBMS verses DODBMS

Bleeding Edge?

Leading Edge?

or

14.

RDBMS verses DODBMS

or

Disk is cheap !?

15.

RDBMS verses DODBMS

SELECT * FROM foo;

db.foo.find();

A query is just a query in a different language

Parle vous BSON?

16.

MongoDB is DODBMS

Open Source?Community? Support?

17.

DODBMS built for 2010 not 1980

3 computer years?

25 computer years?

18.

DODBMS+MongoDB Summary

Pros

Cons

• Fast, document structure is perfect for Drupal entities

19. Takes best advantage of 2010 Operating Systems that manage memory well

20. Takes best advantage of 2010 Hardware the recognizes disk and memory is cheap 21. Open source code can be read and modified (40-64 index example)

• Bleeding edge, software is new and not as well battle tested

22. Lack of GUI based tools

23. Small user community and new language that must be learned 24. 10gen support is excellent, free and paid (but as with all companies they are seeking ways of funding their efforts, so this is changing) 25.

MySQL to MongoDB primer

SELECT * FROM foo WHERE bar = 1;

db.foo.find({bar:1});

This is the session on Practical MongoDB so let's do a few more...

26.

MySQL to MongoDB primer

SELECT some FROM foo WHERE bar = 1;

db.foo.find({bar:1}, {some:1});

27.

MySQL to MongoDB primer

INSERT INTO foo (bar) VALUES (1);

db.foo.insert({bar:1});

Inserts are simple

28.

MySQL to MongoDB primer

CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1);

db.foo.insert({bar:1});

Inserts are simple, but to be fair...

29.

MySQL to MongoDB primer

CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1);

db.foo.insert({bar:1}); db.foo.insert({bar:1});

Inserts are simple, but be more fair...

30.

MySQL to MongoDB primer

db.foo.insert({bar:1}); db.foo.find(); { _id: ObjectId(4d...), bar : 1 }

Inserts are simple, but let's be accurate, mongo auto-creates unique keys

31.

MySQL to MongoDB primer

db.foo.insert({bar:1, _id:123456}); db.foo.find(); { id: 123456 bar : 1 }

Or you can add your own unique key

32.

MySQL to MongoDB primer

INSERT foo set some= 1 WHERE bar = 2;

db.foo.update({bar:2}, {some:2}); db.foo.update({bar:2}, {$set:{some:2}});

Updates aren't that hard either

or

33.

MySQL to MongoDB primer

db.foo.update({_id:2}, {$set:{bar:1}},true);

But then there's upserts!

34.

Drupal 7 Enhancements

DBTNG

Field API

But, we use Drupal, not the database command prompt, so...

chx 35.

Drupal 7 Enhancements

DBTNG

Field API

FieldAPI is perfect for MongoDB!

well, not yet

36.

Drupal Modules for Mongo

• Fields - no table joins

37. Blocks conditions in DB instead of code

38. Queues mostly to avoid another daemon and it's fast enough 39. Watchdog capped collections, never run out of space, counts and groups by id 40. Cache not used on Examiner.com 41. Session perfect fit for D7

http://drupal.org/project/mongodb

42.

MongoDB Watchdog

t() replaceable arguments i.e., page not found: %q

43.

MongoDB Watchdog Details

Arguments filled in on details

44.

Drupal Field API

$query = new EntityFieldQuery; $results ->entityCondition('entity_type', 'node') ->propertyCondition('uid', (int) $account->uid) ->fieldCondition('bar', 'value', 1) ->propertyCondition('status', 1) ->execute(); $nodes = empty($results['node'])? array(): node_load_multiple(array_keys($result['node']));

45.

Php http://php.net/mongo

$query = array( 'uid' => (int) $account->uid, 'bar.value' => 1, 'status' => 1 ); $fields = array( '_id' => TRUE, ); $results = mongodb_collection('fields_current', 'node') ->find($query, $fields); foreach ($results as $result) { }

46.

Indexes Hints

Indexes Hints

Indexes Hints

Always remember the tree

You can only traverseone branch at a time

47.

Indexes Hints

Indexes Hints

Indexes Hints

1. The sort column must be the last column used in the index.

find({a:1}).sort({a:1}); find({a:1}).sort({b:1}) find({a:1,b:2}).sort({c:1});

find({a:1}).sort({c:1});

Good

Bad

Assume an index on a, b, c on all examples to follow

48.

Indexes Hints

Indexes Hints

Indexes Hints

2. The range query must also be the last column in an index, this is an corollary of 1 above.

find({a:1, b:{$gt:2}); find({a:{$gt:1, $lt:10}}) find({a:{$gt:1}).sort({a:1});

find({a:{$gt:1}, b:2});

Good

Bad

49.

Indexes Hints

Indexes Hints

Indexes Hints

3. Only use a range query or a sort on one column.

find(a=1,b=2).sort(c) find(a=1,b>2) find(a=1,b>2,b2).sort(b)

find(a>1,b>2) find(a=1,b>2).sort(c) find(a=1,b in [1, 2]).sort(c)

Good

Bad

Note, I'm switching to psuedo code shorthand, not BSON syntax

50.

Indexes Hints

Indexes Hints

Indexes Hints

4. Conserve indexes by re-ordering columns used in straight = queries If you have the following two queries, then you only need a single index a, b, d, c find(a=1, b=1, d=1) find(a=1, b=1, c=1, d=1)

51.

Summary

• Examiner.com is mostly happy with MongoDB solution.

52. We miss relational nature of MySQL with a GUI query reporting tool, all reports have to be programmed.

53. Site uptime is much higher than before, but still have some instability related to an immature product 54. Drupal 7, Field API, DBTNG, MongoDB, and Community are awesome 55.

Future

• DBTNG, will require hooks for modules to provide translations of relational SQL to Document object store, we could provide this for core, but each contrib module would then also need to provide this not ideal, but it is a solution

56. I think MongoDB is a perfect fit for search, and I'm the right guy to do it, but every time I mention this, people say, but we have SOLR, why would you do this?

57.

What did you think?

Locate this session on the DCC website:

http://chicago2011.drupal.org/sessions

http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal

Click the Take the Survey link.

Thanks!