non-relational databases
DESCRIPTION
San Francisco Perl Mongers presentation, September 3rd, 2009.TRANSCRIPT
![Page 1: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/1.jpg)
Non-Relational Databases
San Francisco Perl MongersKristina ChodorowSeptember 3, 2009
![Page 2: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/2.jpg)
Who am I?
• Software Engineer at 10gen• Perl, PHP, Java drivers
![Page 3: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/3.jpg)
![Page 4: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/4.jpg)
Scaling
![Page 5: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/5.jpg)
Scaling
![Page 6: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/6.jpg)
Technique #1: literally scale
![Page 7: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/7.jpg)
Technique #1: literally scale
$3,200 $4,000,000x 1250 =
(Courtesy of Ask Bjorn Hansen)
![Page 8: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/8.jpg)
Technique #2: shard
now-yesterday yesterday-the day before
the day before-the day before that
![Page 9: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/9.jpg)
Technique #3: master-slave replication
R/W?W
W
W
R
R
![Page 10: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/10.jpg)
Cassandra
• Designed by Facebook• Distributed - eventually
consistent• Column-oriented
![Page 11: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/11.jpg)
Cassandra
• Writes ALWAYS work• Two types of reads: – high performance– guaranteed correct
• R/W atomic within column family
![Page 12: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/12.jpg)
Cassandra
Row-oriented: joe, 34, karen, 25, bob, 50
Column-oriented:joe, karen, bob, 34, 25, 50
![Page 13: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/13.jpg)
Project Voldemort
• Key/value store• Automatic replication/partitioning• 75 Github forks
“It is basically a just a big, distributed, persistent, fault-tolerant hash table.”
![Page 14: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/14.jpg)
Tokyo Cabinet
• Another key/value store• Blazing fast… theoretically• Lua for server-side scripting
![Page 15: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/15.jpg)
Tokyo Cabinet
• Key/value pairs• Array store• Index-able Hash
{ name : “Chris” age : 27, DOB : “January 1”}
![Page 16: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/16.jpg)
CouchDB
• Master-Master Replication• Map/Reduce• REST API
![Page 17: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/17.jpg)
CouchDB Futon
![Page 18: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/18.jpg)
MongoDB
![Page 19: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/19.jpg)
MongoDB
• Ease of use• Scalable• Dynamic queries - similar “feel” to
SQL• Speed of key/value stores (almost)• Power of RDBMSs (almost)
![Page 20: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/20.jpg)
The Perl Driver
Available at Github: www.github.com/mongodb/mongo-perl-driver/
Install the Perl driver $ perl Makefile.PL && sudo make install
Available on CPAN (but old):http://search.cpan.org/~flora/MongoDB-0.01/
![Page 21: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/21.jpg)
Downloading MongoDB
www.mongodb.org
Binaries available for Linux, Mac, Windows, Solaris
![Page 22: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/22.jpg)
Start the DB Server
$ mkdir ~/db$$ tar zxvf mongodb-<OS>-1.0.0.tgz$ cd mongodb-<OS>-1.0.0$ bin/mongod --dbpath ~/db
![Page 23: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/23.jpg)
Connecting to the Database
use MongoDB;
my $connection = MongoDB::Connection->new;
my $db = $connection->get_database(‘test’);
my $collection = $db->get_collection(‘foo’);
![Page 24: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/24.jpg)
Inserting
my $id = $collection->insert( { title => ‘My first blog post’, author => ‘Joe’, content => ‘Hello, world!’
comments => [] });
![Page 25: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/25.jpg)
MongoDB::OIDan autogenerated primary key
my $id = $collection->insert({whatever});
print Dumper($id);--------------------------------------------
$VAR1 = bless( { 'value' => '4a9700dba5f9107c5cbc9a9c' }, 'MongoDB::OID' )
![Page 26: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/26.jpg)
Updating
$collection->update({_id => $id}, {‘$push’ => {comments => {
‘author’ => ‘Fred’,‘comment’ => ‘Dumb post.”
}}});
![Page 27: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/27.jpg)
…which gives us:
print Dumper($collection->find_one());--------------------------------------------{
title : ‘My first blog post’, author : ‘Joe’, content : ‘Hello, world!’ comments : [{ ‘author’ : ‘Fred’, ‘comment’ : ‘Dumb post’ }]}
![Page 28: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/28.jpg)
Magic (not the Perl kind)
$gt, $gte, $lt, $lte, $eq, $neq, $exists, $set, $mod, $where, $in, $nin, $inc$push, $pull, $pop, $pushAll, $popAll
$c->query({ x => {‘$gt’ => 4}})
![Page 29: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/29.jpg)
Querying
my $commented_by_fred = $collection->query({ “comments.author” : “Fred” });
my $commented_by_fred = $collection->query({ “comments.author” : qr/fred/i });
![Page 30: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/30.jpg)
$where
$collection->find_one({‘$where’ : ‘this.y == (this.x + this.z)’});
Will work:{x => 1, y => 4, z => 3}{x => “hi”, y => “hibye”, z => “bye”}
Won’t work:{x => 1, y => 1}
![Page 31: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/31.jpg)
Optimizing $where
$collection->find_one({ ‘name’ => ‘Sally’, ‘age’ => {‘$gt’ => 18}, ‘$where’ => ‘Array.sort(this.interests)[2] == “volleyball”’});
![Page 32: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/32.jpg)
Speaking of indexing…
$collection->ensure_index({“age” => “ascending”});
$collection->ensure_index({ “name” => “descending”, “ts” => “descending”, “comments.author” => “ascending”});
![Page 33: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/33.jpg)
Cursors
my $cursor = $coll->query({“foo” => “bar”});
my $obj;while (my $obj = $cursor->next) { ...}
my @all = $cursor->all;
![Page 34: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/34.jpg)
Paging
my $cursor = $coll->query-> sort({ts => -1})-> skip($page_num * $results_per_page)-> limit($results_per_page);
my @page = $cursor->all;
![Page 35: Non-Relational Databases](https://reader033.vdocuments.net/reader033/viewer/2022050720/5453d0caaf7959050e8b6d0e/html5/thumbnails/35.jpg)
Logging
• insert/update/remove is fast• Capped collections• Schemaless• $inc for counts