five things you need to know about scaling
TRANSCRIPT
The 5 Things You Need To Know
About ScalingA. Jesse Jiryu Davis
Senior Python Engineer
1. Indexes
db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BasicCursor",!! "n" : 10,!! "nscanned" : 100000,!! "millis" : 40,!}!!db.people.createIndex({name: 1});!!db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 10,!! "nscanned" : 10,!! "millis" : 0,!}!
1. Indexes
db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BasicCursor",!! "n" : 10,!! "nscanned" : 100000,!! "millis" : 40,!}!!db.people.createIndex({name: 1});!!db.people.find({name: 'Jesse'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 10,!! "nscanned" : 10,!! "millis" : 0,!}!
1. Indexes
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!
1. Indexes
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!
1. Indexes
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!
1. Indexes
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1_hacks_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!!db.people.dropIndex({name: 1})!!
1. Indexes
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1",!! "n" : 1,!! "nscanned" : 10,!! "millis" : 0,!}!!db.people.createIndex({name: 1, hacks: 1});!!db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!{!! "cursor" : "BtreeCursor name_1_hacks_1",!! "n" : 1,!! "nscanned" : 1,!! "millis" : 0,!}!!db.people.dropIndex({name: 1})!!
1. Indexes
2. File System
• ext3 is bad• ext4 and xfs are good
• Set noatime • This goes in /etc/fstab
2. File System
3. Working Set Size
3. Working Set Size
Disk!
RAM!
4. Disks
4. DisksOption 1: Spinning disk.
100 seeks / sec
4. DisksOption 2: Spinning disks!
~100 seeks / second!
~100 seeks / second! ~100 seeks / second! ~100 seeks / second!
4. DisksOption 3: SSD
5. Shard
Primary!
Secondary!
Secondary!
MongoS!
Primary!
Secondary!
Secondary!
Primary!
Secondary!
Secondary!
Primary!
Secondary!
Secondary!
5. Shard
1. Indexes 2. File System 3. Working Set 4. Disks 5. Shard
1. Indexes 2. File System 3. Working Set 4. Disks 5. Shard
The 5 Things You Need To Know About Scaling
A. Jesse Jiryu Davisbit.ly/scaling-five