Шардинг в mongodb, henrik ingo (mongodb)

Download Шардинг в MongoDB, Henrik Ingo (MongoDB)

Post on 21-Jun-2015

504 views

Category:

Internet

2 download

Embed Size (px)

DESCRIPTION

HighLoad++ 2014.

TRANSCRIPT

  • 1. Solutions Architect, MongoDBHenrik Ingo@h_ingoIntroduction to Sharding

2. AgendaScaling DataMongoDB'sApproachArchitectureConfigurationMechanics 3. MongoDB's Approach to Sharding 4. Partitioning User defines shard key Shard key defines range of data Key space is like points on a line Range is a segment of that line 5. Initially 1 chunkDefault max chunk size: 64mbMongoDB automatically splits & migrates chunkswhen max reachedData Distribution 6. Queries routed to specific shardsMongoDB balances clusterMongoDB migrates data to new nodesRouting and Balancing 7. MongoDB Auto-Sharding Minimal effort required Same interface as single mongod Two steps Enable Sharding for a database Shard collection within database 8. Architecture 9. What is a Shard? Shard is a node of the cluster Shard can be a single mongod or a replica set 10. Sharding infrastructure 11. Configuration 12. Example Cluster 13. mongod --configsvrStarts a configuration server on the default port (27019)Starting the Configuration Server 14. mongos --configdb :27019For 3 configuration servers:mongos --configdb :,:,:This is always how to start a new mongos, even if the cluster is already runningStart the mongos Router 15. mongod --shardsvrStarts a mongodwith the default shard port (27018)Shard is not yet connected to the rest of the clusterShard may have already been running in productionStart the shard database 16. On mongos: sh.addShard(:27018)Adding a replica set: sh.addShard(/)Add the Shard 17. db.runCommand({ listshards:1 }){ "shards" :[{"_id: "shard0000,"host: :27018 } ],"ok" : 1}Verify that the shard was added 18. Enabling Sharding Enable sharding on a databasesh.enableSharding() Shard a collection with the given keysh.shardCollection(.people,{country:1}) Use a compound shard key to prevent duplicatessh.shardCollection(.cars,{year:1,uniqueid:1}) 19. Tag Aware Sharding Tag aware sharding allows you to control thedistribution of your data Tag a range of shard keys sh.addTagRange(,,,) Tag a shard sh.addShardTag(,) 20. Routing Requests 21. Cluster Request Routing Targeted Queries Scatter Gather Queries Scatter Gather Queries with Sort 22. Cluster Request Routing: Targeted Query 23. Routable request received 24. Request routed to appropriate shard 25. Shard returns results 26. Mongos returns results to client 27. Cluster Request Routing: Non-Targeted Query 28. Non-Targeted Request Received 29. Request sent to all shards 30. Shards return results to mongos 31. Mongos returns results to client 32. Cluster Request Routing: Non-Targeted Querywith Sort 33. Non-Targeted request with sort received 34. Request sent to all shards 35. Query and sort performed locally 36. Shards return results to mongos 37. Mongos merges sorted results 38. Mongos returns results to client 39. Shard Key 40. Shard Key Shard key is immutable Shard key values are immutable Shard key must be indexed Shard key limited to 512 bytes in size Shard key used to route queries Choose a field commonly used in queries Only shard key can be unique across shards `_id` field is only unique within individual shard 41. Shard Key Considerations Cardinality Write Distribution Query Isolation Reliability Index Locality 42. {_id: ObjectId(),user: 123,time: Date(),subject: ...,recipients: [],body: ...,attachments: []}Example: Email StorageMost common scenario, can be applied to 90% casesEach document can be up to 16MBEach user may have GBs of storageMost common query: get user emails sorted by timeIndexes on {_id}, {user, time}, {recipients} 43. Example: Email StorageCardinalityWriteScalingQueryIsolationReliabilityIndexLocality 44. Example: Email StorageCardinalityWriteScalingQueryIsolationReliabilityIndexLocality_id Doc level One shardScatter/gatherAll usersaffectedGood 45. Example: Email StorageCardinalityWriteScalingQueryIsolationReliabilityIndexLocality_id Doc level One shardScatter/gatherAll usersaffectedGoodhash(_id) Hash level All ShardsScatter/gatherAll usersaffectedPoor 46. Example: Email StorageCardinalityWriteScalingQueryIsolationReliabilityIndexLocality_id Doc level One shardScatter/gatherAll usersaffectedGoodhash(_id) Hash level All ShardsScatter/gatherAll usersaffectedPooruser Many docs All Shards TargetedSome usersaffectedGood 47. Example: Email StorageCardinalityWriteScalingQueryIsolationReliabilityIndexLocality_id Doc level One shardScatter/gatherAll usersaffectedGoodhash(_id) Hash level All ShardsScatter/gatherAll usersaffectedPooruser Many docs All Shards TargetedSome usersaffectedGooduser, time Doc level All Shards TargetedSome usersaffectedGood 48. @h_ingoQuestions? 49. Solutions Architect, MongoDBHenrik Ingo@h_ingoThank you!