![Page 1: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/1.jpg)
Senior Consulting Engineer, MongoDB
Norman Graham
#MongoDBWorld
Retail Reference Architecture: Real-Time, Geo-Distributed Inventory
![Page 2: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/2.jpg)
Inventory
Inventory
MongoDB
External Inventory
Internal Inventory
Regional Inventory
Purchase Orders
Fulfillment
Promotions
![Page 3: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/3.jpg)
Inventory – Traditional Architecture
Relational DBSystem of Records
NightlyBatches
Analytics, Aggregations
,Reports
Caching Layer
Field Inventory
Internal & External
Apps
Point-in-time Loads
![Page 4: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/4.jpg)
Inventory – Opportunities Missed
• Can’t reliably detect availability
• Can't redirect purchasers to in-store pickup
• Can’t do intra-day replenishment
• Degraded customer experience
• Higher internal expense
![Page 5: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/5.jpg)
Inventory – Principles
• Single view of the inventory
• Used by most services and channels
• Read-dominated workload
• Local, real-time writes
• Bulk writes for refresh
• Geographically distributed
• Horizontally scalable
![Page 6: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/6.jpg)
Requirement Challenge MongoDB
Single view of inventory
Ensure availability of inventory
information on all channels and
services
Developer-friendly, document-oriented
storage
High volume, low latency reads
Anytime, anywhere access to inventory
data without overloading the system of record
Fast, indexed readsLocal reads
Horizontal scaling
Bulk updates,intra-day deltas
Provide window-in-time consistency for
highly available services
Bulk writesFast, in-place
updatesHorizontal scaling
Rapid application development
cycles
Deliver new services rapidly to capture new opportunities
Flexible schemaRich query language
Agile-friendly iterations
Inventory – Requirements
![Page 7: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/7.jpg)
Inventory – Target Architecture
Relational DBSystem of Records
Analytics, Aggregations
,Reports
Field Inventory
Internal & External
Apps
Inventory
Assortments
Shipments
Audits
Products
Stores
Point-in-time Loads
NightlyRefres
h
Real-timeUpdates
![Page 8: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/8.jpg)
Horizontal Scaling
Inventory – Technical Decisions
Store
Inventory
Schema
Indexing
![Page 9: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/9.jpg)
Inventory – Collections
Stores InventoryProducts
Audits AssortmentsShipments
![Page 10: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/10.jpg)
> db.stores.findOne(){ "_id" : ObjectId("53549fd3e4b0aaf5d6d07f35"), "className" : "catalog.Store", "storeId" : "store0", "name" : "Bessemer store", "address" : { "addr1" : "1st Main St", "city" : "Bessemer", "state" : "AL", "zip" : "12345", "country" : "US" }, "location" : [ -86.95444, 33.40178 ], ...}
Stores – Sample Document
![Page 11: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/11.jpg)
Stores – Sample Queries
• Get a store by storeId
db.stores.find({ "storeId" : "store0" })
• Get a store by zip code
db.stores.find({ "address.zip" : "12345" })
![Page 12: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/12.jpg)
What’s near me?
![Page 13: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/13.jpg)
Stores – Sample Geo Queries
• Get nearby stores sorted by distance
db.runCommand({
geoNear : "stores",
near : {
type : "Point",
coordinates : [-82.8006, 40.0908] },
maxDistance : 10000.0,
spherical : true
})
![Page 14: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/14.jpg)
Stores – Sample Geo Queries
• Get the five nearest stores within 10 km
db.stores.find({
location : {
$near : {
$geometry : {
type : "Point",
coordinates : [-82.80, 40.09] },
$maxDistance : 10000.0 } }
}).limit(5)
![Page 15: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/15.jpg)
Stores – Indices
• { "storeId" : 1 }, { "unique" : true }
• { "name" : 1 }
• { "address.zip" : 1 }
• { "location" : "2dsphere" }
![Page 16: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/16.jpg)
> db.inventory.findOne(){ "_id": "5354869f300487d20b2b011d", "storeId": "store0", "location": [-86.95444, 33.40178], "productId": "p0", "vars": [ { "sku": "sku1", "q": 14 }, { "sku": "sku3", "q": 7 }, { "sku": "sku7", "q": 32 }, { "sku": "sku14", "q": 65 }, ... ]}
Inventory – Sample Document
![Page 17: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/17.jpg)
Inventory – Sample Queries
• Get all items in a store
db.inventory.find({ storeId : "store100" })
• Get quantity for an item at a store
db.inventory.find({
"storeId" : "store100",
"productId" : "p200"
})
![Page 18: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/18.jpg)
Inventory – Sample Queries
• Get quantity for a sku at a store
db.inventory.find(
{
"storeId" : "store100",
"productId" : "p200",
"vars.sku" : "sku11736"
},
{ "vars.$" : 1 }
)
![Page 19: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/19.jpg)
Inventory – Sample Update
• Increment / decrement inventory for an item at a store
db.inventory.update(
{
"storeId" : "store100",
"productId" : "p200",
"vars.sku" : "sku11736"
},
{ "$inc" : { "vars.$.q" : 20 } }
)
![Page 20: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/20.jpg)
Inventory – Sample Aggregations
• Aggregate total quantity for a product
db.inventory.aggregate( [
{ $match : { productId : "p200" } },
{ $unwind : "$vars" },
{ $group : {
_id : "result",
count : { $sum : "$vars.q" } } } ] )
{ "_id" : "result", "count" : 101752 }
![Page 21: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/21.jpg)
Inventory – Sample Aggregations
• Aggregate total quantity for a store
db.inventory.aggregate( [
{ $match : { storeId : "store100" } },
{ $unwind : "$vars" },
{ $match : { "vars.q" : { $gt : 0 } } },
{ $group : {
_id : "result",
count : { $sum : 1 } } } ] )
{ "_id" : "result", "count" : 29347 }
![Page 22: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/22.jpg)
Inventory – Sample Aggregations
• Aggregate total quantity for a store
db.inventory.aggregate( [
{ $match : { storeId : "store100" } },
{ $unwind : "$vars" },
{ $group : {
_id : "result",
count : { $sum : "$vars.q" } } } ] )
{ "_id" : "result", "count" : 29347 }
![Page 23: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/23.jpg)
![Page 24: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/24.jpg)
Inventory – Sample Geo-Query
• Get inventory for an item near a point
db.runCommand( { geoNear : "inventory", near : { type : "Point", coordinates : [-82.8006, 40.0908] }, maxDistance : 10000.0, spherical : true, limit : 10, query : { "productId" : "p200", "vars.sku" : "sku11736" } } )
![Page 25: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/25.jpg)
Inventory – Sample Geo-Query
• Get closest store with available sku
db.runCommand( { geoNear : "inventory", near : { type : "Point", coordinates : [-82.800672, 40.090844] }, maxDistance : 10000.0, spherical : true, limit : 1, query : { productId : "p200", vars : { $elemMatch : { sku : "sku11736", q : { $gt : 0 } } } } } )
![Page 26: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/26.jpg)
Inventory – Sample Geo-Aggregation
• Get count of inventory for an item near a point
db.inventory.aggregate( [ { $geoNear: { near : { type : "Point", coordinates : [-82.800672, 40.090844] }, distanceField: "distance", maxDistance: 10000.0, spherical : true, query: { productId : "p200", vars : { $elemMatch : { sku : "sku11736", q : {$gt : 0} } } }, includeLocs: "dist.location", num: 5 } }, { $unwind: "$vars" }, { $match: { "vars.sku" : "sku11736" } }, { $group: { _id: "result", count: {$sum: "$vars.q"} } }])
![Page 27: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/27.jpg)
Inventory – Sample Indices
• { storeId : 1 }
• { productId : 1, storeId : 1 }
• { productId : 1, location : "2dsphere" }
• Why not "vars.sku"?– { productId : 1, storeId : 1, "vars.sku" : 1 }
![Page 28: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/28.jpg)
Horizontal Scaling
Inventory – Technical Decisions
Store
Inventory
Schema
Indexing
![Page 29: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/29.jpg)
ShardEast
ShardCentr
al
ShardWest
East DC
Inventory – Sharding TopologyWest DC Central DC
LegacyInventor
y
Primary
Primary
Primary
![Page 30: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/30.jpg)
Inventory – Shard Key
• Choose shard key– { storeId : 1 } ?– { productId : 1, storeId : 1 } ?– { storeId : 1, productId : 1 } ?
• Set up sharding– sh.enableSharding("inventoryDB")– sh.shardCollection( "inventoryDB.inventory", { storeId : 1, productId : 1 } )
![Page 31: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/31.jpg)
Inventory – Shard Tags
• Set up shard tags– sh.addShardTag("shard0000", "west")– sh.addShardTag("shard0001", "central")– sh.addShardTag("shard0002", "east")
• Set up tag ranges– sh.addTagRange("inventoryDB.inventory", { storeId : 0 }, { storeId : 100}, "west" )– sh.addTagRange("inventoryDB.inventory", { storeId : 100 }, { storeId : 200 }, "central" )– sh.addTagRange("inventoryDB.inventory", { storeId : 200 }, { storeId : 300 }, "east" )
![Page 32: Retail Reference Architecture Part 2: Real-Time, Geo Distributed Inventory](https://reader033.vdocuments.net/reader033/viewer/2022061103/540d5e3f8d7f72747e8b4960/html5/thumbnails/32.jpg)
Senior Consulting Engineer, MongoDB
Norman Graham
#MongoDBWorld
Thank You