Download - Search with a Key-Value Store
![Page 1: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/1.jpg)
Search with a Key-Value Store
![Page 2: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/2.jpg)
Intro to NoSQL
•Key-value store
•Schemaless
•Distributed
•Eventually Consistent
![Page 3: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/3.jpg)
Key-Value
•Single unique key for each value in the database
•Extremely fast look-up
•Easy distribution (no such thing as joins)
![Page 4: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/4.jpg)
Schemaless
•Critical for extremely large data sets
•No alter table commands, each value has no pre-defined fields
![Page 5: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/5.jpg)
Distributed
•Data set is designed to be shared across multiple machines
•Typically makes use of commodity servers with enough RAM to keep the entire data set in memory
![Page 6: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/6.jpg)
Eventually Consistent
•Replica nodes are not notified of changes before a success response is returned to the client
•Makes NoSQL problematic for highly sensitive transactions (finance, etc)
![Page 7: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/7.jpg)
Database Design in NoSQL
•Denormalization is your friend
•Think of collections as views on a data set that
![Page 8: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/8.jpg)
A News Site Using SQL
Users
id
user_name
birthday
Stories
id
date
headline
content
Comment
id
story_id
user_id
content
![Page 9: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/9.jpg)
Loading a Story with SQL
SELECT * FROM stories
SELECT * FROM comments LEFT JOIN users ON users.id = comments.user_id
LEFT JOIN comments children ON children.parent_id = comments.id
WHERE story_id = x
![Page 10: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/10.jpg)
Redesigned in a NoSQL Data Store
Story #dgi3ck
dateheadlinecontent
commentsComment #la529
contentusername
user_image_urluser_idchildren
Comment #mn34icontent
usernameuser_image_url
user_idComment #5bg26content
usernameuser_image_url
user_idchildren
![Page 11: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/11.jpg)
Loading a Story with NoSQL
Stories::get(dgi3ck)
![Page 12: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/12.jpg)
Some Design Considerations
•What is the context in which we will access this data?
•What data do we need to access outside the of this context?
•How often does the data change?
![Page 13: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/13.jpg)
Embedded Data
•NoSQL can support foreign keys
•Some data is more appropriately stored “embedded” in a parent context
•E.g. Comments are rarely (if ever) accessed outside of their parent Story
![Page 14: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/14.jpg)
Cached Data
•Data from an object that needs to be accessed outside of the current context can be cached
•Keep in mind that it may need to be updated
•E.g. a user changes his username, Comments can be updated
![Page 15: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/15.jpg)
Several common NoSQL Stores
•Memcached
•BigTable
•SimpleDB
•MongoDB
![Page 16: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/16.jpg)
Why we chose MongoDB
•Auto-sharding and easy setup for distribution
•JavaScript API
•Powerful indexing capabilities
![Page 17: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/17.jpg)
MongoDB Libraries
•ORM: mongo_mapper
• https://github.com/jnunemaker/mongomapper
•Underlying Connection: mongo
• https://github.com/mongodb/mongo-ruby-driver
•BSON support: bson_ext
• http://rubygems.org/gems/bson_ext
![Page 18: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/18.jpg)
Lifebooker’s Availability Search
• Searches across Services
• Filters
• Time/Date
• Geographical Zone
• Service Category
• Practitioner Gender
• Concurrent Availability
• (and several more)
![Page 19: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/19.jpg)
Services, Discounts and Practitioners
•Services are offered by Providers
•Providers have Practitioners (Employees)
•Discounts are applied to Providers for a Service in a given time
![Page 20: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/20.jpg)
Modeling this Data in MongoDB
![Page 21: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/21.jpg)
Embedding with MongoMapper
![Page 22: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/22.jpg)
Indexing and Searching
•Mongo offers powerful indexing capabilities
•Arrays are “first-class citizens”
•Complex indices allow for great performance
![Page 23: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/23.jpg)
Creating Meta-Data
•With complex data structures, creating meta-data before_save will allow you to make that data easily searchable
•E.g. the maximum discount on a given day for a service
![Page 24: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/24.jpg)
Creating Indices
![Page 25: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/25.jpg)
Querying
•Uses DataMapper/Arel Syntax
•Chains conditions, ordering and offset
![Page 26: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/26.jpg)
Filtering Complex Data Structures
•MongoDB offers a JavaScript API for MapReduce
•Map - transform and filter data
•Reduce - combine multiple rows into a single record
![Page 27: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/27.jpg)
A Simple Use-Case
![Page 28: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/28.jpg)
Using MapReduce to Filter
Filter
![Page 29: Search with a Key-Value Store](https://reader036.vdocuments.net/reader036/viewer/2022081519/56814577550346895db24732/html5/thumbnails/29.jpg)
The Results•Scheduled to go live within 2 weeks
•With sharding/distribution, tests show almost no dip in response time with more than 10x the current data set
•20x faster than MySQL implementation
•100ms vs 2000ms (or more)