Advanced Couchbase LiteJens Alfke — Mobile Architect, Couchbase
• Advanced Querying• Prefix matching• The power of compound keys• Grouping• Pseudo-joins
• Conflict Resolution• What goes on during a conflict?• Detecting documents in conflict• Resolving conflicts
Topics
Fancy Queries
Getting the most out of map/reduce
1. String Matching
• Doing string prefix matching?• Normalize (lowercase) the string• Set the startKey to the prefix• Set the endKey to the prefix plus a high Unicode char
Prefix Matching
…jedjemimajenniferjensjenzieljen⊱℥⧝jerry…
"jen"
"jen\uFFFE"
• Searching for a value in multiple properties?• Emit each property as a key
Prefix Matching
emit(doc.firstName, value) emit(doc.lastName, value)
…
Example: Name Lookup
normalize(str) { return str.toLowercase.stripDiacriticals}
map(doc) { value = [doc.firstName, doc.lastName] emit(normalize(doc.firstName), value) emit(normalize(doc.lastName), value)}
// Search for name starting with "jen":query.startKey = normalize("jen")query.endKey = query.startKey + "\uFFFE"
2. Compound Keys
emit([k1, k2, k3], value)
• The view index compares arrays item-by-item
• This creates a hierarchical sort
The Joy Of Compound Keys
primary = doc.yearsecondary = doc.titleemit([primary, secondary], value)
[2001, "Crows, The"][2001, "Vizier Of One"][2005, "Melon Growing For Dummies"][2005, "Zamboni Simulator"][2010, "Aardvark Trees"]
• Use key ranges to pick specific primary key values
Search Ranges With Compound Keys
query.startKey = [2005]query.endKey = [2005, {}]
sorts after anything else
• Results are always sorted by key
• But you may want a different display order
• If so, you need to sort manually
Search Order vs. Display Order
rows = query.run().allObjectsrows.sort({a, b -> a.value.rating < b.value.rating})
3. Grouping
query.groupLevel = 2
• Query.groupLevel coalesces adjacent rowswhose key prefixes match• Key is shortened to the common prefix• Value is computed by view’s reduce function
Compound Keys For Grouping
[2001, "Crows, The"][2001, "Vizier Of One"][2005, "Melon Growing"][2005, "Zamboni Simulator"][2010, "Aardvark Trees"]
[2001] ➞ 2
[2005] ➞ 2
[2010] ➞ 1
groupLevel=1
Example: Column View Of Music Library
{ "_id": "EA735A02", "Album": "Atomizer", "Artist": "Big Black", "Name": "Kerosene", "TotalTime": 365, "TrackNumber": 4}
Example: Column View Of Music Library
map(doc) { emit([doc.Artist, doc.Album, doc.TrackNumber, doc.Name], doc.TotalTime)}
reduce(keys, values, rereduce) { return sum(values)}
Example: Column View Of Music Library
// Column 1: List all artists:query.groupLevel = 1for row in query.run() { print row.key[0], row.value }
// Column 3: List tracks of “Atomizer”:query.groupLevel = 0 // no grouping!query.startKey = ["Big Black", "Atomizer"]query.endKey = ["Big Black", "Atomizer", {}]for row in query.run() { print row.key[3], row.value }
// Column 2: List albums by Big Black:query.groupLevel = 2query.startKey = ["Big Black"]query.endKey = ["Big Black", {}]for row in query.run() { print row.key[1], row.value }
4. Pseudo-Joins
by grouping document types
• Usually a view will index only one type of document
• …but you can combine types to do “pseudo-joins”
Joining by mixing document types
Example: Blog Posts With Comment Counts
{"_id": "4312DFC8",
"type": "post", "title": "lunch today", "date": "2014-09-24", "body": "…"}
{"_id": "79111745",
"type": "comment", "post": "4312DFC8", "date": "2014-09-25", "body": "…"}
Blog Post Comment
Example: Blog Posts With Comment Counts
"4312DFC8" ➞ "Welcome""76AFB751" ➞ "Some news""76AFB751" ➞ null"82EB5C7F" ➞ "Question""82EB5C7F" ➞ null"82EB5C7F" ➞ null"82EB5C7F" ➞ null"82EB5C7F" ➞ null
"4312DFC8" ➞ ["Welcome", 0]"76AFB751" ➞ ["Some news", 1]
"82EB5C7F" ➞ ["Question", 4]
groupLevel=1
Example: Blog Posts With Comment Counts
map(doc) { switch doc.type { case "post": emit(doc._id, doc.title) case "comment": emit(doc.post, null) }}
reduce(keys, values) { //Note: ignoring rereduce title = values.first({x -> x != null}) return [title, count(values)-1]}
// List every post's title and comment count:query.groupLevel = 1for row in query.run() { print row.value[0], row.value[1] }
• Easy prefix matching
• Custom sorting
• In-memory filtering
• Fancy “query planner”
Upcoming Query Enhancements!
Don’t miss the next session:The Future Of Couchbase Lite
Conflict Resolution
• Different revisions with the same parent revision
What Are Conflicts?
Resolving A Conflict: Diagram
name: jillscore: 2
name: Jillscore: 2
name: jillscore: 3
99cf02e8
• Different revisions with the same parent revision
• Conflicts are prevented when working locally• Document.putProperties returns a “conflict” error
• Cannot be prevented in a distributed system• Multiple clients can update a document, then sync• Locking documents is infeasible
• A conflict is a branch in the revision tree
What Are Conflicts?
• How to detect conflicts?
• How to resolve conflicts?
Dealing With Conflicts
• Checking a document for conflicts• Document.getConflictingRevisions()
• Querying the database for conflicts• All-documents query with OnlyConflicts mode
Detecting Conflicts
query = database.createAllDocsQuery()query.allDocsMode = OnlyConflictsfor row in query.run() { conflicts = row.conflictingRevisions() myResolveConflict(row.document, conflicts)}
• Determine the resolved/merged properties
• Update one of the conflicting revisions
• Delete the other(s)
Resolving A Conflict
Resolving A Conflict: Diagram
name: jillscore: 2
name: Jillscore: 2
name: jillscore: 3
name: Jillscore: 3
_deleted: true
Resolving A Conflict: Pseudocode
myResolveConflict(doc, conflicts) {mergedProps = myMergeRevisions(conflicts)current = doc.currentRevisionfor rev in conflicts {
newRev = rev.createRevision()if rev == current {
newRev.properties = mergedProps} else {
newRev.isDeletion = true}newRev.save()
}}
app-specific merge strategy
• Any client can resolve conflicts• Resolution will be synced to everyone else• Resolution can be done on the server too
• If multiple clients resolve the same conflict• It might be fine (if the resolutions are identical)• Or it might create a new conflict to resolve
Interesting Facts
Questions?