Developing with Couchbase: Document Your World

Michael NitschingerEngineer, Developer Solutions

What to Expect:

• JSON Basics & Documents

• Mind-set Changes

• Building an application around JSON

• Document Structuring & Modeling

• Views and Indexes within Couchbase

JSON Basics – what is JSON?

Java Script Object Notation

• Created by Douglas Crockford

• Text Based Format

• Subset of JavaScript

• Human Readable

JSON Basics – Why JSON?

• Compact

• Understandable

• Data structures map nicely

• Widely adopted

Supported JSON Types:

String:

Numbers: – (Int. & Floating Point)

"A String"

Boolean:

{“value” : false}

Object:

{ ”name" : “Michael N.” “twitter" : “@daschl", ”age" : 25, "title" : ”Engineer"}

22 & 55.2

Supported JSON Types - Lists:

Array:

["one", "two", "three"]foos : [ { ”bar1":"value1", ”bar2":"value2" }, { ”bar3":"value3", ”bar4":"value4" }]

List of Objects:

Complex, Nested Objects:

{ tweet, tweet… }

JSON Documents within Couchbase

• Documents

• have a unique ID

• can store JSON (also binary) as the value

• have meta information attached

• If it’s JSON, the document can be used in Views (secondary

indexing)

meta{“id”: “foo@couchbase.com”,“rev”: “1-0002bce0000000000”,“flags”: 0,“expiration”: 0,“type”: “json”}

document{“uid”: 1234,“firstname”: “Michael”,“lastname”: “Nitschinger”,“age”: 25,“favorite_colors”: [“blue”, “green”],“email”: “foo@couchbase.com”}

Meta Information Including Key (ID)

All Keys Unique and Kept in RAM at all times.

Document Value

Most Recent In Ram And Persisted To Disk

JSON Document Structure

User Objectstring uid

string firstname

string lastname

int age

array favorite_colors

string email

u::foo@couchbase.com{“uid”: 1234,“firstname”: “Michael”,“lastname”: “Nitschinger”,“age”: 17,“favorite_colors”: [“blue”, “green”],“email”: “foo@couchbase.com”}

User Objectstring uid

string firstname

string lastname

int age

array favorite_colors

string email

set()

get()

Objects Serialized to JSON and Back

u::foo@couchbase.com{“uid”: 1234,“firstname”: “Michael”,“lastname”: “Nitschinger”,“age”: 25,“favorite_colors”: [“blue”, “green”],“email”: “foo@couchbase.com”}

The Mind-Set Change

• Information spread across (multiple) Tables,

• Rigid and inflexible schema

• Records need to look the same

• Complex JOINS, WHERE Clauses and ORDER BY Clauses

The Move from Relational Modeling

Our ‘Recipe’ table uses “JOINS” to aggregate info from other Tables.

The Move to NoSQL

• Gets and Sets are quick, do not fear them!

• Schema is dictated by the application, not the database.

• No ALTER_TABLE needed!• Each Document can look different.

Modeling an Application…The JSON way

Application in which people can

• vote on other User’s Videos

• see a Global Ranking of the Best and Worst Videos!

Rate My Vine…

Top Rated Vines

Cooking w/ Hugh Fearnley-Whittingstall

I love doing Housework

What happened to Amanda Bynes

Random Access Memories

I don’t even know

Twerking gone wrong

Too cold to Dance

How To Scare Your Friends

176

143

120

112

107

98

74

37

• Open Source Sample App for Couchbase

• Built on Ruby, Rails & Couchbase­ Couchbase-Model Ruby Gem for Active-Record style (easy) data

modeling­ Puma as web server for concurrent connections

Technology Used:

•Users must Auth with Twitter before Submitting Vines•We simply register their Name, Twitter Username & Avatar upon T-auth

User.rb

•Standard JSON structure with simple String fields

•This JSON is editable within the Couchbase Console

How that looks as JSON in Couchbase:

Key created by a hash of Twitter UID

Explicit ‘type’ of Document

•Vines need a Name, A Video URL, a User and a Score

Vine.rb

• Product Manager has informed us that we need to add a new field for Facebook Sharing into our Vine Videos

• In a relational world, we would have to do schema migrations

• In the Couchbase world, it’s much easier!

The Joys of a Flexible Schema!

•User_ID included so we know who each Vine belongs to

•Score is inside each Vine document. This brings it’s own challenges, but Couchbase solves them!

Again, the JSON within Couchbase

Random Hash generated Key

User_ID reference

•We have chosen to have the Score inside each Vine doc.

•We need to be able to deal with concurrent score updates.

Optimistic Concurrency

{ “score" : 174}

UPDATE UPDATE

• To handle the Concurrent updates, we can utilise Couchbase’s built in CAS concurrency controls.

• We simply write a new Update method in our application controller to use the CAS value on update.

CAS – Compare and Swap

•Just as in SQL, our JSON Documents also have various types of ‘Relationship’.

•For example, a User can own many Videos as a 1 to many relationship.

Document Relationships

video:1{ type: “vine”, title: “My Epic Video”, owner: “ingenthr”}

user:ingenthr{ type: “user”, name: “Matt Ingenthron”, id: “ingenthr”}

video:2{ type: “vine”, title: “I NEED A HORSE!”, owner: “ingenthr”}

• Product Manager has informed us we need to add a Comment mechanism to our Vine Videos.

• We need to decide the best way to approach this in JSON document design.

Single vs. Multiple Documents

{

}

Single Multiple

vs.

Document

Comment

Comment

Comment

• Comments are nested within their respective Vine documents.

• Great when we know we have a finite amount of Results.

Single vs. Multiple - Single

{ "type": "vine", "user_id": "145237874", "title": "I NEED A HORSE", "vine_url": "https://vine.co/v/b2jjzY0Wqg5", "video_url": "https://mtc.cdn.vine.co……, "score": 247, "comments": [ {"format": "markdown", "body": "I LOVE this video!"}, {"format": "markdown", "body": "BEST video I have ever seen!"}, ]}

7b18b847292338bc29

•Comments are split from the parent document.

•Comments use referential ID’s, incremented by 1

Single vs. Multiple - Multiple

{ "type": "vine", "user_id": "145237874", "title": "I NEED A HORSE", "score": 247,}

7b18b847292338bc29 { "format": "markdown", "body": "I LOVE this video!”}

7b18b847292338bc29::1

{ "format": "markdown", "body": “BEST video ever!”}

7b18b847292338bc29::2

•Couchbase has no built in mechanism for Versioning.

•There are many ways to approach document Versioning.­Copy the versions of the document into new documents,­Copy the versions of the document into a list of nested documents,­Store the list of mutated / modified attributes:

• In nested Element,

• In separate Documents.

•In this case, we’re going to look at the simplest way…

Versioning our Documents:

•Get the current version of the document,

•Increment the version number,

•Create the version with the new key "mykey::v1”,

•Save the document in it’s current version.

Versioning our Documents:

Current Version:

Version 1:

Version 2:

mykey

mykey::v1

mykey::v2

Questions?