introduction to nosql with couchbasefiles.meetup.com/18258964/introduction to nosql...
TRANSCRIPT
![Page 1: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/1.jpg)
Introduction to NoSQL with Couchbase Couchbase Meetup – Bangalore
#cbmeetupblr
![Page 2: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/2.jpg)
About the Speaker
• Clarence J M Tauro – [email protected] – @javapsyche on Twitter – Senior Instructor, Couchbase – ~11 Years Professional Teaching and Consulting Experience – PhD in Computer Science from Christ University [thesis
accepted] – Hard-core Dog lover
![Page 3: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/3.jpg)
Disclaimer
• Disclaimer: The views expressed in this presentation are our own and do not necessarily reflect the views of Couchbase
![Page 4: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/4.jpg)
Objectives
• Introduction to NoSQL • Are ACID Properties always desirable? • Basically available, Soft state, Eventually consistent
(BASE) • The CAP Theorem • Introducing Couchbase • Couchbase Operations
![Page 5: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/5.jpg)
Introduction
RDBMS - predominant technology for storing structured data in web and business applications
“one size fits all” - thinking concerning data-stores has been questioned Apply NoSQL databases for the persistence layer/Polyglot Programming
![Page 6: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/6.jpg)
ACID Properties
• ATOMICITY
• CONSISTENCY
• ISOLATION
• DURABILITY
![Page 7: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/7.jpg)
Are ACID Properties always desirable?
• ... But what about: – Latency – Partition Tolerance – High Availability – Scalability
![Page 8: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/8.jpg)
the system is available, but not
necessarily all items in it at any given
point in time
after a certain time all nodes are consistent, but
at any given time this might not be the case
information (state) the user put into the system that will go
away if the user doesn't maintain it
BASE
![Page 9: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/9.jpg)
NoSQL Common Traits
• Non-relational • Schema-free/Schema-on-read • Eventual consistency • Open source • Distributed • “web-scale”
![Page 10: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/10.jpg)
The CAP Theorem
• Consistency – can all nodes see identical data, at all times?
• Availability – can all nodes be read from and written to, at all times?
• Partition Tolerance – will nodes function normally, even when the cluster breaks?
Consistency
Partition Tolerance
Availability
CHOOSE ANY TWO
![Page 11: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/11.jpg)
The CAP Theorem
• CP: Consistency and Partition Tolerance - Immediately consistent data across a horizontally scaled
cluster, even with network problems - Couchbase
• AP: Availability and Partition Tolerance - Always services requests, across multiple data centers,
even with network problems, data eventually consistent - Apache HBase or Cassandra, Couchbase (XDCR)
• CA: Consistency and Availability - Always services requests with immediately consistent
data, in a vertically scaled system - MySQL, Oracle, Microsoft SQL Server
![Page 12: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/12.jpg)
What do you do with the Data?
Operational Use • Real time intelligence • Focus on data flows and
processes • Extremely fast (in-
memory) reads • Extremely fast (log
append) writes • Improve the current
outcome
Analytical Use • Batched workloads • Vast data aggregations • Retrospective analyses • Focus on data pools • Improve future outcomes
![Page 13: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/13.jpg)
Hadoop and NoSQL
Operational Velocity Analytical Volume
Real-time operational database systems
improve current outcomes
Batch-oriented analytical database systems improve future outcomes
Hadoop NoSQL
![Page 14: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/14.jpg)
Types of NoSQL
• Key-value stores • Wide Column stores • Document stores • Graph databases
![Page 15: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/15.jpg)
Key-Value Stores
• The most common; not-necessarily the most popular • Key and a simple value - Speed - Scale - Simplicity • Find simple values by key extremely fast
Clarence user::1234
Melisa user::1235
Michael user::1236
![Page 16: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/16.jpg)
Document Stores
• Key and a structured value (document) - Speed - Scale - Flexibility • Read/write ever-changing data about people, places,
and things, at cloud-scale
user::1234 { name: 'Frank', age: 37, kids: ['Sue', 'Ann', 'Bob'] }
user::1235 { name: 'Carolyn', age: 56, kids: ['Tina'] }
user::1236 { name: 'Tessa', age: 24}
![Page 17: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/17.jpg)
Wide Column Stores
• Key and nested set of tuples - Write vast volumes of data, with eventually consistent
read access
user::1234
name: text Frank
age: number 37
kid: text
Sue
Ann
Bob
user::1235
name: text Carolyn
age: number 56
kid: text Tina
![Page 18: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/18.jpg)
Graph Databases
• Linked list of keyed objects - Relationships • Monitor complex, dynamically networked connections
user::1234 Frank
37
Sue
Ann
Bob
user::1235
Carolyn 56
Tina
user::1236 Tessa
24
![Page 19: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/19.jpg)
Polyglot Programming
• Enterprise will have a variety of different data storage technologies for different kinds of data
• We need to ask how we want to manipulate the data. This will help us figure out which persistence technologies are appropriate - User Sessions: Couchbase (Memcached)/Redis - Financial Data: RDBMS - Shopping Cart: Riak/Couchbase (Memcached) - Recommendation Systems: Neo4J - Product Catalog: Couchbase/MongoDB - Reporting: RDBMS/Couchbase Views - Analytics: Couchbase/Cassandra
![Page 20: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/20.jpg)
History of Couchbase
NorthScale developed a key-value storage engine
Apache CouchDB database project
Membase and CouchOne joined forces in February 2011 to create Couchbase, the first and only provider
of a comprehensive, end-to-end family of NoSQL database products
![Page 21: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/21.jpg)
What is Couchbase Server?
• Couchbase Server • Is a “document” database solution • Has key/value based orientation • Is geared for JSON • Has no tables and no fixed schema • Runs on a networked cluster of nodes • Is highly scalable • Is lightning fast read/write • Has caching and persistence layers • Automatically fails-over
• Couchbase Server is best suited for fast-changing data items of relatively small size
![Page 22: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/22.jpg)
JavaScript Object Notation
{ "firstName": "Clarence", "lastName": "Tauro", "age": 25, "address": { "streetAddress": "21 2nd Street", "city": "Bangalore", "state": "KA", "postalCode": "560059" }, "phoneNumber": [ { "type": "home", "number": "988 621-‐7674" } ] }
JSON is a lightweight data-interchange format easy for humans to read and
write
![Page 23: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/23.jpg)
What is a Couchbase Document?
{ "visibility": "PRIVATE", "name": "Eclectic Summer Mix", "userName": "suzyqrocks", "type": "org.couchmusic.domain.Playlist", "created": 1422138028037, "updated": 1422138028072, "tracks": [] }
{ "id": "playlist:12345", "rev": "1-‐0004ebc0000000000", "flags": 0, "expiration": 0, "type": "json" }
Document Content
(Most recent in RAM and persisted to disk)
Document Metadata
(All keys unique and kept in RAM)
![Page 24: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/24.jpg)
Couchbase Server Architecture
![Page 25: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/25.jpg)
• Technology Stack for Data Manager: Couchbase Client SDK (“Smart Client”) Client Query API1 and Query Engine (Views) Cache Layer: RAM Cache Persistence Layer: Couchbase
Couchbase Server Architecture
![Page 26: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/26.jpg)
• Technology Stack for Cluster Manager: Node Level – multiple vBuckets
• Default 1024 vBuckets/number of nodes Cluster Level – multiple nodes (with 1 .. * buckets)1
Datacenter Level – multiple clusters (optional XDCR)2
Erlang (cluster management and process supervision)3
Couchbase Server Architecture
![Page 27: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/27.jpg)
Anatomy of a Couchbase Application
Couchbase Client Software
Cluster Map
NS Server EP Engine
NS Server EP Engine
NS Server EP Engine
{Server List} 1. REST request 8091
5. Create, Read, Update and Delete Documents
Becomes a Smart Client
![Page 28: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/28.jpg)
3 3 2 Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Doc 1 Doc 1
Doc 1
To other node
Single Node – Couchbase Write Operation
Couchbase Server Node
![Page 29: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/29.jpg)
3 3 2 Managed Cache
Dis
k Q
ueue
Replication Queue
App Server
Doc 1’
Doc 1
Doc 1’ Doc 1
Doc 1’
Disk
To other node
Single Node – Couchbase Update Operation
Couchbase Server Node
![Page 30: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/30.jpg)
GET
D
oc 1
3 3 2
Dis
k Q
ueue
Replication Queue
App Server
Doc 1
Doc 1 Doc 1
Managed Cache
Disk
To other node
Single Node – Couchbase Read Operation
Couchbase Server Node
![Page 31: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/31.jpg)
3 3 2
2
Dis
k Q
ueue
Replication Queue
App Server
Couchbase Server Node
Doc 1
Doc 6 Doc 5 Doc 4 Doc 3 Doc 2
Doc 1
Doc 6 Doc 5 Doc 4 Doc 3 Doc 2
Managed Cache
Disk
To other node
Single Node – Couchbase Cache Eviction
![Page 32: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/32.jpg)
3 3 2
2
Dis
k Q
ueue
Replication Queue
App Server
Couchbase Server Node
Doc 1
Doc 3 Doc 5 Doc 2 Doc 4
Doc 6 Doc 5 Doc 4 Doc 3 Doc 2
Doc 4
GET
D
oc 1
Doc 1
Doc 1
Managed Cache
Disk
To other node
Single Node – Couchbase Cache Miss
![Page 33: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/33.jpg)
Other – New Features of Couchbase 4.0
• Multi-dimensional Scaling • N1QL • GSI • XDCR
![Page 34: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/34.jpg)
Goodies!
• Tweet about this event with - #cbmeetupblr - @couchbase or #couchbase - @javapsyche
• The person with the maximum retweets receives a special goodie in the next meetup
![Page 35: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/35.jpg)
Training
Get Started with Couchbase Server 4.0: www.couchbase.com/beta
Get Trained on Couchbase: http://training.couchbase.com
CD220: Developing Couchbase NoSQL Applications Oct 13 – Oct 15 2015
CS300: Couchbase NoSQL Server Administration
Sep 28 – Oct 1
Enroll Today!
![Page 36: Introduction to NoSQL with Couchbasefiles.meetup.com/18258964/Introduction to NoSQL with...Introduction RDBMS - predominant technology for storing structured data in web and business](https://reader030.vdocuments.net/reader030/viewer/2022040109/5e772ebd4fd50c42f24e9b1e/html5/thumbnails/36.jpg)
Questions?