webinar - einfuehrung in couchbase
DESCRIPTION
Couchbase Server 2.0 ist eine dokumentenorientierte NoSQL Datenbank. Features wie native JSON Unterstützung, umfangreiche Abfragemechanismen und die Möglichkeit zwischen mehreren Datacentern zu replizieren sind seit der neuen Version enthalten. Zusätzlich wurden alt bekannte Funktionen wie Performanz und einfache Skalierbarkeit weiter verbessert.TRANSCRIPT
![Page 1: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/1.jpg)
Einführung inCouchbase Server 2.0
![Page 2: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/2.jpg)
• Developer Advocate at Couchbase, Inc.
• Maintainer of the Couchbase Java SDK
• Speaking at Conferences and Meetups
• Living and Working in Vienna, Austria
{“about”: “me”}
![Page 3: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/3.jpg)
1. Introduction to Couchbase Server 2.0
2. Cluster Management
3. Operations & Document Design
4. Introduction to Views
Agenda
![Page 4: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/4.jpg)
Couchbase Server 2.0
![Page 5: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/5.jpg)
Big Release in December 2012
![Page 6: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/6.jpg)
Couchbase Open Source Project
• One of the leading NoSQL database projects focused on distributed database technology and surrounding ecosystem
• Supports both key-value and document-oriented use cases
• All components are available under the Apache 2.0 Public License
• Obtained as packaged software in both enterprise and community editions.
![Page 7: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/7.jpg)
Easy Scalabili
ty
Consistent High
Performance
Always On
24x365
Grow cluster without application changes, without downtime with a single click
Consistent sub-millisecond read and write response times
with consistent high throughput
No downtime for software upgrades, hardware maintenance, etc.
JSONJSONJSON
JSONJSON
PERFORMANCE
Flexible Data Model
JSON document model with no fixed schema.
Core Principles
![Page 8: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/8.jpg)
New in 2.0
JSON support Indexing and Querying
Cross data center replicationIncremental Map Reduce
JSONJSONJSON
JSONJSON
![Page 9: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/9.jpg)
Couchbase Server 2.0 Architecture
Hea
rtbe
at
Proc
ess
mon
itor
Glo
bal s
ingl
eton
sup
ervi
sor
Confi
gura
tion
man
ager
on each node
Reba
lanc
e or
ches
trat
or
Nod
e he
alth
mon
itor
one per cluster
vBuc
ket s
tate
and
repl
icati
on m
anag
er
httpRE
ST m
anag
emen
t API
/Web
UI
HTTP8091
Erlang port mapper4369
Distributed Erlang21100 - 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached
New Persistence Layer
8092Query API
Que
ry E
ngin
e
Data Manager Cluster Manager
![Page 10: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/10.jpg)
33 2
Single node - Couchbase Write Operation
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
![Page 11: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/11.jpg)
GET
Doc
1
33 2
Single node - Couchbase Read Operation
Dis
k Q
ueue
Replication Queue
App Server
Doc 1
Doc 1Doc 1
Managed Cache
Disk
To other node
Couchbase Server Node
![Page 12: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/12.jpg)
COUCHBASE SERVER CLUSTER
Basic Operation
• Docs distributed evenly across servers
• Each server stores both active and replica docsOnly one server active at a time
• Client library provides app with simple interface to database
• Cluster map provides map to which server doc is onApp never needs to know
• App reads, writes, updates docs
• Multiple app servers can access same document at same time
User Configured Replica Count = 1
READ/WRITE/UPDATE
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 1
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc
SERVER 2
Doc 8
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
REPLICA
Doc 6
Doc 3
Doc 2
Doc
Doc
Doc
REPLICA
Doc 7
Doc 9
Doc 5
Doc
Doc
Doc
SERVER 3
Doc 6
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
Doc 9
![Page 13: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/13.jpg)
Add Nodes to Cluster
• Two servers addedOne-click operation
• Docs automatically rebalanced across clusterEven distribution of docsMinimum doc movement
• Cluster map updated
• App database calls now distributed over larger number of servers
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3 SERVER 4 SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc
Doc 8 Doc
Doc 9 Doc
Doc 2 Doc
Doc 8 Doc
Doc 5 Doc
Doc 6
READ/WRITE/UPDATE READ/WRITE/UPDATE
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
COUCHBASE SERVER CLUSTER
User Configured Replica Count = 1
![Page 14: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/14.jpg)
Fail Over Node
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3 SERVER 4 SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc 9
Doc 8
Doc Doc 6 Doc
Doc
Doc 5 Doc
Doc 2
Doc 8 Doc
Doc
• App servers accessing docs
• Requests to Server 3 fail
• Cluster detects server failedPromotes replicas of docs to activeUpdates cluster map
• Requests for docs now go to appropriate server
• Typically rebalance would follow
Doc
Doc 1 Doc 3
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
User Configured Replica Count = 1
COUCHBASE SERVER CLUSTER
![Page 15: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/15.jpg)
![Page 16: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/16.jpg)
![Page 17: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/17.jpg)
Q & A
![Page 18: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/18.jpg)
Operations & Document Design
![Page 19: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/19.jpg)
Fundamentals
• Every Document has a Key assigned to it
• Keys must be max. 255 UTF-8 chars long must be unique in a bucket (“database”) are completely under the control of the application
• Values can be any binary blob (but bonus points when JSON!) can be up to 20MB in size
![Page 20: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/20.jpg)
Basic Store & Retrieve Operations
• get(key)Retrieve a document
• set(key, value)Store a document or replace if it exists
• add(key, value)Store a document and fail if it exists
• replace(key, value)Replace a document and fail if it does not exist
![Page 21: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/21.jpg)
Lots of other Operations
• View operations
• Atomic Counters (increment, decrement)
• Append/Prepend
• CAS (Compare and Set) Optimistic Locking
• “Get with Lock” Write Lock on Objects
• Bulk Operations Saves network overhead
• Stats
![Page 22: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/22.jpg)
Couchbase 2.0 Bonus Points: JSON
• JSON is a lightweight format to represent document structure in a language-independent manner.
• If JSON documents are stored, the View engine can be used.
• Allows to build secondary indexes on your datasets.
• Makes it possible to ask questions like“Give me all user documents by lastname”.
![Page 23: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/23.jpg)
The BIG mental adjustment
• In SQL, we tend to avoid hitting the database as much as possible. Use JOINs and let the DB optimizer figure out what to do
• In Couchbase, get’s and set‘s are so fast they are trivial, not bottlenecks, this is hard for many people to accept at first; Multiple get statements are commonplace, don’t avoid it!
![Page 24: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/24.jpg)
meta{“id”: “u::[email protected]”,“rev”: “1-0002bce0000000000”,“flags”: 0,“expiration”: 0,“type”: “json”}
document{“uid”: 123456,“firstname”: “Michael”,“lastname”: “Nitschinger”,“age”: 25,“favorite_colors”: [“blue”, “black”],“email”: “michael.nitschinger@...”}
MetaInformationIncluding Key
All Keys Unique and Kept in RAM
DocumentValue
Most Recent In Ram And Persisted To Disk
Document Structure with JSON
![Page 25: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/25.jpg)
Views & Indexes
• Views can cover a few different use cases Simple secondary indexes (the most common) Complex secondary, tertiary and composite indexes Aggregation functions (reduction)
•Example: count the number of North American Ales
Organizing related data
• Built using Map/Reduce Map function creates the index (b-tree) Reduce function summarizes (reduces) information Written using superfast Javascript (V8)
![Page 26: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/26.jpg)
A View in Action
![Page 27: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/27.jpg)
Developing with Couchbase
![Page 28: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/28.jpg)
Aggregate View of Data
http://martinfowler.com/bliki/AggregateOrientedDatabase.html
![Page 29: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/29.jpg)
What is an aggregate?
WhenyouneedtoretrievedatafromRDBMS,youare"aggregating"or"denormalizing"thedataforyourapplicationthroughquerieswithjoins,whereclausesandorderbyclauses.
InDocumentDatabases,insteadofbreakingdataintotablesandforeignkeys,youstoretheaggregatedatatogetherinJSONdocument(s).
![Page 30: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/30.jpg)
Store and Retrieve Aggregates
• Easier to Distribute Data• More Flexibility• Reduced Latency
order::1001{uid: ji22jd,customer: Ann,line_items: [
{ sku: 0321293533, quan: 3, unit_price: 48.0 },{ sku: 0321601912, quan: 1, unit_price: 39.0 },{ sku: 0131495054, quan: 1, unit_price: 51.0 }
],payment: { type: Amex, expiry: 04/2001,
last5: 12345 }}
![Page 31: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/31.jpg)
Relational vs Document Model
31
RelationaldatamodelHighly-structured table organization with rigidly-defined data formats and record structure.
C1 C2 C3 C4
DocumentdatamodelCollection of complex documents witharbitrary, nested data formats andvarying “record” format.
JSONJSON
JSON
{
}
![Page 32: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/32.jpg)
SQL Normalized Tables
32
Addresses
1 DEN 30303CO
2 MV 94040CA
3 CHI 60609IL
Users
KEY First ZIP_IDLast
4 NY 10010NY
1 Jasdeep 2Jaitla
2 Joe 2Smith
3 Ali 2Dodson
4 John 3Doe
ZIP_ID CITY ZIPSTATE
Togetinformationaboutspecificuser,youperformajoinacrosstwotables
foreign key
SELECT*FROMUsersuINNERJOINAddressesaONu.zip_id=a.zip_idWHEREkey=1
![Page 33: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/33.jpg)
Documents are Aggregates
33
+
Addresses
1 DEN 30303CO
2
3 CHI 60609IL
4 NY 10010NY
ZIP_ID CITY ZIPSTATE
Users
KEY First ZIP_IDLast
2
2 Joe 2Smith
3 Ali 2Dodson
4 John 3Doe
Alldatainasingledocument
{“ID”:1,“First”:“Jasdeep”,“Last”:“Jaitla”,“ZIP”:“94103”,“CITY”:“SF”,“STATE”:“CA”}
JSON
=
couchbase.get(“user::1”)
Document Data is an Aggregate
1 Jasdeep Jaitla
94103CASF
1 Jasdeep Jaitla
![Page 34: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/34.jpg)
Document Keys
Keys have to be unique in a Bucket (database) which acts as a namespace.
Keystendtohavethreecategories: human-readable, randomly generated, and hybrid
Humanreadablekeysuseuniquedescriptors:email addresses, social media id’s, phone numbers, blog post titles, usernames, sku’s, etc.
![Page 35: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/35.jpg)
www.couchbase.com/develop
PythonRuby
Go Clojure
Official SDKs
Community & Libraries
![Page 36: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/36.jpg)
Couchbase Views
![Page 37: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/37.jpg)
Indexing Architecture
33 2Managed Cache Disk Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc1Doc1
Doc1
To other node
View Engine
Doc1
Doc Updated in RAM Cache First
Indexer Updates Indexes After On Disk, in Batches
All Documents & Updates Pass Through View Engine
![Page 38: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/38.jpg)
Buckets >> Design Documents >> Views
Couchbase Bucket
Design Document 1 Design Document 2
View ViewViewViewView
Indexers Are Allocated Per Design Doc
All Updated at Same TimeAll Updated at Same TimeAll Updated at Same Time
Can Only Access Data in the Bucket NamespaceCan Only Access Data in the Bucket Namespace
![Page 39: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/39.jpg)
Map() Function => Index
function(doc, meta) {emit(doc.username, doc.email)
}indexed key output value(s)create row
json doc doc metadata
EveryDocumentpassesthroughViewMap()functions
Map
![Page 40: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/40.jpg)
Single Element Keys (Text Key)
function(doc, meta) {emit(doc.email, null)
}text key
Map
doc.email meta.id
[email protected] u::1
[email protected] u::2
[email protected] u::3
![Page 41: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/41.jpg)
Compound Keys (Array)
function(doc, meta) {emit(dateToArray(doc.timestamp), 1)
} array key
ArrayBasedIndexKeysgetsortedasStrings,butcanbegroupedbyarrayelements
Map
dateToArray(doc.timestamp) value
[2012,10,9,18,45] 1
[2012,9,26,11,15] 1
[2012,8,13,2,12] 1
![Page 42: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/42.jpg)
Reduce Values (doc.abv) with _stats
add _stats built-in reduction
![Page 43: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/43.jpg)
Result Set - Brewery ID’s by Beer
brewery_id
document key (of the beer)
alcohol by volume (abv)
![Page 44: Webinar - Einfuehrung in Couchbase](https://reader035.vdocuments.net/reader035/viewer/2022081519/555dcdb9d8b42aec698b50cb/html5/thumbnails/44.jpg)
Q & A