couchbase_tlv_2014_introduction to couchbase
TRANSCRIPT
Introduction to Couchbase Server
Ilam Siva
Senior Product Manager
Couchbase ServerNoSQL Document Database
Couchbase Open Source Project
• Leading NoSQL database project focused on distributed database technology and surrounding ecosystem
• Supports both key-value and document-oriented use cases
• All components are available under the Apache 2.0 Public License
• Obtained as packaged software in both enterprise and community editions.
Couchbase Open Source Project
In this session
• Overview of Couchbase Server features
• What’s new in Couchbase Server 2.5 (HOT OFF THE SHELF!)
• Architectural Overview and Couchbase Operations
• Live Demo with a peak into new features
Easy Scalability
Consistent High Performance
Always On
24x365
Grow cluster without application changes, without downtime with a single click
Consistent sub-millisecond read and write response times
with consistent high throughput
No downtime for software upgrades, hardware maintenance, etc.
JSONJSON
JSON
JSONJSON
Flexible Data Model
JSON document model with no fixed schema.
Couchbase Server
Core Couchbase Server Features
Built-in clustering – All nodes equal
Data replication with auto-failover
Zero-downtime maintenance
Built-in managed cached
Append-only storage layer
Online compaction
Monitoring and admin API & UI
SDK for a variety of languages
2.1 introduced New in 2.2
Multi-threaded persistence engine
Optimistic XDCR
CBHealthcheck – Cluster health check tool
Hostname management
Rebalance progress indicators
New XDCR protocol based on memcached
Read-only admin user
Automated and optimized purge management
CBRecovery Data recovery tool from remote clusters
Non-root, non-sudo install
New in 2.1 and 2.2
Coming Soon!!! Couchbase 2.5!
AVAILABILITY AND RELIABILITY
Rack awareness
Secure cross data center replication
Better client connection management
SECURITY
EASE OF ADMINISTRATION
Couchbase Server Architecture
Hea
rtb
eat
Pro
cess
mo
nit
or
Glo
bal
sin
glet
on
su
per
viso
r
Co
nfi
gura
tio
n m
anag
er
on each node
Reb
alan
ce o
rch
estr
ato
r
No
de
hea
lth
mo
nit
or
one per cluster
vBu
cket
sta
te a
nd
rep
licat
ion
man
ager
httpR
EST
man
age
me
nt
AP
I/W
eb
UI
HTTP
8091Erlang port mapper
4369Distributed Erlang
21100 - 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached
New Persistence Layer
8092Query API
Qu
ery
En
gin
e
Data Manager Cluster Manager
Couchbase Server Architecture
Replication, Rebalance, Shard State Manager
REST management API/Web UI
8091Admin Console
Erla
ng
/OTP
11210 / 11211Data access ports
Object-managedCache
Multi-threaded Persistence Engine
8092Query API
Qu
ery
En
gin
e
http
Data Manager Cluster Manager
Couchbase Operations
Auto sharding and Cluster Map
Hash function (KEY)
vB1 vB2 vB3 vB4 vB5 vB6
Ph
ysic
alse
rve
rs
A A B B C C
vB7 vB8
D D
More scalability requiredAdd node
Logi
cal
Par
titi
on
s
Cluster Map
33 2
Single node –Couchbase Write Operation
Managed Cache
Dis
k Q
ueu
e
Disk
Replication Queue
App Server
Couchbase Server Node
To other node
Doc 1
Doc 1Doc 1
33 2
Single node –Couchbase Update Operation
Managed Cache
Dis
k Q
ueu
e
Disk
Replication Queue
App Server
Couchbase Server Node
To other node
Doc 1Doc 1
Doc 1
Doc 1
Doc 1
33 2
Single node –Couchbase Read Operation
Managed Cache
Dis
k Q
ueu
e
Disk
Replication Queue
App Server
Couchbase Server Node
To other node
Doc 1
Get Doc 1
Doc 1Doc 1
Managed Cache
Dis
k Q
ueu
e
Disk
Replication Queue
Couchbase Server Node
To other node
Doc 1
Doc 1
Doc 2Doc 3Doc 4Doc 5Doc 6
Cache Ejection
App ServerDoc 2Doc 3Doc 4Doc 5Doc 6
Managed Cache
Dis
k Q
ueu
e
Disk
Replication Queue
Couchbase Server Node
To other node
Doc 1
Doc 2Doc 3Doc 4Doc 5Doc 6
Cache Miss
App Server
Doc 2Doc 3Doc 4Doc 5Doc 6
Get Doc 1
Doc 1
Doc 1Doc 1
Couchbase Server Cluster
Basic Operation
User Configured Replica Count = 1
Read/write/update
Active
SERVER 1
Active
SERVER 2
Active
SERVER 3
App Server 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
App Server 2
Doc 5
Doc 2
Doc 9
Doc
Doc
Doc
Doc 4
Doc 7
Doc 8
Doc
Doc
Doc
Doc 1
Doc 3
Doc 6
Doc
Doc
Doc
Replica Replica Replica
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Doc 6
Doc 3
Doc 2
Doc
Doc
Doc
Doc 7
Doc 9
Doc 5
Doc
Doc
Doc
• Docs distributed evenly across servers
• Each server stores both active and replica docs Only one server active at a time
• Client library provides app with simple interface to database
• Cluster map provides map to which server doc is on App never needs to know
• App reads, writes, updates docs
• Multiple app servers can access same document at same time
Add Nodes to Cluster
SERVER 4 SERVER 5
Replica
Active
Replica
Active
Read/write/update
App Server 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
App Server 2
User Configured Replica Count = 1
Couchbase Server Cluster
Active
SERVER 1
Doc 5
Doc 2
Doc 9
Doc
Doc
Doc
Replica
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Active
SERVER 2
Doc 4
Doc 7
Doc 8
Doc
Doc
Doc
Replica
Doc 6
Doc 3
Doc 2
Doc
Doc
Doc
Active
SERVER 3
Doc 1
Doc 3
Doc 6
Doc
Doc
Doc
Replica
Doc 7
Doc 9
Doc 5
Doc
Doc
Doc
Read/write/update
• Two servers added withone-click operation
• Docs automatically rebalance across cluster Even distribution of docs Minimum doc movement
• Cluster map updated
• App database calls now distributed over larger number of servers
Fail Over Node
User Configured Replica Count = 1
SERVER 4 SERVER 5
Replica
Active
Replica
Active
App Server 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
App Server 2
Couchbase Server Cluster
Active
SERVER 1
Doc 5
Doc 2
Doc 9Doc
Doc
Doc
Replica
Doc 4
Doc 1
Doc 8Doc
Doc
Doc
Active
SERVER 2
Doc 4
Doc 7 Doc 8
Doc
Doc Doc
Replica
Doc 6
Doc 3 Doc 2
Doc
Doc Doc
Active
SERVER 3
Doc 1
Doc 3
Doc 6Doc
Doc
Doc
Replica
Doc 7
Doc 9
Doc 5Doc
Doc
Doc
• App servers accessing docs
• Requests to Server 3 fail
• Cluster detects server failed– Promotes replicas of docs
to active– Updates cluster map
• Requests for docs now go to appropriate server
• Typically rebalance would follow
Doc 1 Doc 3
Doc
Demo Time
Indexing and Querying – The basics
• Define materialized views on JSON documents and then query across the data set
• Using views you can define• Primary indexes
• Simple secondary indexes (most common use case)
• Complex secondary, tertiary and composite indexes
• Aggregations (reduction)
• Indexes are eventually indexed
• Queries are eventually consistent
• Built using Map/Reduce technology • Map and Reduce functions are written in Javascript
COUCHBASE SERVER CLUSTER
Indexing and Querying
User Configured Replica Count = 1
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 1
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
Doc 9
• Indexing work is distributed amongst nodes
• Large data set possible
• Parallelize the effort
• Each node has index for data stored on it
• Queries combine the results from required nodes
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 2
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Doc 9
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 3
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Doc 9
Query
Cross Data Center Replication – The basics
• Replicate your Couchbase data across clusters
• Clusters may be spread across geos
• Configured on a per-bucket (per-database) basis
• Supports unidirectional and bidirectional operation
• Application can read and write from both clusters
Active – Active replication
• Replication throughput scales out linearly
• Different from intra-cluster replication
33 2
Cross data center replication – Data flow2
Managed Cache
Dis
k Q
ueu
e
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
XDCR Engine
Doc 1
To other cluster
SERVER 3SERVER 1 SERVER 2
Couchbase Server – San Francisco
SERVER 3SERVER 1 SERVER 2
Couchbase Server – New York
Optimistic replication
Per replication Tunable Parameters
Optimized protocolbased on memcached
Reliability and performance at scale
Cross Data Center Replication (XDCR)
Demo Time
A brief look into 2.5!
Rack awareness in Couchbase Server
Secure Cross Data Center Replication
Why is secure cross data center replication important ?
• More and more sensitive data is getting stored in NoSQL databases
• Certain industries have security and compliance regulations to follow
Healthcare (HIPAA)
Financial services (PCI)
Public education (FERPA)
• Keep sensitive information across the internet encrypted so that only the intended recipient can understand it.
A look into the future!
Couchbase Query Language
N1QL
Our next generation query language for JSON
Read “Nickel”
Learn more about N1QL at query.couchbase.com
In Dev Preview
www.couchbase.com/download
Couchbase Server
