introduction to couchbase

Introduction to Couchbase Server

Dipti Borkar Director, Product Management

Anil KumarProduct Management

Couchbase ServerNoSQL Document Database

Couchbase Open Source Project

• Leading NoSQL database project focused on distributed database technology and surrounding ecosystem

• Supports both key-value and document-oriented use cases

• All components are available under the Apache 2.0 Public License

• Obtained as packaged software in both enterprise and community editions.

Couchbase Open Source Project

Easy Scalabili

ty

Consistent High

Performance

Always On

24x365

Grow cluster without application changes, without downtime with a single click

Consistent sub-millisecond read and write response times

with consistent high throughput

No downtime for software upgrades, hardware maintenance, etc.

JSONJSONJSON

JSONJSON

PERFORMANCE

Flexible Data Model

JSON document model with no fixed schema.

Couchbase Server

Core Couchbase Server Features

Built-in clustering – All nodes equal

Data replication with auto-failover

Zero-downtime maintenance

Built-in managed cached

Append-only storage layer

Online compaction

Monitoring and admin API & UI

SDK for a variety of languages

2.0 introduced

JSON support Indexing and Querying

Cross data center replicationIncremental Map Reduce

JSONJSONJSON

JSONJSON

2.1 introduced New in 2.2

Multi-threaded persistence engine

Optimistic XDCR

CBHealthcheck – Cluster health check tool

Hostname management

Rebalance progress indicators

New XDCR protocol based on memcached

Read-only admin user

Automated and optimized purge management

CBRecovery Data recovery tool from remote clusters

Non-root, non-sudo install

Couchbase Server Architecture

Hea

rtbe

at

Proc

ess

mon

itor

Glo

bal s

ingl

eton

sup

ervi

sor

Confi

gura

tion

man

ager

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpRE

ST m

anag

emen

t API

/Web

UI

HTTP8091

Erlang port mapper4369

Distributed Erlang21100 - 21199

Erlang/OTP

storage interface

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

Memcached

New Persistence Layer

8092Query API

Que

ry E

ngin

e

Data Manager Cluster Manager

Couchbase Server Architecture

Replication, Rebalance, Shard State Manager

REST management API/Web UI

8091Admin Console

Erla

ng /

OTP

11210 / 11211Data access ports

Object-managedCache

Multi-threaded Persistence Engine

8092Query API

Que

ry E

ngin

e

http

Data Manager Cluster Manager

Couchbase Operations

33 2

Single node - Couchbase Write Operation

Managed Cache

Dis

k Q

ueue

Disk

Replication Queue

App Server

Couchbase Server Node

Doc 1Doc 1

Doc 1

To other node

33 2

Single node - Couchbase Update Operation

Managed Cache

Dis

k Q

ueue

Replication Queue

App Server

Doc 1’

Doc 1

Doc 1’Doc 1

Doc 1’

Disk

To other node


GET

Doc

1

33 2

Single node - Couchbase Read Operation

Dis

k Q

ueue

Replication Queue

App Server

Doc 1

Doc 1Doc 1

Managed Cache

Disk

To other node


33 2

Single node – Couchbase Cache Miss2

Dis

k Q

ueue

Replication Queue

App Server


Doc 1

Doc 3Doc 5 Doc 2Doc 4

Doc 6 Doc 5 Doc 4 Doc 3 Doc 2

Doc 4

GET

Doc

1

Doc 1

Doc 1

Managed Cache

Disk

To other node

COUCHBASE SERVER CLUSTER

Basic Operation

• Docs distributed evenly across servers

• Each server stores both active and replica docsOnly one server active at a time

• Client library provides app with simple interface to database

• Cluster map provides map to which server doc is onApp never needs to know

• App reads, writes, updates docs

• Multiple app servers can access same document at same time

User Configured Replica Count = 1

READ/WRITE/UPDATE

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 1

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc

SERVER 2

Doc 8

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

REPLICA

Doc 6

Doc 3

Doc 2

Doc

Doc

Doc

REPLICA

Doc 7

Doc 9

Doc 5

Doc

Doc

Doc

SERVER 3

Doc 6

APP SERVER 1

COUCHBASE Client Library

CLUSTER MAP


CLUSTER MAP

APP SERVER 2

Doc 9

Add Nodes to Cluster

• Two servers addedOne-click operation

• Docs automatically rebalanced across clusterEven distribution of docsMinimum doc movement

• Cluster map updated

• App database calls now distributed over larger number of servers

REPLICA

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc 4

Doc 1

Doc

Doc

SERVER 1

REPLICA

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc 6

Doc 3

Doc

Doc

SERVER 2

REPLICA

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc 7

Doc 9

Doc

Doc

SERVER 3 SERVER 4 SERVER 5

REPLICA

ACTIVE

REPLICA

ACTIVE

Doc

Doc 8 Doc

Doc 9 Doc

Doc 2 Doc

Doc 8 Doc

Doc 5 Doc

Doc 6

READ/WRITE/UPDATE READ/WRITE/UPDATE

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2



Fail Over Node

REPLICA

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc 4

Doc 1

Doc

Doc

SERVER 1

REPLICA

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc 6

Doc 3

Doc

Doc

SERVER 2

REPLICA

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc 7

Doc 9

Doc

Doc

SERVER 3 SERVER 4 SERVER 5

REPLICA

ACTIVE

REPLICA

ACTIVE

Doc 9

Doc 8

Doc Doc 6 Doc

Doc

Doc 5 Doc

Doc 2

Doc 8 Doc

Doc

• App servers accessing docs

• Requests to Server 3 fail

• Cluster detects server failedPromotes replicas of docs to activeUpdates cluster map

• Requests for docs now go to appropriate server

• Typically rebalance would follow

Doc

Doc 1 Doc 3

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2



Demo Time

Indexing and Querying – The basics

• Define materialized views on JSON documents and then query across the data set

• Using views you can define• Primary indexes • Simple secondary indexes (most common use case)• Complex secondary, tertiary and composite indexes• Aggregations (reduction)

• Indexes are eventually indexed

• Queries are eventually consistent

• Built using Map/Reduce technology • Map and Reduce functions are written in Javascript


Indexing and Querying


ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 1

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2

Doc 9

• Indexing work is distributed amongst nodes

• Large data set possible

• Parallelize the effort

• Each node has index for data stored on it

• Queries combine the results from required nodes

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 2

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

Doc 9

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 3

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

Doc 9

Query

Cross Data Center Replication – The basics

• Replicate your Couchbase data across clusters

• Clusters may be spread across geos

• Configured on a per-bucket (per-database) basis

• Supports unidirectional and bidirectional operation

• Application can read and write from both clusters Active – Active replication

• Replication throughput scales out linearly

• Different from intra-cluster replication

33 2

Cross data center replication – Data flow

2

Managed Cache

Dis

k Q

ueue

Disk

Replication Queue

App Server


Doc 1Doc 1

Doc 1

To other node

XDCR Engine

Doc 1

To other cluster

SERVER 3SERVER 1 SERVER 2

Couchbase Server – San Francisco

SERVER 3SERVER 1 SERVER 2

Couchbase Server – New York

Optimistic replication

Per replication Tunable Parameters

Optimized protocolbased on memcached

Reliability and performance at scale

Cross Data Center Replication (XDCR)

Demo Time

What’s else is New?

Couchbase Query Language

N1QL

Our next generation query language for JSON

Read “Nickel”

In Dev Preview

www.couchbase.com/download

Couchbase Server

Thank [email protected]

@anilkumar1129

Download Couchbase Server 2.2 http://www.couchbase.com/download

introduction to couchbase

Technology