NoSQL Databases

Cloudant & Redis

Nikolay TomitovTechnical TrainerSoftAcad Training Center

www.softacad.bg

Schema-free Databases

Documental Databases Relational Databases

JSON Data Format

www.softacad.bg

Meet Cloudant

www.softacad.bg

Cloudant == managed installation of Apache CouchDB in the cloud

Cloudant scales within & between data centers◦ Global networks of servers◦ Built-in replication and sync◦ Push the data closer to the user & application◦ Disaster recovery

Global Data Network

www.softacad.bg

Eliminates data layer-related complexity, delay, cost, and risk

Performance & scale with no worries Build epic apps, easily

◦ Enables advanced app features and analytics◦ Full-text search, off-line computing, replication,

incremental MapReduce, code and data◦ distribution...◦ Accelerates release cycles (reduces time to

value) Start for free, pay as you grow

Nice Features

www.softacad.bg

www.softacad.bg

Provision Signup is free, and access is immediate

Define No schema, usually just JSON Access RESTful API View Define data views via Javascript Search Integrated, Lucene-like full-text search Analyze Incremental, chainable MapReduce Operate Dashboards to monitor, compact,

replicate Grow Automatic

Developingwith Cloudant

Data is stored and returned in JSON format

Queried via HTTP RESTful API

Index building language: Javascript

Simple and intuitive interface

CouchDB Inside

www.softacad.bg

• Create:HTTP PUT /db/test

• Read:HTTP GET /db/test

• Update:HTTP PUT /db/test

• Delete:HTTP DELETE /db/test

REST Interface – DEMO

www.softacad.bg

Views: the way to arrange data to answer our questions

Method to build views:Incremental MapReduce using

JavaScript

Querying the DB

www.softacad.bg

DEMO – Simple MapReduce

www.softacad.bg

Example

www.softacad.bg

How to Findsum(checkouts) ?

www.softacad.bg

Simple Map: emit(checkout)

www.softacad.bg

Simple Reduce: sum(checkouts)

www.softacad.bg

Simple Reduce: sum(checkouts)

www.softacad.bg

Views can be Distributed

www.softacad.bg

Practice Map Reduce at:

http://blog.mudynamics.com/wp-content/uploads/2009/04/icouch.html

http://couchdb.apache.org/downloads.html

Practice

www.softacad.bg

Meet Redis

“Redis is an open source, advanced key-value storeIt is often referred to as a data structure server Since keys can contain strings, hashes, lists, sets and sorted sets

www.softacad.bg

www.softacad.bg

Features First It’s really fast

◦Non-blocking I/O, single threaded◦100,000+ read/writes a second

It complements yourexisting storage layer

It is very feasible for small apps

But..trade performance for durabilitywww.softacad.bg

www.softacad.bg

Short keys perform better

Could not contain whitespaces

Common convention is : obj-type:id:field◦user:77:pass = abcd

SHA1(data) makes a useful key too

Keys

www.softacad.bg

strings (up to 1GB) Lists of strings Sets of strings Sorted sets Hashes Pubsub channels

Supported Data Types

www.softacad.bg

EXISTS[key] DEL[key] TYPE[key] RENAMENX[key] EXPIRE [key] EXPIREAT[key] TTL[key]

Commands on All Keys

www.softacad.bg

SET[key value]

MGET[key1 key2 key3..]

GET [key]

INCR/DECR [key]

INCRBY/DECRBY[key]

Commands on Strings

www.softacad.bg

Store session data + creation time

Hit counts◦ key =MD5(url)◦ Incr hits-by-key:$key

APPEND can be used for logging

Redis as shared memory location

Progress bars, polling

Common Uses

www.softacad.bg

More durability -> less performance

Default behaviour fork-and-save a snapshot to disk every…◦15 mins if one key has changed◦5 mins if 10 keys have changed◦60 secs if 10000 keys have changed

SAVE/BGSAVE commands trigger a save

Durability

www.softacad.bg

DEMO (JAVA)

Try it :◦ http://try.redis-db.com

Redis Admin UI◦ http://www.servicestack.net/mythz_blog/?p=381

Check It Out

www.softacad.bg

Amazon SimpleDB

www.softacad.bg

Non–relational data store Designed to be highly available and flexible

Available only on Amazon WS

SimpleDB: Overview

www.softacad.bg

Low touch – almost no administration needed

Highly available – automatic replication and fail over

Flexible – not bound by a schema Simple to use – supports only the most

common operations Designed for integration with other

Amazon Web Services Secure RESTful interface

SimpleDB: Functionality

www.softacad.bg

Free tier – 1 GB/month, 25 SimpleDB machine hours

Only outgoing transfers are charged Data transfers in the same region are free

Monthly rates between $0.12 and $0.05

SimpleDB: Pricing

www.softacad.bg

Optimized reads and writes Easily integrated with other AWS Uses the HTTP protocol for communication

Ideal for storing non – structured or semi – structured data

SimpleDB: Pros

www.softacad.bg

Limited functionality compared to relational and other non – relational products

Complex queries need a lot of coding No fulltext indexing All queries have the N + 1 problem

SimpleDB: Cons

DemoAmazon Simple DB

Google Datastore (GDS)

www.softacad.bg

Schemaless object datastore Designed to scale Hierarchical Available only on Google App Engine

GDS: Overview

www.softacad.bg

No administration needed Flexible – not bound by a schema Designed for no downtime Automatic replication Queries must run against predefined

indexes Rich query engine Atomic transactions APIs available for all the languages

supported by GAE

GDS: Functionality

www.softacad.bg

Free quota: 1GB, 200 indexes Monthly rates: $0.24/gigabyte; between $0.01 and $0.10 for 100k database operations

GDS: Pricing

www.softacad.bg

Fast Hierarchical queries boost speed Atomic transactions supported Large and active community The Java API include implementations of JDO and JPA

GDS: Pros

www.softacad.bg

Limited functionality◦Compared to relational databases and

other non–relational data stores Indexes require additional configuration

GDS: Cons

DemoGoogle App Engine Datastore

MongoDB

www.softacad.bg

Non – relational, schema free database

Highly scalable Engineered for maximum speed Available as a stand alone product A number of hosted MongoDB solutions available◦MongoLab◦MongoHQ◦Mongo Machine

Overview

www.softacad.bg

Almost administration free High availability – replication and sharding A rich set of features

◦ Complex queries◦ Aggregate functions◦ Map/Reduce operations◦ Indexes◦ Stored procedures◦ BLOB storage (GridFS)

Database drivers available for a large list of programming languages

MongoDB: Functionality

MongoDB:Hosted Solutions

MongoLab MongoHQ

Easy setup (No billing info needed)

Intuitive interface 240MB free Deploy on Amazon

EC2, Rackspace or Joynet

MongoDB Version 1.8

Easy setup (Credit card needed)

Intuitive interface 16MB free Deploy only on

Amazon EC2 Different MongoDB

Version according to price plan

www.softacad.bg

Fast Rich feature set Mature product Large and active community

MongoDB: Pros

www.softacad.bg

No relational database style transactions

No data validation

MongoDB: Cons

DemoMongoDB on MongoLab

www.softacad.bg

1. Register for a free Cloudant account at https://cloudant.com.

2. Write a simple "Dictionary" application in Java, C#, PHP, or JavaScript to perform the following in Cloudant:

◦ Add a dictionary entry (word + translation)◦ List all words and their translations◦ Find the translation of given wordThe UI of the application is up to you (it could be Web-based, GUI or console-based).You may download a Cloudant client library from http://wiki.apache.org/couchdb/Related_Projects

Homework

www.softacad.bg

3. Implement the previous task ("Dictionary") with Redis

◦ Register for a free "Redis To Go" account at: https://redistogo.com

◦ Download the client libraries for your favorite programming language from http://redis.io/clients

4. Implement the previous task ("Dictionary") with MongoDB (register at MongoLab or MongoHQ)

5. Optionally (if you want) implement the previous task ("Dictionary") with Google Datastore or Amazon SimpleDB

Homework (2)