developing for node.js with mysql and nosql

Insert Picture Here

Developing for Node.JS with MySQL and NoSQL

J.D. Duncan, john.duncan@oracle.com Craig Russell, craig.russell@oracle.com

MySQL Cluster Architecture

MySQL Cluster Data Nodes

Data Layer

Clients

Application Layer

Management

Data Layer

Clients

Application Layer

ManagementManagement

Data Layer

Application Layer

ManagementManagement

Clients

mysql-js: a MySQL connector for Node.JS

Included in MySQL Cluster 7.3 in storage/ndb/nodejs

Also at github: https://github.com/mysql/mysql-js

One JavaScript API with two back-end adapters The MySQL back end uses node-mysql (Felix Geisendorfer's all-

JavaScript MySQL connector) The NDB (MySQL Cluster) back end uses native NDBAPI

Twitter-like demo application in samples/tweet/

What is Node.JS

A server-side web application platform Single-threaded* and event-driven Application code is written in JavaScript JavaScript code runs in a very fast VM Node.JS includes a useful set of standard libraries

* One JavaScript thread; a pool of background worker threads perform blocking I/O.

Why Node.JS

Servers like lighttpd, nginx, and tornado can solve the C10K problem for static or cached content

Node.JS can solve the C10K problem for a dynamic web application– ... but what about 10,000 back-end database connections?

mysql-js with NDB aims to solve the C10K problem for a dynamic and database-driven web application

The "C10K" problem: how to handle 10,000+ client connections

Sample CodeNode HTTP Server

function handleRequest(request, response) { var page = getContent(request.url); response.statusCode = page ? 200 : 404; response.write(page); response.end();}

function startWebServer() { http.createServer(handleRequest).listen(8080); console.log("Server started");}

Sample Code with database accessNode HTTP Server and mysql-jsfunction dbConnected(connectionError, sessionFactory) { function handleRequest(request, response) { function onDbResponse(err, data) { var page = renderPage(data); response.write(page); response.end(); } function onDbSession(err, session) { runDbOperation(session, request, onDbResponse); } sessionFactory.openSession(null, onDbSession); } http.createServer(handleRequest).listen(8080);}nosql.connect(... , dbConnected);

User's View

Twitter-like Demo Application

Create user Find user by name Delete user by name

(cascading) Create a tweet Find tweet by id Delete tweet by id (cascading) Make User A a follower of

User B List followers of user

List who user is following Get the most recent tweets Get the 20 latest tweets by a

user Get the 20 latest tweets

@mentioning a user Get the 20 latest tweets

containing a #hashtag

JavaScript Constructor

function Tweet(author, message) { if(author !== undefined) { this.date_created = new Date(); this.author = author; this.message = message; }}

Creates an instance of a Domain Object

SQL Table

CREATE TABLE tweet ( id bigint unsigned auto_increment not null, author varchar(20), message varchar(140), date_created timestamp(2), ...)

Mapping

var nosql = require("../mysql-js");

// Map SQL Tables to JS Constructorsvar mappings = [];mappings.push( new nosql.TableMapping('tweet'). applyToClass(Tweet));

Explicit Mappings are not required

All operations take either a constructor or table name If table name is used, default mapping is created Default mapping maps all columns to JS properties, where the JS type

of each property is the default for its corresponding SQL Data Type

connect function

connect is asynchronous First parameter is object with connection properties Second parameter is array of table names or constructors

– Metadata for these will be resolved before returning session factory Third parameter is callback (err, sessionFactory) err will be falsy if no error

connect Example/* Connection Properties */dbProperties = { adapter : "ndb", ndb_connectstring : "localhost:1186" };

// Map SQL Tables to JS Constructorsmappings = [];mappings.push(new nosql.TableMapping('tweet'). applyToClass(Tweet));// Connectnosql.connect(dbProperties, mappings, runCmdlineOperation, operation);

The connect callbackfunction dbConnected(connectionError, sessionFactory) { function handleRequest(request, response) { function onDbResponse(err, data) { var page = renderPage(data); response.write(page); response.end(); } function onDbSession(err, session) { runDbOperation(session, request, onDbResponse); } sessionFactory.openSession(null, onDbSession); } http.createServer(handleRequest).listen(8080);}nosql.connect(dbProperties, mappings, dbConnected);

find Example

function findAuthor(name, callback) { session.find(Author, name, function(err, result) { if (err) { callback(err); } else { // result is the row in the author table callback(null, result); });}

find function on Session

find is asynchronous First parameter is table name or constructor (mapped class) Second parameter is key to use locating the record

– Primary key– Unique key

Third parameter is callback (err, result)– err will be falsy if no error– result will be new object with mapped properties

persist Example

function persistAuthor(name, full, callback) {

var author = new Author(name, full);

session.persist(author, function(err) { if (err) {

callback(err);

} else {

callback(null, author);

persist function on Session

persist is asynchronous First parameter is instance of mapped class Second parameter callback (err)

– err will be falsy if no error Variants allow persist into default mapped table (no constructor)

update Example

function updateAuthor(author, new_full_name, callback) { author.full_name = new_full_name; session.update(author, callback); }

update function on Session

update is asynchronous First parameter is instance of mapped class

– Modified properties are written to database Second parameter callback (err)

– err will be falsy if no error Variants allow update of default mapped table (no constructor)

remove Example

function removeByAuthorName(name, callback) { session.remove(Author, name, function(err, result) {

if (err) { callback(err); } else { // row has been removed callback(null); });}

remove function on Session

remove is asynchronous– First parameter is table name or constructor (mapped class)– Second parameter is key to use locating the record

Primary key Unique key

– Third parameter is callback (err) err will be falsy if no error

Alternately– First parameter is instance of constructor– Second parameter is callback (err)

Transactions

/* Insert a tweet. - Start a transaction. - Persist the tweet. - Create & persist #hashtag & @user records - Increment the author's tweet count. - Then commit the transaction. */function InsertTweetOperation(params, data) { session.currentTransaction().begin(); ... session.currentTransaction().commit();}

Query Example// Last 20 tweets @user

TweetsAtUserOperation.run = function(tag) { if(tag.charAt(0) == "@") tag = tag.substring(1); this.session.createQuery(Mention, function(error, query) { var queryParams = { "order" : "desc", "limit" : 20 , "tag" : tag}; query.where(query.at_user.eq(query.param("tag"))); query.execute(queryParams, fetchTweetsInBatch); });};

query function

Session is Query factory using asynchronous api query has a filter based on constructor (mapped class) properties Filter has comparators and boolean operations

– eq, ne, gt, ge, le, lt, between,isNull,isNotNull comparators– and, or, not,andNot,orNot operations

Query execution is asynchronous– Filter determines query strategy

Primary/unique key lookup; index scan; table scan– Properties govern query execution– Results are given in callback

Query Comparators

Comparators compare properties to parameters Query Domain Type property names correspond to Constructor field

names (properties) Parameters are created by name

– qdt.param('date_low')

Properties are referenced by field name in Constructor– qdt.date_created

Comparators are properties of qdt properties– qdt.date_created.gt(qdt.param('date_low'));

Comparators return predicates

Query Operators

Predicates are used as query filters via where function Predicates are results of comparators or operators Operators combine predicates using and, or, andNot, orNot, not

– var predicate1 = qdt.date_created.gt(qdt.param('date_low'));

– var predicate2 = qdt.date_created.lt(qdt.param('date_high'));

– var predicate = predicate1.and(predicate2);

– var predicate = predicate3.andNot(predicate4);

– qdt.where(predicate)

Batching

function fetchTweetsInBatch(error, mentions, finalCallback) { var resultData = []; function addTweetToResults(err, tweet) { if(tweet && ! err) resultData.push(tweet); } var batch = session.createBatch(); while(var mention = mentions.pop()); batch.find(Tweet, mention.id, addTweetToResults);

batch.execute(finalCallback, resultData); }

Batching

Batching allows parallel database operations– Any combination of find, persist, update, remove

Batch is created using session.createBatch() Operations (with their own callbacks) are added to the batch Batch is executed with its own callback

– Operation callbacks are run first– Then the batch callback

Get mysql-js & run the Tweet demo### First install MySQL Cluster 7.3

### Then get the latest code from github:% git clone https://github.com/mysql/mysql-js.git

### build it% node configure.js% node-gyp configure build

### Go to the demo directory% cd samples/tweet

Get mysql-js & run the Tweet demo### create the schema% mysql -u root < tweet.sql

### Run with the mysql adapter% node tweet.js -a mysql put user mr_jdd \ 'John David Duncan'

### Run with the ndb adapter% node tweet.js -a ndb get user mr_jdd

### Look at some more examples% cat test_tweet.sh

developing for node.js with mysql and nosql

Technology

mysql & nosql: the best of both...

mysql & nosql from a php perspective

full stack development with node.js and nosql (nic raboy &...

mysql vs nosql

mysqlのnosql機能 - mysql json & http plugin for mysql

nosql and mysql webinar - best of both worlds

mysql nosql opensource and beyond

simple way for mysql to nosql

your guide to mysql (no "nosql" stuff)

table of contents · mysql cluster 7.3 ea: node.js nosql...

mysql user camp : 20-june-14 : mysql new features and nosql...

mysql nosql apis

mysql roadmap nosql ha fev17

mysql 5.7 - nosql Возможности nosql, json,...

mysql: combining sql and nosql · mysql: combining sql and...

nosql атакует: json функции в mysql...

codefest 2013. Аверин С. —...

node.js + nosql

oracle nosql database node.js driver getting started with...

oracle nosql database node.js driver getting started · pdf...