intro to node js

I N T R O T O N O D E . J S- L E V E L O N E -

INTRO TO NODE.JS

WHAT IS NODE.JS?

It’s fast because it’s mostly C code

Allows you to build scalable network applications using JavaScript on the server-side.

V8 JavaScript Runtime

Node.js

INTRO TO NODE.JS

WHAT COULD YOU BUILD?• Websocket Server• Fast File Upload Client• Ad Server• Any Real-Time Data Apps

Like a chat server

INTRO TO NODE.JS

WHAT IS NODE.JS NOT ?• A Web Framework• For Beginners It’s very low level• Multi-threaded

You can think of it as a single threaded server

INTRO TO NODE.JS

OBJECTIVE: PRINT FILE CONTENTS

This is a “Callback”

Read file from Filesystem, set equal to “contents”Print contents

• Blocking Code

• Non-Blocking Code

Do something else

Read file from Filesystemwhenever you’re complete, print the contents

Do Something else

console.log(contents);

INTRO TO NODE.JS

BLOCKING VS NON-BLOCKING

var contents = fs.readFileSync('/etc/hosts');console.log(contents);

console.log('Doing something else');

• Blocking Code

• Non-Blocking Code

console.log('Doing something else');Stop process unt

il complete

fs.readFile('/etc/hosts', function(err, contents) {

});

fs.readFile('/etc/hosts', function(err, contents) {console.log(contents);

});

INTRO TO NODE.JS

CALLBACK ALTERNATE SYNTAX

var callback = function(err, contents) { console.log(contents);}

fs.readFile('/etc/hosts', callback);

Same as

INTRO TO NODE.JS

BLOCKING VS NON-BLOCKING

blocking

0snon-blocking

10s5s

0s 10s5s

fs.readFile('/etc/hosts', callback);fs.readFile('/etc/inetcfg', callback);

var callback = function(err, contents) { console.log(contents);}

hello.js

NODE.JS HELLO DOG

$ curl http://localhost:8080

Hello, this is dog.

How we require modules

Status code in headerResponse body

Close the connectionListen for connections on this port

$ node hello.js Run the server

var http = require('http');

http.createServer(function(request, response) {response.writeHead(200);response.write("Hello, this is dog.");response.end();

}).listen(8080);

console.log('Listening on port 8080...');

Listening on port 8080...

THE EVENT LOOPvar http = require('http');http.createServer(function(request, response) {

}).listen(8080);console.log('Listening on port 8080...');

Starts the Event Loop when finished

...

Known Eventsrequest

Checkingfor

Events

Run the Callback

INTRO TO NODE.JS

WHY JAVASCRIPT ?“JavaScript has certain characteristics that make it very different than other dynamic languages, namely that it has no concept of threads. Its model of concurrency is completely based around events.” - Ryan Dahl

THE EVENT LOOPKnown Events

requestCheckingfor

Eventsconnection

close

Event Queue

closerequest

Events processed one at a time

INTRO TO NODE.JS

WITH LONG RUNNING PROCESS

Represent long running process


http.createServer(function(request, response) {

}).listen(8080);

response.writeHead(200);

response.write("Dog is done.");response.end();

setTimeout(function(){

}, 5000); 5000ms = 5 seconds

response.write("Dog is running.");

INTRO TO NODE.JS

TWO CALLBACKS HEREvar http = require('http');

http.createServer(function(request, response) {response.writeHead(200);

request

timeout

}).listen(8080);



}, 5000);


TWO CALLBACKS TIMELINE

0s 10s5s

Request comes in, triggers request event

Request Callback executessetTimeout registered

Request comes in, triggers request event

Request Callback executessetTimeout registered

triggers setTimeout eventsetTimeout Callback executes

triggers setTimeout eventsetTimeout Callback

request

timeout

WITH BLOCKING TIMELINE

0s 10s5s

Request comes in, triggers request event Request Callback executes

setTimeout executed

Request comes in, waits for server

Request comes in

triggers setTimeout eventsetTimeout Callback executedWasted Time

Request Callback executes

INTRO TO NODE.JS

• Calls out to web services

TYPICAL BLOCKING THINGS

• Reads/Writes on the Database• Calls to extensions

E V E N T S- L E V E L T W O -

E V E N T S

EVENTS IN THE DOM

The DOM

The DOM triggers Events

click

events

you can listen for those events

submit

hover

When ‘click’ event is triggered

attach

$("p").on("click", function(){ ... });

E V E N T S

EVENTS IN NODE

EventEmitter

Many objects in Node emit events

net.Serverrequestevent

EventEmitterfs.readStream

dataevent

E V E N T S

CUSTOM EVENT EMITTERS

var logger = new EventEmitter();

logger.emit('error', 'Spilled Milk');

ERR: Spilled Milk

logger.emit('error', 'Eggs Cracked');

var EventEmitter = require('events').EventEmitter;

error warn info

listen for error eventlogger.on('error', function(message){

console.log('ERR: ' + message);});

ERR: Eggs Cracked

events

E V E N T S

EVENTS IN NODE

EventEmitter

Many objects in Node emit eventsnet.Server

requestevent

When ‘request’ event is emitted

function(request, response){ .. }

emit

attach

E V E N T S

HTTP ECHO SERVERhttp.createServer(function(request, response){ ... });

But what is really going on here?http://nodejs.org/api/

E V E N T S

BREAKING IT DOWNhttp.createServer(function(request, response){ ... });

http.createServer(function(request, response){ ... });

E V E N T S

ALTERNATE SYNTAX

var server = http.createServer();function(request, response){ ... });server.on('request',

This is how we add

Same as

function(){ ... });server.on('close',

add event listeners

S T R E A M S- L E V E L T H R E E -

S T R E A M S

WHAT ARE STREAMS?

Start ProcessingImmediately

Streams can be readable, writeable, or both

S T R E A M S

STREAMING RESPONSE

Our clients receive "Dog is running."

"Dog is done."

(5 seconds later)


}).listen(8080);




}, 5000);


readable stream writable stream

S T R E A M S

HOW TO READ FROM THE REQUEST ?

EventEmitterReadable Stream data

eventsemit

Lets print what we receive from the request.http.createServer(function(request, response) {

request.on('data', function(chunk) {console.log(chunk.toString());

});

response.end();

}).listen(8080)

request.on('end', function() {

});


end

S T R E A M S

LETS CREATE AN ECHO SERVERhttp.createServer(function(request, response) {

request.on('data', function(chunk) {

});

response.end();

}).listen(8080)

request.on('end', function() {

});


response.write(chunk); request.pipe(response);

S T R E A M S

LETS CREATE AN ECHO SERVER!http.createServer(function(request, response) {

}).listen(8080)request.pipe(response);

$ curl -d 'hello' http://localhost:8080

Hello on client


cat 'bleh.txt' | grep 'something'Kinda like on the command line

S T R E A M S

READING AND WRITING A FILEvar fs = require('fs');

var file = fs.createReadStream("readme.md");var newFile = fs.createWriteStream("readme_copy.md");

require filesystem module

file.pipe(newFile);

S T R E A M S

UPLOAD A FILEvar fs = require('fs');

var newFile = fs.createWriteStream("readme_copy.md");



request.pipe(newFile);

}).listen(8080);

$ curl --upload-file readme.md http://localhost:8080

uploaded!

response.end('uploaded!');request.on('end', function() {

});

THE AWESOME STREAMING

client storage

server

original filetransferred file

non-blocking

0s 10s5s

BACK PRESSURE!

client storage

server

original filetransferred file

Writable stream slowerthan readable stream

Using pipe solves this problem

THINK OF A MILK JUGmilkStream.pause();

milkStream.resume();});

Once milk jug is drained

PIPE SOLVES BACKPRESSURE

readStream.resume();});

Pause when writeStream is full

writeStream.on('drain', function(){

readStream.on('data', function(chunk) {writeStream.write(chunk);

});

var buffer_good =if (!buffer_good) readStream.pause(); returns false

if kernel buffer fullResume when ready to write again

readStream.pipe(writeStream);All encapsulated in

S T R E A M S

FILE UPLOADING PROGRESS

S T R E A M S

FILE UPLOADING PROGRESS$ curl --upload-file file.jpg http://localhost:8080

progress: 3%progress: 6%progress: 9%progress: 12%progress: 13%...progress: 99%progress: 100%

Outputs:

• HTTP Server• File System

We’re going to need:

S T R E A M S

DOCUMENTATION http://nodejs.org/api/

Stability Scores

S T R E A M S

REMEMBER THIS CODE?var fs = require('fs');

var newFile = fs.createWriteStream("readme_copy.md");




}).listen(8080);

response.end('uploaded!');request.on('end', function() {

});

S T R E A M S

REMEMBER THIS CODE?var newFile = fs.createWriteStream("readme_copy.md");



}).listen(8080);...

request.on('data', function(chunk) {uploadedBytes += chunk.length;var progress = (uploadedBytes / fileBytes) * 100;response.write("progress: " + parseInt(progress, 10) + "%\n");

});

var uploadedBytes = 0;var fileBytes = request.headers['content-length'];

S T R E A M S

SHOWING PROGRESS

M O D U L E S- L E V E L F O U R -

M O D U L E S

REQUIRING MODULEShttp.js

How does it find these files?


var fs = require('fs'); fs.js

How does ‘require’ return the libraries?

M O D U L E S

LETS CREATE OUR OWN MODULE

custom_hello.js

custom_goodbye.js

app.js

exports = hello;

var hello = require('./custom_hello');

hello();

exports defines what require returns

var hello = function() { console.log("hello!");}

exports.goodbye = function() { console.log("bye!");}

var gb = require('./custom_goodbye');

gb.goodbye();

require('./custom_goodbye').goodbye(); If we only need to call once

M O D U L E S

EXPORT MULTIPLE FUNCTIONS

my_module.js

app.js

var foo = function() { ... }var bar = function() { ... }

exports.foo = fooexports.bar = bar

var myMod = require('./my_module');myMod.foo();myMod.bar();

my_module.js

foo

bar

var baz = function() { ... }baz

“private”

M O D U L E S

MAKING HTTP REQUESTSapp.js

logs response body

begins request


var options = { host: 'localhost', port: 8080, path: '/', method: 'POST' }

var request = http.request(options, function(response){response.on('data', function(data){console.log(data);

});});

request.end();request.write(message);

finishes request

var message = "Here's looking at you, kid.";

M O D U L E S

ENCAPSULATING THE FUNCTIONapp.jsvar http = require('http');

var makeRequest = function(message) { var options = { host: 'localhost', port: 8080, path: '/', method: 'POST' }

var request = http.request(options, function(response){response.on('data', function(data){console.log(data);

});});

request.end();}

makeRequest("Here's looking at you, kid.");

request.write(message);

Text

M O D U L E S

CREATING & USING A MODULEmake_request.jsvar http = require('http');

var makeRequest = function(message) {

}

exports = makeRequest;

...

app.jsvar makeRequest = require('./make_request');

makeRequest("Here's looking at you, kid");makeRequest("Hello, this is dog");

Where does require look for modules?

M O D U L E S

REQUIRE SEARCH

var make_request = require('make_request')

/Home/eric/my_app/app.js

• /Home/eric/my_app/node_modules/

var make_request = require('./make_request')var make_request = require('../make_request')var make_request = require('/Users/eric/nodes/make_request')

• /Home/eric/node_modules/make_request.js• /Home/node_modules/make_request.js• /node_modules/make_request.js

look in same directorylook in parent directory

Search in node_modules directories

M O D U L E S

NPM: THE USERLAND SEAPackage manager for node

• Comes with node• Module Repository• Dependency Management• Easily publish modules• “Local Only”“Core” is small. “Userland” is large.

M O D U L E S

INSTALLING A NPM MODULE

$ npm install request https://github.com/mikeal/requestIn /Home/my_app

Home my_app request node_modules

Installs into local node_modules directory

var request = require('request');In /Home/my_app/app.js

Loads from local node_modules directory

M O D U L E S

LOCAL VS GLOBAL

Global npm modules can’t be required

$ npm install coffee-script -gInstall modules with executables globally

$ coffee app.coffee

var coffee = require('coffee-script');

$ npm install coffee-script

var coffee = require('coffee-script');

global

Install them locally

M O D U L E S

FINDING MODULESnpm registry

$ npm search request

npm command line

github search

toolbox.no.de

M O D U L E S

DEFINING YOUR DEPENDENCIESmy_app/package.json

version number

$ npm install

my_app connect node_modules

Installs into the node_modules directory

{"name": "My App","version": "1",

"dependencies": { "connect": "1.8.7" }}

M O D U L E S

DEPENDENCIES

my_app connect node_modules

Installs sub-dependencies

connect node_modules qs

connect node_modules mime

connect node_modules formidable

No conflicting modules! "dependencies": { "connect": "1.8.7" }

my_app/package.json

M O D U L E S

SEMANTIC VERSIONING"connect": "1.8.7" 1 8 7

Major Minor Patch. .

http://semver.org/ "connect": "~1.8.7" >=1.8.7 <1.9.0 Considered safe

"connect": "~1.8" >=1.8 <2.0.0 API could change

"connect": "~1" >=1.0.0 <2.0.0 DangerousRanges

E X P R E S S- L E V E L F I V E -

E X P R E S S

EXPRESS“Sinatra inspired web development framework for Node.js -- insanely fast, flexible, and simple”

• Easy route URLs to callbacks• Middleware (from Connect)• Environment based configuration• Redirection helpers• File Uploads

E X P R E S S

INTRODUCING EXPRESS$ npm install express

var express = require('express');

var app = express.createServer();

app.get('/', function(request, response) { response.sendfile(__dirname + "/index.html");});

app.listen(8080);

$ curl http://localhost:8080/> 200 OK

root route

current directory

E X P R E S S

EXPRESS ROUTESapp.js

route definition

get the last 10 tweets for screen_name

pipe the request to response

var request = require('request');var url = require('url');

app.get('/tweets/:username', function(req, response) {

var username = req.params.username;

options = { protocol: "http:", host: 'api.twitter.com', pathname: '/1/statuses/user_timeline.json', query: { screen_name: username, count: 10} }

var twitterUrl = url.format(options); request(twitterUrl).pipe(response);});

E X P R E S S

EXPRESS ROUTES

E X P R E S S

EXPRESS + HTML

E X P R E S S

EXPRESS TEMPLATESapp.js

tweets.ejs

app.get('/tweets/:username', function(req, response) {...request(url, function(err, res, body) {var tweets = JSON.parse(body);response.render('tweets.ejs', {tweets: tweets, name: username});

});});

<h1>Tweets for @<%= name %></h1><ul> <% tweets.forEach(function(tweet){ %> <li><%= tweet.text %></li> <% }); %></ul>

E X P R E S S

EXPRESS TEMPLATES

E X P R E S S

TEMPLATE LAYOUTS

<!DOCTYPE html><html> <head> <title>Tweets</title> </head> <body> <%- body %> </body></html>

<h1>Tweets for @<%= name %></h1><ul> <% tweets.forEach(function(tweet){ %> <li><%= tweet.text %></li> <% }); %></ul>

tweets.ejs

layout.ejs

E X P R E S S

EXPRESS TEMPLATES

S O C K E T . I O- L E V E L S I X -

S O C K E T . I O

CHATTR

S O C K E T . I O

WEBSOCKETS

browser traditional serverTraditional request/response cycle

Using duplexed websocket connection

S O C K E T . I O

WEBSOCKETS

browser socket.io

S O C K E T . I O

SOCKET.IO FOR WEBSOCKETS

var socket = require('socket.io');var app = express.createServer();var io = socket.listen(app);

Abstracts websockets with fallbacks

io.sockets.on('connection', function(client) {

});console.log('Client connected...');

<script src="/socket.io/socket.io.js"></script>

var server = io.connect('http://localhost:8080');<script>

</script>

$ npm install socket.io

app.js

index.html

S O C K E T . I O

SENDING MESSAGES TO CLIENTio.sockets.on('connection', function(client) {

});

console.log('Client connected...');

<script src="/socket.io/socket.io.js"></script>


</script>

app.js

index.html

client.emit('messages', { hello: 'world' });

server.on('messages', function (data) {

});alert(data.hello);

emit the ‘messages’ event on the client

listen for ‘messages’ events

S O C K E T . I O

CHATTR HELLO WORLD

S O C K E T . I O

SENDING MESSAGES TO SERVERio.sockets.on('connection', function(client) {

});


</script>

app.js

index.html

client.on('messages', function (data) {

});console.log(data);

$('#chat_form').submit(function(e){var message = $('#chat_input').val();

socket.emit('messages', message);});

listen for ‘messages’ events

emit the ‘messages’ event on the server

S O C K E T . I O

CHATTR HELLO WORLD

S O C K E T . I O

BROADCASTING MESSAGES

clients

server

app.jssocket.broadcast.emit("message", 'Hello');

S O C K E T . I O

BROADCASTING MESSAGESio.sockets.on('connection', function(client) {

});

<script>

</script>

app.js

index.html

client.on('messages', function (data) {

});

...

broadcast message to all other clients connectedclient.broadcast.emit("messages", data);

server.on('messages', function(data) { insertMessage(data) });insert message into the chat

S O C K E T . I O

BROADCASTING MESSAGES

S O C K E T . I O

SAVING DATA ON THE SOCKETio.sockets.on('connection', function(client) {

});


</script>

app.js

index.html

client.on('join', function(name) {client.set('nickname', name);

});set the nickname associated

with this client

server.on('connect', function(data) {$('#status').html('Connected to chattr');nickname = prompt("What is your nickname?");

server.emit('join', nickname);});

notify the server of theusers nickname

S O C K E T . I O

SAVING DATA ON THE CLIENTio.sockets.on('connection', function(client) {

});

app.jsclient.on('join', function(name) {

client.set('nickname', name);});

set the nickname associated with this client

client.on('messages', function(data){

});

client.broadcast.emit("chat", name + ": " + message);client.get('nickname', function(err, name) {

});

get the nickname of this client before broadcasting message

broadcast with the name and message

S O C K E T . I O

SAVING DATA ON THE CLIENT

P E R S I S T I N G D A T A- L E V E L S E V E N -

P E R S I S T I N G D A T A

RECENT MESSAGES

});});

});


RECENT MESSAGESio.sockets.on('connection', function(client) {client.on('join', function(name) {client.set('nickname', name);client.broadcast.emit("chat", name + " joined the chat");

});client.on("messages", function(message){

client.get("nickname", function(error, name) {client.broadcast.emit("messages", name + ": " + message);

app.js

});});

});


STORING MESSAGES

io.sockets.on('connection', function(client) {client.on("messages", function(message){

client.get("nickname", function(error, name) {

app.js

storeMessage(name, message);

var messages = []; store messages in arrayvar storeMessage = function(name, data){

messages.push({name: name, data: data});if (messages.length > 10) {

messages.shift();}

}

add message to end of array

if more than 10 messages long,remove the last one

when client sends a message call storeMessage


EMITTING MESSAGESio.sockets.on('connection', function(client) {

});

app.js

client.on('join', function(name) {

});

messages.forEach(function(message) { client.emit("messages", message.name + ": " + message.data); }); iterate through messages array

and emit a message on the connectingclient for each one

...


RECENT MESSAGES


PERSISTING STORES• MongoDB• CouchDB• PostgreSQL• Memcached• Riak

All non-blocking!

Redis is a key-value store


REDIS DATA STRUCTURES

Strings SET, GET, APPEND, DECR, INCR...

Hashes HSET, HGET, HDEL, HGETALL...

Lists LPUSH, LREM, LTRIM, RPOP, LINSERT...

Sets SADD, SREM, SMOVE, SMEMBERS...

Sorted Sets ZADD, ZREM, ZSCORE, ZRANK...

data structure commands


REDIS COMMAND DOCUMENTATION


NODE REDIS

client.get("message1", function(err, reply){ console.log(reply);});


REDIS

key value

"hello, yes this is dog"

var redis = require('redis');var client = redis.createClient();

client.set("message1", "hello, yes this is dog");client.set("message2", "hello, no this is spider");

commands are non-blocking

$ npm install redis


REDIS LISTS: PUSHINGAdd a string to the “messages” list

client.lpush("messages", message, function(err, reply){ console.log(reply);}); "1”

var message = "Hello, no this is spider";client.lpush("messages", message, function(err, reply){ console.log(reply);}); "2”

replies with list length

Add another string to “messages”

var message = "Hello, this is dog";


REDIS LISTS: RETRIEVINGUsing LPUSH & LTRIM

trim keeps first two stringsand removes the rest

Retrieving from list

client.lrange("messages", 0, -1, function(err, messages){ console.log(messages);})

["Hello, no this is spider", "Oh sorry, wrong number"]

replies with all strings in list

var message = "Oh sorry, wrong number";client.lpush("messages", message, function(err, reply){client.ltrim("messages", 0, 1);

});


CONVERTING MESSAGES TO REDISvar storeMessage = function(name, data){

messages.push({name: name, data: data});

if (messages.length > 10) { messages.shift(); }}

Let’s use the List data-structure

app.js


CONVERTING STOREMESSAGE

var storeMessage = function(name, data){

}

var redisClient = redis.createClient();

var message = JSON.stringify({name: name, data: data});

redisClient.lpush("messages", message, function(err, response) {redisClient.ltrim("messages", 0, 10);

});keeps newest 10 items

app.js

need to turn object into string to store in redis


OUTPUT FROM LISTclient.on('join', function(name) {

});

messages.forEach(function(message) {client.emit("messages", message.name + ": " + message.data);

});

app.js


OUTPUT FROM LIST

client.on('join', function(name) {

});

messages.forEach(function(message) {

client.emit("messages", message.name + ": " + message.data);

});

redisClient.lrange("messages", 0, -1, function(err, messages){messages = messages.reverse();

message = JSON.parse(message);

});

app.js

reverse so they are emittedin correct order

parse into JSON object


IN ACTION


CURRENT CHATTER LISTSets are lists of unique data

client.sadd("names", "Dog");client.sadd("names", "Spider");client.sadd("names", "Gregg");

client.srem("names", "Spider");

client.smembers("names", function(err, names){ console.log(names);});

["Dog", "Gregg"]

reply with all members of set

add & remove members of the names set


ADDING CHATTERSclient.on('join', function(name){

client.broadcast.emit("add chatter", name); redisClient.sadd("chatters", name);});

app.jsnotify other clients a chatter has joined

add name to chatters set

index.htmlserver.on('add chatter', insertChatter);var insertChatter = function(name) {

var chatter = $('<li>'+name+'</li>').data('name', name); $('#chatters').append(chatter);}


ADDING CHATTERS (CONT)client.on('join', function(name){

client.broadcast.emit("add chatter", name);

redisClient.sadd("chatters", name);});

app.jsnotify other clients a chatter has joined

add name to chatters set

emit all the currently logged in chattersto the newly connected client

redisClient.smembers('names', function(err, names) { names.forEach(function(name){ client.emit('add chatter', name); });});


REMOVING CHATTERSclient.on('disconnect', function(name){ app.js

index.html

client.get('nickname', function(err, name){client.broadcast.emit("remove chatter", name);

redisClient.srem("chatters", name); });});

remove chatter when they disconnect from server

server.on('remove chatter', removeChatter);

var removeChatter = function(name) { $('#chatters li[data-name=' + name + ']').remove();}


WELCOME TO CHATTR

intro to node js

Technology