node.js streams talk
DESCRIPTION
Node.js talk on streamsTRANSCRIPT
ROW ROW ROW YOUR BOAT
by @zladuric
We have gathered here today
● How to build Node.js Streams?
● A talk to introduce the topic of streams inJavaScript, how this works in Node.js and howit didn't work out before.
● You're my lab!
by @zladuric
What are we talking about
● The past - what was there before● The present - what are we doing now● The future - how you can get and use this
● The client - the browser side of things● The server - Streams API
But first - why a stream
Regular stuff● Comes as a whole blob
● Takes up memory
● When action is broken at 95%,redo the whole thing
Streamed stuff● Just like regular, but chunked
into bits
● When broken, we can still reuseparts we got
● PIPES!!1
The past
Let's hope it dies!
All right, I'll tell ya
On the client● Until a few years ago, we could
not do much with streams in JS
● Mostly for video/audio transferwith Flash, Java Applets
On the server● Comes largely from unix
network sockets
● stdin, stdout, stderr
● Created to streamline (yeah, Iknow) I/O operations
● In web: Flash Media server, Red5
● Encoders/decoders, format converters● Bridges between backend services, more I/O
WTF?
Node.js past
● Streams API was (is?) very volatile
● Backwards compatibility
● Main problems:
– Your stream could ignore stream.pause();)
– Stream starts right away – whether you're ready ornot
The present
Node.js “streams2”:
Stability: 2 – Unstable
● Node.js is unstable ;)
● No breaking changes expected for Node.js 0.12
Stream in Node.js
● All streams are instances of an EventEmmiter
(events and listeners/hooks)● var Readable = require('stream').Readable;
– Implement your own _read();
● var Writable = require('stream').Writable;– Implement your own _write();
● var Duplex = implement both;
● var Transform = convert input then pass to output– Implement _transform(); and flush();
How to be streamin' in Node
$(#button).on('click', handler); anyone?
Dumb example
stream.Readable
● Flowing and non-flowing?
● Common events and methods
● Examples: fs.createReadStream,http.createServer listeners, tcp and net sockets.
Translate stream events into english
When stream says: It means:● readable “I have stuff to be read.”
● data “Here's a chunk.”
● end “I'm all out, you've read it.”
● close “Bye bye!” (file.close);
● error “Pain starts here.”
Pushmi-pullyu
● Flowing mode
stream.on('data', function(chunk) {
// chunk.length bytes
// or if we set encoding, chunk is a string
// or if objectMode: true, then values/objs
});
● Non-flowing:
var chunk = stream.read();
Borin Simple example
● this.push(); → stream.read();
● objectMode is handy
Writable stream
● To implement a writable stream, implement_write method, drain
Transform streams
Splits input intolines
● _transform() - whatto do on input data
● _flush – what to dowhen input ends
And now pipes!
var stream = getSomeReadStream()
.pipe(toConverterStream())
.pipe(toEncoderStream())
.pipe(toTransportStream())
.pipe(toTheConsumerStream());
Piping examples
Read file line by line Top secret
Steal Google logo
Do you use it?
The client
● WebRTC, WebSockets, Workers – they allstream data
● XMLHttpRequest
● User Media (MediaStream) API,Blob/FileSystem API, Audio, Geolocation etc.
Example
http://jszgb.com/ - hacked to steal selfies
<video id="video" autoplay></video>
<script> // simplified navigator.webkitGetUserMedia({video: true},function(localMediaStream) { console.log('getting some video'); var video = document.getElementById('video'); video.src =window.URL.createObjectURL(localMediaStream); }, console.log);</script>
Other useful options, events andmethods
● options.HighWaterMark = 1024;
● stream.setEncoding('utf-8');
● pause/resume a stream
● wrap(); - for old libs < Node v0.10
● pipe()/unpipe(), unshift()
● 'finish', 'close', 'drain' events
The future
● Node.js is relatively stable
● Whatwg is trying to standardize stream APIs
from: Domenic Denicola
Question time
1. Was my english ok?
2. How boring was I, from 1 – 10?
Your turn.
Zlatko Đurić