node.js and backend development - nthu-datalab.github.io · outline •node.js –events and...
TRANSCRIPT
![Page 1: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/1.jpg)
Node.js and Backend Development
Shan-Hung Wu
CS, NTHU
![Page 2: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/2.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
2
![Page 3: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/3.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
3
![Page 4: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/4.jpg)
Clone hello-node
• Yargs– Command line interface (CLI) tools
• Usage:
4
$ node src/main.js <command> [options]
$ npm install --save yargs
![Page 5: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/5.jpg)
• Javscript runtime engine
• Why Node.js in the backend?
– Event-based and asynchronous I/O; very fast
– NPM and large ecosystem
5
![Page 6: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/6.jpg)
Node Runtime
• Global objects:– window global (scope)– document process (.env, .argv(), .exit()
etc.)– os, fs, module.exports
• Node.js supports ES6– No need for Babel– No import (use require() instead)
• See API Docs6
$ node app.js
$ node // REPL
![Page 7: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/7.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
7
![Page 8: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/8.jpg)
WeatherMood Postson Node.js
8
Why fast as an backend engine?
![Page 9: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/9.jpg)
Asynchronous I/O
• Shortens exec time
– If with parallel I/O channels
9
A1
A2
A4
A6 A3 A5CPU
I/O 1
I/O 2
Time
A1
A2
A4
A6A3 A5CPU
I/O 1
I/O 2
Time
// A.js
... // 1
fs.readReadFile('file.name', (err, data) => {
... // 3
}); // 2
http.get('url', res => {
... // 5
}); // 4
... // 6
Async
Sync
![Page 10: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/10.jpg)
Single-Threaded Event Loop
10
A1CPU
I/O 1
I/O 2
// User A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// User B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
Process
![Page 11: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/11.jpg)
Single-Threaded Event Loop
11
A1
A2
CPU
I/O 1
I/O 2
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
Process
![Page 12: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/12.jpg)
Single-Threaded Event Loop
12
A1
A2
A4
CPU
I/O 1
I/O 2
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
Process
![Page 13: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/13.jpg)
Single-Threaded Event Loop
13
A1
A2
A4
A6CPU
I/O 1
I/O 2
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
Process
![Page 14: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/14.jpg)
Single-Threaded Event Loop
14
A1
A2
A4
A6CPU
I/O 1
I/O 2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
Process
![Page 15: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/15.jpg)
Single-Threaded Event Loop
15
A1
A2
A4
A6CPU
I/O 1
I/O 2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
A3
Process
![Page 16: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/16.jpg)
Single-Threaded Event Loop
16
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
A3
Process
![Page 17: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/17.jpg)
Single-Threaded Event Loop
17
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
A3
Process
![Page 18: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/18.jpg)
Single-Threaded Event Loop
18
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
A3
B6
Process
![Page 19: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/19.jpg)
Single-Threaded Event Loop
19
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
B6 A3
Process
![Page 20: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/20.jpg)
Single-Threaded Event Loop
20
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
A5
B6 A3
Process
![Page 21: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/21.jpg)
Single-Threaded Event Loop
21
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
A5
B6 A3B3
Process
![Page 22: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/22.jpg)
Single-Threaded Event Loop
22
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
B3
B6 A3 A5
Process
![Page 23: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/23.jpg)
Single-Threaded Event Loop
23
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
B6 A3 A5 B3
Process
![Page 24: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/24.jpg)
Single-Threaded Event Loop
24
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
// A.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
// B.js
... // 1
fs.readReadFile(..., () => {
... // 3
}); // 2
http.get(..., () => {
... // 5
}); // 4
... // 6
B4
B6 A3 A5 B3 B5
Process
![Page 25: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/25.jpg)
Advantages
• Interleaved exec of multiple code blocks
• Short response time
• Pipelining between CPU and I/Os– High throughput
• Avoids overhead of concurrency control (e.g., locks, thread scheduling, etc.)– Higher throughput than muti-threaded engines
25
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
B4
B6 A3 A5 B3 B5
![Page 26: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/26.jpg)
Multi-Core Machines?
• Cluster module that runs one Node.js process per core
• Scales up linearly, if
– Workload can be partitioned evenly by processes
– I/Os are not saturated
26
![Page 27: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/27.jpg)
When Not to Use Node.js?
• CPU-bound tasks
• Or, use child_process.exec() (or spawn() for streaming output from child)
– Foreground core + background cores
27
A1
A2
A4
A6CPU
I/O 1
I/O 2
B2
B1
B4
B6 A3 A5 B3 B5
![Page 28: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/28.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
28
![Page 29: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/29.jpg)
NPM
• Package manager
• Packages vs. modules?
29
$ npm init
$ npm install --[save|save-dev] <pkg-name>
var m = require('module');
![Page 30: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/30.jpg)
--save vs. --save-dev?
• Given dependency tree:
• People who clone/fork your package will download the following packages:
30
Your proj Pkg 1
Pkg 2
Pkg 1 Pkg 3
Pkg 4
Pkg 2 Pkg 3
Pkg 5
{Pkg 1, Pkg 2, Pkg 3} // via 'npm install'
![Page 31: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/31.jpg)
Command Line Interface (CLI)
• Option 1: to parse process.argv yourself
• Option 2: yargs.argv
– Defines commands and their options
– Help
– Sanity checks
31
$ npm install --save yargs
![Page 32: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/32.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
32
![Page 33: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/33.jpg)
Debugging
• Chrome inspector equivalent?
33
// in src
debugger;
$ node debug src/main.js <command> [options]
debug> n
debug> c
debug> repl
![Page 34: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/34.jpg)
Node Inspector (Experimental)
• Then paste “chrome-devtools://..” into Chrome
34
$ node --inspect --debug-brk src/main.js ...
![Page 35: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/35.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
35
![Page 36: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/36.jpg)
Node.js as Backend Engine
• Typically, web workloads are I/O bound
– High throughput
– Low latency
• If designed well, API requests can be easily partitioned across multiple processes
– Scales linearly
36
![Page 37: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/37.jpg)
Clone weathermood-server
• Checkout the file branch
• Express– A web app framework based on Node.js
• Body-Parser– An Express middleware for parsing request body
• Nodemon– Auto-restarter (remember “webpack -w”?)
37
$ npm install --save express body-parser
$ npm install –save-dev nodemon
![Page 38: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/38.jpg)
Web App Backend Development
1. Prepare static resources– E.g., *.html, *.css, client-side JS, images, etc.
– In dist/ of branch server-file of weathermood
2. Define API for AJAX calls– Dynamic resources, i.e., API for AJAX calls
3. Code API
4. Deploy web app to hosting server(s)– Web App ≠ web server(s)
– For now, local machine as development server
38
![Page 39: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/39.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
39
![Page 40: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/40.jpg)
Defining APIs
• E.g., listing posts, create a post, vote, etc.
• But HTTP defines only 4 methods– GET, POST, PUT, DELETE
• Option 1: define new “verbs”– Always POST to the same URL
– Define body (can follow SOAP)
• Option 2: define new “nouns”– E.g., vote POST vote
– Different URLs for different nouns/resources
– REST makes your backend simple and scalable
40
![Page 41: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/41.jpg)
URL Mappings
• Each resource type maps to 2 URL types– Collection URLs vs resource URLs
• List post: GET /posts?seatchText=...&...• Create post: POST /posts• Vote “clear”: POST /posts/${id}/clearVotes
41
URLs\Methods GET POST PUT DELETE
http://${host}/${resource}s
List all resources (satisfying query “?...”).
Create a new resource (unknown ID).
Replace the entire collection.
Delete the entire collection.
http://${host}/${resource}s/${id}
Read a specific resource.
Treat this resource as a collection and create a new member.
Update this resource or create one (known ID).
Delete this resource.
![Page 42: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/42.jpg)
HTTP Response Codes
• GET: – 200 OK (with body)
• POST: – 200 OK (with body)– 201 Created (with header Location showing ID)– 204 No Content
• PUT and DELETE:– 200 OK (with body)– 204 No Content
• Error: – 400 Bad Request, 401 Unauthorized, 404 Not Found, or
500 Internal Server Error
42
![Page 43: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/43.jpg)
Requirements
• Stateless: session state (e.g., shopping cart) cannot be kept in client side – Session state sent with requests using cookies
– So, requests from clients can be partitioned easily (scalable web servers)
• GET requests much have no side effect– Allows proxy nodes in the routing path
• PUT and DELETE requests must be idempotent: duplicated requests has no effect
43
![Page 44: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/44.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
44
![Page 45: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/45.jpg)
Express
• Normal flow: routers model calls– Request parsing: req.query, req.params, req.body
– Responses: res.status(), res.json(), res.sendStatus()
• Error flow: throw error handler middleware responses
45
![Page 46: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/46.jpg)
Middleware
• Error handling
• Serving static files
• Filtering
• Logging
• Validation
• Authentication and authorization
• and more...
46
![Page 47: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/47.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
47
![Page 49: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/49.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
49
![Page 50: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/50.jpg)
Cloud Computing
• Not only services
• Also platforms or infrastructure
– Pay only what you used
50
![Page 51: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/51.jpg)
IaaS, PaaS, and SaaS
51
![Page 52: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/52.jpg)
Web App Backend:PaaS or IaaS?
52
![Page 53: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/53.jpg)
Development Efficiency
• Iterations: bug fixes, updates, finding PMF, etc.
• Time is money!
• IaaS: high management cost, low elastisity
• PaaS: low cost and elastic, but
“Well, it works on my computer…”
![Page 54: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/54.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
54
![Page 55: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/55.jpg)
• PaaS
• You define your runtime
55
![Page 56: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/56.jpg)
Virtual Machines vs. Containers
• Lightweight, elastic
• Consistent runtime
56
![Page 57: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/57.jpg)
“Dockerizing” an App
1. Build a Docker Image
2. Upload the image to targeted server(s)
3. Launch container from image
57
![Page 58: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/58.jpg)
Example: Local Development Server
• Install Docker Community Edition first
58
// in project folder
$ vim Dockerfile
$ vim .dockerignore
$ docker build -t <name:tag> .
$ docker images
$ docker run -p 80:8080 -d <image>
$ docker <stop|restart> <container>
$ docker ps [-a]
$ docker system prune
![Page 59: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/59.jpg)
Watch out Data Loss!
• Modifications to filesystem are local to a container
– Gone if container deleted
• If necessary, use data volumes to persists data across containers
– Basically, specially-designated directories
59
![Page 60: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/60.jpg)
Beyond development server?
60
![Page 61: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/61.jpg)
Outline
• Node.js– Events and asynchronous I/O– NPM and CLI tools– Debugging
• Backend Development using Express– RESTful API– Express: routers and middleware– Testing and debugging with Postman
• Deployment– Cloud computing and Docker– Amazon Elastic Beanstalk
61
![Page 62: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/62.jpg)
“Typical”Environment
• DDoS protection• Static vs. dynamic
resources• High availability• LB, caching,
acceleration (SSL), etc.• Auto-scaling groups
• Scale up vs. out• Slave DBMS can serve
read-only requests
62
M S
Route Balancer
User Requests
Load Balancer (L3-7)
CDN
Storage
Web Server Cluster
Avail. Zone (AZ)
Reverse Proxy (L7)
Mem. Cache
Avail. Zone
DBMS
![Page 63: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/63.jpg)
Solutions
Node Amazon (AWS) Alternatives
Route Balancer Route 53 Cloudflare
CDN CloudFront Cloudflare, Google Cloud CDN
Storage Simple Storage Service (S3) HDFS
Load Balancer Elastic Load Balancing (ELB) Nginx
Reverse Proxy Nginx
Web Server Elastic Compute Cloud (EC2) Heroku
Mem. Cache ElastiCache Redis, Memcached
DBMS Relational DB Service (RDS) PostgreSQL, MySQL, MongoDB
63
• Web app on EC2 (with Docker and Nginx preinstalled)
• But what about the rests?
![Page 64: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/64.jpg)
AWS Elastic Beanstalk
64
• Single AZ by default
![Page 65: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/65.jpg)
Staging Environment
65
![Page 66: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/66.jpg)
Instructions1. Create an AWS account
– Free-tier for the first year, and free credits from AWS Educate– Credit card needed (one per group); no immediate charge
2. Install Elastic Beanstalk CLI– Install Python runtime (e.g., Aanaconda) first
66
$ pip install --upgrade awsebcli
$ eb –version
// in project folder
$ eb init // create eb application
$ eb create [--single] // create environment
$ eb terminate <env>
// upates
$ git commit
$ eb deploy <env> // deploys the latest commit
$ eb use <env> // env for current branch
![Page 67: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/67.jpg)
“Typical” Environment
67
User Requests
• How to enable CDN?
• Add AZs easily in console
![Page 68: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/68.jpg)
Cache Control
• CloudFront and client browsers use Cache-Control response header to cache static files
68
// in server.js
app.use(express.static('dist', {
maxAge: ... // in ms. For client browsers and proxies
}));
// or
app.use(express.static('dist', {
setHeaders: (res, path, stat) => {
// in seconds. For proxies only
res.set('Cache-Control', 'public, s-maxage=...');
}
}));
• Be careful about file versioning
![Page 69: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/69.jpg)
Readings (Optional)
• HTTPS
– Production environments
– Staging environments
• Custom domain names
69
![Page 70: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/70.jpg)
Worker Environments
• For CPU-bound tasks (e.g., data analytics)
70
![Page 71: Node.js and Backend Development - nthu-datalab.github.io · Outline •Node.js –Events and asynchronous I/O –NPM and CLI tools –Debugging •Backend Development using Express](https://reader030.vdocuments.net/reader030/viewer/2022040218/5e00f75eae70e23cde0e35bf/html5/thumbnails/71.jpg)
Assignment: TODOs• RESTful API• Server-side routers, model, etc.• Client-side AJAX calls• AWS Deployment (one per group)
71