20 tips for building a scalable and robust node.js stack that developers love
DESCRIPTION
The platform team at eBay recently embarked on a mission to provide a new Node.js-based web development stack that developers would love using. Despite being an entirely new ecosystem to most developers, we wanted to introduce a stack that had a minimal learning curve, performed extremely well and made front-end web development fun again. We recognized that as web applications grow in complexity it becomes increasingly important to provide a web development stack that shields developers from potential problems and allows for maximum developer productivity. The team decided to rethink some of the industry norms to see how we could simplify the toolchain and make it easier for developers to build an application that interacts with a myriad of complex infrastructure pieces. In addition, we also never lost focus on the requirement that the stack must scale extremely well. In this talk we will share our tips and solutions, including but not limited to: Avoid build tools and build steps Be flexible, don't use a restrictive framework Avoid introducing “magic” and still hide complexity Expect services to fail and have a backup plan Minimize developer idle time and provide instant feedback Automate using scaffolding and command line tools Organize project files to enforce modularity and best practices Keep documentation up-to-date and easy to digest Automatically optimize the delivery of JavaScript and CSS resources This talk is a must for all web developers who are interested in improving their productivity and building high performance web applications that are easy to maintain. RaptorJS: http://raptorjs.org/TRANSCRIPT
20 TIPS FOR BUILDING A SCALABLE AND ROBUST NODE.JS STACK THAT DEVELOPERS LOVE
@
Patrick Steele-Idem
@psteeleidem
github.com/patrick-steele-idem
Senior Platform Architect @
These tips are based on lessons I learned from building and supporting web development stacks for both the JVM and Node.js in companies big and small.
AVOID “MAGIC”
If developers cannot reason about a system then they will not be happy or productive
MORE ON MAGIC…
• What is Magic?
• Runtime code that changes how other code runs
• Non-obvious and overly clever tricks• Abstractions and indirections
• Avoiding Magic:
• Prefer explicit over implicit• Code comments and clear documentation
KEEP DOCUMENTATION WITH THE CODE
The documentation you are reading is out-of-date and confusing.
Would you like to like to continue?
README files for the win. Learn Markdown. Branch docs with code.
DON’T FORK THE COMMUNITY
How many package managers do you really need?(I’m looking at you Bower, Component, Ender, Jam, etc.)
GIVE DEVELOPERS CHOICES
Don’t build a restrictive framework!
KEEP AN EYE ON THE FUTURE
Do not become too attached to today’s technologies and practices
THINK MODULAR, NOT MONOLITHIC!
Image Source: http://www.flickr.com/photos/albatros/
Much easier to digest small, bite-sized modules compared to a monolithic framework
MULTI-VERSION WITHOUT CONFLICTS
Image Source: http://www.flickr.com/photos/albatros/
Do not force everyone to agree on the same version of a module! Dependencies are your friends.
FOLLOW SEMANTIC VERSIONING
• Increment MAJOR version when you make incompatible API changes,
• Increment MINOR version when you add functionality in a backwards-compatible manner, and
• Increment PATCH version when you make backwards-compatible bug fixes.
• Notes:
• 0.x.y – Anything goes• Pre-release labels to indicate “beta” versions
• More info:
• http://semver.org/
ENCOURAGE A MODULAR DIRECTORY STRUCTURE
Organizedby Modules:
Organized by MVC and Resource Type:
ENABLE AUTO RESTART AND LIVE CODING
Minimize time spent bouncing between windows and waiting.Make sure server startup time never exceeds 2s.
http://www.flickr.com/photos/mbiebusch/
AVOID THE BUILD STEP
In development, do as much at runtime as possible.
http://xkcd.com/303/
INTERNAL OPEN SOURCE
Cultivate a culture of collaboration within a company and break down cross-team boundaries
PROVIDE AN INTERNAL SUPPORT FORUM
Empower and encourage users to help each other. Redirect emails to the support forum. Redirect open
source questions to public support forums.
Image Source: http://www.flickr.com/photos/31065898@N08/
DON’T BUILD A STACK IN ISOLATION
Work in conjunction with an application team to build a new stack.
Image source: http://www.flickr.com/photos/epublicist/
BE HELPFUL WITH ERROR MESSAGES
MY APPLICATION STOPPED WORKING AND WE LOST A TON OF MONEY
BUT I FOUND A VERY HELPFUL MESSAGE IN THE SERVER LOGS, SO I’VE GOT THAT GOING FOR
ME, WHICH IS NICE.
Expect services to go down. Expect bad input. Catch errors early. Log the full stack trace.
GO REACTIVE
React to events. React to load. React to failure. React to users.
Which is worse, a service that fails fast or an unresponsive service?
Write tests and find out.
http://www.reactivemanifesto.org/
http://www.flickr.com/photos/dougww/
MANAGE WORKER STATE
Start
Warm-up
Put into Traffic
Unhealthy
Take out of Traffic
Kill
Proactively handle garbage collection, errors and high CPU and memory usage
Do not forget this step!
Be proactive
Be able to detect
AVOID “BLOCKING” MIDDLEWARE
Middleware should not add significant overhead to every request.
20ms 40ms 60ms 80ms 100ms
Page Controller
Middleware Your Code
SUPPORT ASYNCHRONOUS RENDERING
HTMLTemplate
Sy
nc
hro
no
us
As
yn
ch
ron
ou
s
View Model
No Delay
Data Providers
1. Setup data providers2. Go straight to the
template!
ControllerInput
HTMLTemplateView Model
Long Delay
ControllerInput
1. Make service calls2. Wait for all data to come
back3. Prepare view model
PROVIDE A COMMAND LINE TOOL
Easily support environment setup, app creation, deployment, scaffolding, upgrades, etc.
EVALUATE THIS SESSION
Sign-in: www.eclipsecon.org
Select session from schedule
Evaluate:
1
2
3