microservices, containers and docker for dummies
TRANSCRIPT
Microservices, Containers and Docker
by Joost HietbrinkApril, 2015
Joost?Linkedin - Github
YelloYelloSold in 2008 to Yellow Pages Fastest Site Builder
StartupBootcamp Batch 2014
Goal?
Make you look smart in the pub!
Buzzword bingo..& bignumber bingo..
Agenda: Netflix
Microservices? Containers?
Docker?
Netflix
Microservices? Containers?
Docker?
$32.900.000.000 company (32.9B)
$2.390.000 revenue per employee (#1) (Apple $2,21M)
57.000.000 subscribers
40-60 countries (only US until 2010)
1997 (yep.. 18 years ago)
Reed Hastings CEO Netflix
1997 (yep.. 18 years ago)
1998 (yep.. 17 years ago)
2007 (10 years later..)
1.000.000.000 DVDs delivered
2007 (10 years later..)
2008
2010
Move to cloud (AWS)
2010
2015• 10x customers
• 20x visits
• 100x traffic
• 5x devs200 devs → 1.000 product devs
• 2x costs of IT Infrastructure in 2008
> 1/3 of US traffic is Netflix7.000.000.000 hours streamed (Q3 2014)
10.000.000.000.000.000.000 bytes (10.000PB)
10.000 starts / second
2.000.000.000 edge API requests per day
2.000.000 payment transactions per day
60.000 instances (AWS)
40.000.000.000.000.000 bytes S3 storage (40PB)
>500 microservices
1.500 employees (1.000 devs)
~30 engineering teams (microservices)
Netflix
Microservices? Containers?
Docker?
Loosely coupled service oriented architecture with bounded contexts
— Adrian Cockcroft, ex-Cloud Architect @ Netflix
SOA > Microservice
• Componentization via Services
• Organized around Business Capabilities
• Products not Projects
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
Characteristics of a Microservice Architecture
— Martin Fowler, loud-mouth
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's
communication structure.
— Melvyn Conway, 1967
• Componentization via Services
• Organized around Business Capabilities
• Products not Projects
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
Characteristics of a Microservice Architecture
— Martin Fowler, loud-mouth
Unix Philosophy
• Write programs that do one thing and do it well.
• Write programs to work together.
— M. Douglas McIlroy (Inventor Unix Pipe)
(Head Bell Labs Computing Sciences Research Center)
- 1978 -
➜ ~ tr 'A-Z' 'a-z' < doc.txt | tr -cs 'a-z' '\n' | sort | uniq | comm -23 - /usr/share/dict/words
• Componentization via Services
• Organized around Business Capabilities
• Products not Projects
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
Characteristics of a Microservice Architecture
— Martin Fowler, loud-mouth
Why?
• Smaller footprint:
• easier to develop
• easier deploy
• own responsibility
• Loosely coupled:
• develop independently (own stack)
• update independently (own versions)
• scale independently
• fail independently (own bugs)
How?
Netflix
Microservices? Containers?
Docker?
March 2013
“docker”
jan 2014 - $15.000.000 sep 2014 - $40.000.000 apr 2015 - $95.000.000
Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation
of operating-system-level virtualization on Linux.
Static website
Web frontend
User DB
Queue Analytics DB
Background workers
API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap 2
postgresql + pgv8 + v8
hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis-sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client
Development VM
QA server
Public Cloud
Disaster recovery
Contributor’s laptop
Production Servers
The ChallengeMultip
licity
of S
tacks
Multip
licity
of
hardware
environm
ents
Production Cluster
Customer Data Center
Do services and apps interact
appropriately?
Can I migrate
smoothly and quickly?
The Matrix From Hell
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM QA Server Single Prod
Server Onsite Cluster Public Cloud Contributor’s laptop
Customer Servers
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Multip
licity
of G
oods
Multip
ilicity of
metho
ds fo
r tran
sportin
g/storing
Do I worry about how
goods interact (e.g. coffee beans next to
spices)
Can I transport quickly and sm
oothly (e.g. from
boat to train to truck)
Cargo Transport Pre-1960
? ? ? ? ? ? ?? ? ? ? ? ? ?? ? ? ? ? ? ?? ? ? ? ? ? ?? ? ? ? ? ? ?? ? ? ? ? ? ?
Also a matrix from hell
Multip
licity
of G
oods
Multip
licity
of
metho
ds fo
r transportin
g/storing
Do I worry about how
goods interact (e.g. coffee beans next to
spices)
Can I transport quickly and sm
oothly (e.g. from
boat to train to truck)
Solution: Intermodal Shipping Container
…in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another
A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.
Static website Web frontend User DB Queue Analytics DB
Development VM
QA server Public Cloud Contributor’s laptop
Docker is a shipping container system for code Multip
licity
of Stacks
Multip
licity
of
hardware
environm
ents
Production Cluster
Customer Data Center
Do services and apps interact
appropriately?
Can I migrate
smoothly and quickly
…that can be manipulated using standard operations and run consistently on virtually any hardware platform
An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM QA Server Single Prod
Server Onsite Cluster Public Cloud Contributor’s laptop
Customer Servers
Docker eliminates the matrix from Hell
Docker > Containers• images (Dockerfile)
• layered filesystem (versioning)
• index/hub/repository (push, pull)
• machine: create hosts
• compose: linking
• swarm: clustering
so.. what?
for you as an Enterprise
• deploy faster
• separate per microservices
• no downtime
• microservice teams
• scale faster
• a-b testing
• make more money
• for us all
• greener
for you as a Developer
• get it running your project (microservices :))
• on your computer
• on every computer (production, test, that designer that needs to work a few hours)
• no more “works on my machine”
• easily deploy everywhere
• ‘stand on the shoulder of giants’
• deploy like Netflix
• ‘stand on the shoulders of many midgets’
• try things super quickly
• click to deploy (tutum.co)
