An Open Platform to Build, Ship, and Run Distributed Applications

Who am I?

• Nick Stinemates | @nickstinemates | bitbucket.org/keeb • Work: Head of Business Development / Tech Alliances @

Docker

A long history of work with Atlassian Products

•  JIRA critical to Issue and Agile management • Over 400 Bamboo Jobs running / day • Workflow built with SCM integratgion

• … and a special thanks for introducing me to Docker and for the amazing early support of the Docker project

Agenda

•  The Matrix from Hell and the Need for Containers •  Docker & Containers: how they work •  Docker Overview and Docker Ecosystem •  Current Usage and Adoption •  Creating an Open Environment •  Docker Roadmap & Business Model

~2000   2014  

Long  lived   Development  is  itera2ve  and  constant  

Monolithic  and  built  on  a  single  stack   Built  from  loosely  coupled  components  

Deployed  to  a  single  server   Deployed  to  a  mul2tude  of  servers  

Apps Have Fundamentally Changed

Static website

Web frontend

User DB

Queue Analytics DB

Background workers

API endpoint

nginx 1.5 + modsecurity + openssl + bootstrap 2

postgresql + pgv8 + v8

hadoop + hive + thrift + OpenJDK

Ruby + Rails + sass + Unicorn

Redis + redis-sentinel

Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs

Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Development VM

QA server

Public Cloud

Disaster recovery

Contributor’s laptop

Production Servers

The Problem in 2014 Mul*p

licity

 of  

Stacks  

Mul*p

licity

 of  

hardware  

environm

ents   Production Cluster

Customer Data Center

Do  services  and  apps  interact  appropriately?  

Can  I  migrate  

smoothly  and  quickly?  

The Matrix From Hell

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM QA Server Single Prod

Server Onsite Cluster

Public Cloud

Contributor’s laptop

Customer Servers

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Mul*p

licity

 of  

Goo

ds  

Mul*p

ilicity  of  

metho

ds  fo

r  tran

spor*n

g/storing  

Do  I  worry  about  

how  goods  interact  

(e.g.  coffee  beans  next  to  spices)  

Can  I  transport  quickly  and  sm

oothly  (e.g.  from

 boat  to  train  to  truck)  

An Inspiration…and some really ancient history: Cargo Transport Pre-1960

Mul2p

licity

 of  

Good

s  

Mul2p

licity

 of  

metho

ds  fo

r  transpor2n

g/storing  

Do  I  worry  about  

how  goods  interact  

(e.g.  coffee  beans  next  to  spices)  

Can  I  transport  quickly  and  sm

oothly  (e.g.  from

 boat  to  train  to  truck)  

Solution: Intermodal Shipping Container

…in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another  

A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.  

This spawned an Intermodal Shipping Container Ecosystem

•  90%  of  all  cargo  now  shipped  in  a  standard  container  •  Order  of  magnitude  reduc2on  in  cost  and  2me  to  load  and  unload  

ships  •  Massive  reduc2on  in  losses  due  to  theK  or  damage  •  Huge  reduc2on  in  freight  cost  as  percent  of  final  goods  (from  >25%  to  

<3%)  à massive  globaliza2on  •  5000  ships  deliver  200M  containers  per  year  

Static website Web frontend User DB Queue Analytics DB

Development VM QA server Public Cloud Contributor’s laptop

Let’s create a shipping container system for applications Mul2p

licity

 of  

Stacks  

Mul2p

licity

 of  

hardware  

environm

ents  

Production Cluster Customer Data Center

Do  services  and  apps  interact  appropriately?  

Can  I  migrate  

smoothly  and  quickly  

…that can be manipulated using standard operations and run consistently on virtually any hardware platform  

An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…  

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM QA Server Single Prod

Server Onsite Cluster

Public Cloud

Contributor’s laptop

Customer Servers

And eliminate the matrix from Hell

App  A              

Step One: Create a lightweight container (vs. VMs)

Hypervisor  (Type  2)  

Host  OS  

Server  

       

Guest  OS          

       

Bins/  Libs          

App  A’              

       

Guest  OS          

       

Bins/  Libs          

App  B              

       

Guest  OS          

       

Bins/  Libs          

App  A’  

Host  OS  

Server  

       

Bins/Libs  

       

App  A  

       

Bins/Libs  

       

App  B  

App  B’  

App  B’  

App  B’  VM  

Container  

Containers  are  isolated,  but  share  OS  kernel    and,  where  appropriate,  bins/libraries    

     

Guest  OS          

       

Guest  OS          

…result  is  significantly  faster  deployment,    much  less  overhead,  easier  migra2on,  faster  restart  

Step 2: Make the containers easy to use, standardized, interoperable, automatable

 •  With  Docker,  low  level  containers  get  the  

following:  -­‐  Ease  of    use,  tooling  -­‐  Re-­‐usable  components  -­‐  Ability  to  run  on  any  Linux  server  today:  physical,  virtual,  

VM,  cloud,  OpenStack,  +++  -­‐  (Stay  tuned  for  other  O/S’s)  

-­‐  Ability  to  move  between  any  of  the  above  in  a  maber  of  seconds-­‐  no  modifica2on  or  delay  

-­‐  Ability  to  share  containerized  components  -­‐  Interoperability  with  all  exis2ng  devops  tools  -­‐  Self  contained  environment—no  dependency  hell  -­‐  Tools  for  how  containers  work  together:  linking,  nes2ng,  

discovery,  orchestra2on,  ++  -­‐  “Containeriza2on”  is  really  “Dockeriza2on”  

 

Step 3: Make containers super lightweight

       

Bins/  Libs          

App  A  

Original  App  (No  OS  to  take  

up  space,  resources,  or  require  restart)  

App  Δ    

       Bins/  

App  A              

       

Bins/  Libs          

App  A’              

       

Guest  OS          

       

Bins/  Libs          

Modified  App    Copy  on  write  allows  us  to  only  save  the  diffs  Between  container  A  and  container  A’  

VMs  Every  app,  every  copy  of  an  

app,  and  every  slight  modifica2on  of  the  app  requires  a  new  virtual  server  

App  A              

       

Guest  OS                

       

Bins/  Libs          

Copy  of  App  

No  OS.  Can  Share  bins/libs  

App  A  

       

Guest  OS                

       

Guest  OS                

VMs   Containers  

A  set  of  slides  from  IBM:  Comparing  Docker  to  VMs  

9/22/14   16  

Step 4: Build an open platform

An  Open  Pla*orm  to  Build,  Ship,  and  Run  Distributed  Applica;ons

An Open Platform…

API  

   

Engine   Hub  open  source  soKware  at  the  heart    

of  the  Docker  plakorm    cloud-­‐based  plakorm  services  for  distributed  

applica2ons    

API  

An Open Platform…

Any  App  

Any  infrastructure  •  Physical  •  Virtual  cloud  

+  25K  apps  +  10K  projects  

API  

   

Engine   Hub  cloud-­‐based  plakorm  services  for  distributed  

applica2ons    

API  

open  source  soKware  at  the  heart    of  the  Docker  plakorm    

…to Build, Ship, and Run

Build   Ship  

Run  

Dev  

QA  

Source  

Staging  

Physical  

Virtual  

Cloud  

Infrastructure  Management  

Infrastructure  Management  

DockerFile  

Source  Code    Repository  

TEST  TEST  

TEST  TEST  

TEST  

GCE   RAX   IBM  

Mac/Win  Dev  Machine  

Boot2Docker  

Docker  

Analytics DB

Prod  Machine  

Linux  OS  

Docker  

Docker  

++  

Users   Collab  

Provenance   Policy  

Docker  Hub  

Registries  

Public   Curated   Private  

Docker  Hub  API  

Third  Party  Tools  

Prod  Machine  

Linux  OS  

Docker  

Docker  

Prod  Machine  

Linux  OS  

Docker  

Docker  

VM  

Docker  

Docker  

VM  

Docker  

Docker  

VM  

Docker  

Docker  

QA  Machine  

Linux  OS  

Docker  

Docker  

…Distributed Applications

Data

User DB

Queue

Analytics DB

API Endpoint

Web Frontend

Data

Background Workers

Development VM

QA Server

Public Cloud

Disaster Recovery

Contributor’s Laptop Production Servers

Production Cluster

Customer Data Center

Step 5: Build an Ecosystem

Official Repos & 14K+ Dockerized Apps

Community 540+ Contributors 250+ Meetups on Docker 7M Downloads 10K Projects on GitHub

Support

Enterprise Support Robust Documentation Implementation, Integration, Training Network of Partners

The Docker Platform Docker Engine

Docker Hub

Build, Ship, and Run

Partners

Content

Users

Official  Repos  &  25  K  Dockerized  Apps  

Partner Ecosystem

Who is using Docker?

And  hundreds  of  others….these  are  just  a  sample  of  those  who  have  published  use  cases  

Since DockerCon (as of 7/25)

• Major inflection points in almost all metrics •  Projects: From 6.5K to >10K •  Contributors: From 475à545 •  Downloads: From 2.75M to ~8M •  Boot2Docker Downloads: 400Kà600K •  Repositories: 14Kà25K

• Significant activity in enterprise pipeline (200+ F500 opportunities, including major financials, energy, bioinformatics, government)

•  Joint announcements, developments with virtually every major IT company

Conclusion

• Multiple forces are driving a reconsideration of how applications should be created, built, deployed, scaled, and managed

• We believe that the right approach is to decouple application management from infrastructure management

• Container based approach (vs. VM approach) provides right level of abstraction

•  Enables infrastructure to be managed consistently and stably •  Enables applications to be built flexibly and deployed flexibly •  Provides greater degree of visibility, control, and management of

what runs where and what components are allowed • Massive cost, speed, efficiency savings • Docker is becoming the standard for containerization

Learn More

•  All Videos and Talks from DockerCon: http://bit.ly/1AnvHFz •  Demo of DockerHub: see Ben Golub keynote from DockerCon, http://bit.ly/1xHqUL8

demo starts at 29:00 •  Sign up for a DockerHub account: https://hub.docker.com/ •  Security & Isolation: see above, also blog.docker.io •  Use Cases: https://docker.com/resources/usecases/ •  Docker project: www.docker.com/ •  Follow Docker on Twitter: twitter.com/docker •  Take the Docker interactive tutorial: https://docker.com/tryit/ •  Join Docker on IRC: botbot.me/freenode/docker/ •  Go to the Docker repository on GitHub: github.com/docker/docker/ •  Go to a meetup: https://www.docker.com/community/meetups/