scale big with docker — moboom 2014
DESCRIPTION
TRANSCRIPT
![Page 1: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/1.jpg)
Scale Big With Docker
February 2014—Docker 0.8.0
![Page 2: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/2.jpg)
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
![Page 3: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/3.jpg)
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
![Page 4: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/4.jpg)
Deploy everything
● webapps● backends● SQL, NoSQL● big data● message queues● … and more
![Page 5: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/5.jpg)
Deploy almost everywhere
● Linux servers● VMs or bare metal● Any distro● Kernel 3.8 (or RHEL 2.6.32)
![Page 6: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/6.jpg)
Deploy reliably & consistently
● If it works locally, it will work on the server● With exactly the same behavior● Regardless of versions● Regardless of distros● Regardless of dependencies
![Page 7: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/7.jpg)
Deploy easily
● Don't learn a new CM tool● Reuse components and recipes● Don't install anything fancy locally
(One single local VM to handle all your apps)
![Page 8: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/8.jpg)
Deploy at scale
● Build once● Deploy anywhere● Deploy quickly
– Images are lightweight– Docker runtime itself is lightweight– Fast start/stop times
● Manage Docker with its REST API
![Page 9: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/9.jpg)
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
![Page 10: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/10.jpg)
Problem: shipping goods? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
![Page 11: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/11.jpg)
Solution:the intermodal shipping container
![Page 12: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/12.jpg)
![Page 13: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/13.jpg)
Problem: shipping code
djangoweb frontend ? ? ? ? ? ?
node.jsasync API ? ? ? ? ? ?
background workers ? ? ? ? ? ?
SQL database
? ? ? ? ? ?
distributed DB, big data
? ? ? ? ? ?
message queue
? ? ? ? ? ?
mylaptop
your laptop
QA staging prod on cloud VM
prod on bare metal
![Page 14: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/14.jpg)
Solution:the Linux container
![Page 15: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/15.jpg)
High level approach:it's a lightweight VM
● I can SSH into it● I have root access● I can apt-get/yum install packages ● I don't see the host system (or other containers)
« Machine Container »
![Page 16: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/16.jpg)
Low level approach:it's chroot on steroids
● I can run a single program● container = isolated process(es)● share kernel with host● no device emulation → no overhead
« Application Container »
![Page 17: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/17.jpg)
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
![Page 18: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/18.jpg)
![Page 19: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/19.jpg)
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
![Page 20: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/20.jpg)
One-time setup
● On your servers (Linux)– Packages (Ubuntu, Debian, Fedora, Gentoo, Arch...)
– Single binary install (Golang FTW!)
– Easy provisioning on Rackspace, Digital Ocean, EC2, GCE...
● On your dev env (Linux, OS X, Windows)– Vagrantfile
– boot2docker (25 MB VM image)
– Natively (if you run Linux)
●
![Page 21: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/21.jpg)
The Docker workflow 1/2
● Work in dev environment(local machine or container)
● Other services can be perfect copies of prod● Whenever you want to test « for real »:
– Build in seconds
– Run instantly
![Page 22: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/22.jpg)
The Docker workflow 2/2
● Push your build to a registry (public or private)● Run it in CI/CD● Run it in production● Rejoice
Something goes wrong?Just re-run previous version (it's still there!)
![Page 23: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/23.jpg)
I'm starting a new project!
● Each service will be in its own container(s)● Build an image for each service● Pin dependencies (packages etc.) accurately● Link services together
→ Recommended workflow!
![Page 24: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/24.jpg)
I'm Dockerizing some code!
● Making a single service runnable on Docker● Create a « Dockerfile »
(Think Makefile, Vagrantfile...)● If it's on GitHub, create a Trusted Build
(will automatically build ready-to-run images)● In any case, register it on index.docker.io
![Page 25: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/25.jpg)
I'm migrating to Docker!
● Easy if there is only one service per server– Dockerize the service
– Containerize administrative functions (logs...)
● Otherwise, try to split services● You don't need to move everything to Docker● Don't use containers as VMs, even if you can!
![Page 26: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/26.jpg)
I'm already using Chef/Puppet/Salt/Ansible/...!
● Plan A: generic images– Put your configuration management tool in an image
– Run that image just like you would run a VM
– Great for mixed environments
● Plan B: specialized images– Put your configuration management tool in an image
– Run that image, freeze the result, run it!
– Faster and safer
![Page 27: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/27.jpg)
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
![Page 28: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/28.jpg)
Authoring imageswith run/commit
![Page 29: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/29.jpg)
1) docker run ubuntu bash
2) apt-get install this and that
3) docker commit <containerid> <imagename>
4) docker run <imagename> bash
5) git clone git://.../mycode
6) pip install -r requirements.txt
7) docker commit <containerid> <imagename>
8) repeat steps 4-7 as necessary
9) docker tag <imagename> <user/image>
10) docker push <user/image>
![Page 30: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/30.jpg)
Authoring imageswith run/commit
● Pros– Convenient, nothing to learn
– Can roll back/forward if needed
● Cons– Manual process
– Iterative changes stack up
– Full rebuilds are boring, error-prone
![Page 31: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/31.jpg)
Authoring imageswith a Dockerfile
![Page 32: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/32.jpg)
FROM ubuntu
RUN apt-get -y updateRUN apt-get install -y g++RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101CMD ["/usr/local/bin/couchdb"]
docker build -t jpetazzo/couchdb .
![Page 33: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/33.jpg)
Authoring imageswith a Dockerfile
● Almost nothing to learn● Rebuilds are easy● Caching system makes rebuilds faster● Single file to define the whole environment!
![Page 34: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/34.jpg)
Outline
● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future
![Page 35: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/35.jpg)
Docker: the community
● Docker: >330 contributors● Docker inc.: <20 engineers ● latest milestone (0.8): >120 contributors● ~50% of all commits by external contributors● GitHub repository: >1400 forks
...and counting!
![Page 36: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/36.jpg)
Coming Soon
● Network acceleration● Container-specific metrics● Consolidated logging● Plugins (compute backends...)
Those things are already possible,but will soon be part of the core.
![Page 37: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/37.jpg)
Docker 1.0
● Multi-arch, multi-OS● Stable control API● Stable plugin API ● Resiliency● Signature● Clustering
![Page 38: Scale Big With Docker — Moboom 2014](https://reader034.vdocuments.net/reader034/viewer/2022051819/54c6bb634a7959b72d8b4651/html5/thumbnails/38.jpg)
Thank you! Questions?
http://docker.io/
http://docker.com/
https://github.com/dotcloud/docker
@docker
@jpetazzo