creating effective docker images
TRANSCRIPT
![Page 1: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/1.jpg)
Creating Effective Images
Sr Technical Evangelist, AWS@abbyfuller
Abby Fuller
![Page 2: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/2.jpg)
• How do layers work?• The basics for building minimal images• High level best practices for Windows containers• Dockerfiles: the good, the bad, and the bloated• Let’s get (language) specific• Tools are here to help• Looking forward to the future
Agenda
![Page 3: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/3.jpg)
How do layers work?
![Page 4: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/4.jpg)
What are container layers?
read-only container layers
Thin read-write layer
![Page 5: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/5.jpg)
Why do I care how many layers I have?More layers mean a larger image. The larger the image, the longer that it takes to both build, and push and pull from a registry.
Smaller images mean faster builds and deploys. This also means a smaller attack surface.
![Page 6: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/6.jpg)
OK, so how can I reduce my layers?Sharing is caring.• Use shared base images where
possible• Limit the data written to the
container layer• Chain RUN statements• Prevent cache misses at build for as
long as possible
![Page 7: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/7.jpg)
Building minimal images: the basics
![Page 8: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/8.jpg)
A Dockerfile is a series of instructions for building images.
![Page 9: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/9.jpg)
Cache rules everything around me
CACHE
![Page 10: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/10.jpg)
Let’s start with a DockerfileFROM ubuntu:latestLABEL maintainer [email protected] apt-get update -y && apt-get install -y python-pip python-dev build-essentialCOPY . /appWORKDIR /appRUN pip install –r requirements.txtEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 11: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/11.jpg)
First step: choosing the right baseFrom the stock ubuntu image:
ubuntu latest 2b1dc137b502 52 seconds ago 458 MB
From python:2.7-alpine:
alpine latest d3145c9ba1fa 2 minutes ago 86.8 MB
![Page 12: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/12.jpg)
Slightly better- choose a different distroalpine latest d3145c9ba1fa 3.9 MBpython 3.6.1-slim 60952d8b5aeb 200 MBdebian latest 8cedef9d7368 123 MBpython 2.7-alpine b63d02d8829b 71.5 MBubuntu latest 0ef2e08ed3fa 130 MBfedora latest 4daa661b467f 231 MB
![Page 13: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/13.jpg)
When do I want a full base OS?(I do actually like Ubuntu!)
• Security• Compliance• Ease of development
![Page 14: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/14.jpg)
Let’s look at our original Ubuntu imageFROM ubuntu:latestRUN apt-get update -y && apt-get install -y python-pip python-dev build-essentialLABEL maintainer [email protected] . /appWORKDIR /appRUN pip install –r requirements.txtEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 15: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/15.jpg)
Simple changes, big resultsFROM python:2.7-alpineLABEL maintainer [email protected] . /appWORKDIR /appRUN pip install –r requirements.txtEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 16: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/16.jpg)
Fewer cache invalidations=smaller imagesFROM python:2.7-alpineLABEL maintainer [email protected] requirements.txt /appRUN pip install –r /app/requirements.txtCOPY . /appWORKDIR /appEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 17: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/17.jpg)
Got application code?FROM python:2.7-alpineLABEL maintainer [email protected] ADD requirements.txt /appONBUILD RUN pip install –r /app/requirements.txtONBUILD COPY . /appWORKDIR /appEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 18: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/18.jpg)
Let’s recap.TL;DR: layers represent filesystem differences. Layers add up quickly with big consequences.
![Page 19: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/19.jpg)
Some high-level best practices: Windows
![Page 20: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/20.jpg)
Port over existing VM workloadsConvert an existing Windows image:
ConvertTo-Dockerfile -ImagePath c:\docker\myimage.wim
Convert from VHD:
ConvertTo-Dockerfile -ImagePath c:\vms\test.vhd -Artifact IIS -ArtifactParam windows-container -OutputPath c:\windows-containercd c:\windows-containerdocker build -t windows-container . docker run -d -p 80:80 windows-container
![Page 21: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/21.jpg)
Some things to think aboutWatch what you build:
c: c:\ / /windows c:/windows
Building any of those PATHs will make your image very large!
![Page 22: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/22.jpg)
Avoid installing packages with MSIMSI installations are not space efficient. This is not the same as Linux distros, where you can add, use, and remove the installation files!
$ Windows/Installer/<package>.msi
Windows saves these files for uninstalls :(
![Page 23: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/23.jpg)
Coming up soon
Run Linux containers “as-is” on Windows Server!
![Page 24: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/24.jpg)
Here’s whats really cool thoughBuild and run everything the same, regardless of container OS, host OS, or tools. Just docker build and docker run.
![Page 25: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/25.jpg)
…but I’m not a Windows expertSo go to see Elton instead! He’ll talk on modernizing .NET apps at 17:10 for the Using Docker track. He literally wrote the book.
![Page 26: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/26.jpg)
Dockerfiles: the good, the bad, and the bloated
![Page 27: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/27.jpg)
Let’s start out bigFROM ubuntu:latestLABEL maintainer [email protected] apt-get update -y RUN apt-get install -y python-pip python-dev build-essentialCOPY . /appWORKDIR /appRUN pip install -r requirements.txtEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 28: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/28.jpg)
A little bit betterFROM ubuntu:latestLABEL maintainer [email protected] apt-get update -y && apt-get install -y python-pip python-dev build-essential –no-install-recommendsCOPY . /appWORKDIR /appRUN pip install -r requirements.txtEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 29: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/29.jpg)
Let’s try a different baseFROM python:2.7-alpineLABEL maintainer [email protected] . /appWORKDIR /appRUN pip install -r requirements.txtEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 30: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/30.jpg)
Or, let’s try a custom base containerFROM 621169296726.dkr.ecr.us-east-1.amazonaws.com/dockercon-base:latestLABEL maintainer [email protected] . /appWORKDIR /appEXPOSE 5000ENTRYPOINT ["python"]CMD ["application.py"]
![Page 31: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/31.jpg)
Use RUN statements effectivelyRUN apt-get update && apt-get install -y \ aufs-tools \ automake \ build-essential \ ruby1.9.1 \ ruby1.9.1-dev \ s3cmd=1.1.* \ && rm -rf /var/lib/apt/lists/*
![Page 32: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/32.jpg)
Switching USER adds layersRUN groupadd –r dockercon && useradd –r –g dockercon dockerconUSER dockerconRUN apt-get update && apt-get install -y \ aufs-tools \ automake \ build-essentialUSER rootCOPY . /app
![Page 33: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/33.jpg)
Avoid ADDing large filesBAD:
ADD http://cruft.com/bigthing.tar.xz /app/cruft/ RUN tar -xvf /app/cruft/bigthing.tar.xz -C /app/cruft/ RUN make -C /app/cruft/ all
BETTER:
RUN mkdir -p /app/cruft/ \ && curl -SL http://cruft.com/bigthing.tar.xz \ | tar -xJC /app/cruft/ && make -C /app/cruft/ all
![Page 34: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/34.jpg)
BESTRUN mkdir -p /app/cruft/ \ && curl -SL http://cruft.com/bigthing.tar.xz \ | tar -xvf /app/cruft/ \ && make -C /app/cruft/ all && \rm /app/cruft/bigthing.tar.xz
![Page 35: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/35.jpg)
Let’s get (language) specific
![Page 36: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/36.jpg)
A few language-specific best practicesUse the right tool: not every language needs to be built the same way.
•Where possible, use two images: one to build an artifact, and one from base•Official language images can be huge: more space effective to use a more minimal image, but there are tradeoffs
![Page 37: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/37.jpg)
First stop: GolangCompile, then COPY binary:$ go build -o dockercon . $ docker build -t dockercon .
Dockerfile:FROM scratch COPY ./dockercon /dockerconENTRYPOINT ["/dockercon"]
![Page 38: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/38.jpg)
Quick detour: what’s scratch?Special, empty Dockerfile.
Use this to build your own base images.Or, use to build minimal images that run a binary and nothing else:
FROM scratchCOPY hello /CMD [ “/hello” ]
Want more on scratch? Start here.
![Page 39: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/39.jpg)
Back to business: RubyOfficial images for Ruby are extra huge. A new base + a little extra work pays off.FROM alpine:3.2 LABEL maintainer [email protected] apk update && apk upgrade && apk add \ curl \ bashruby \ ruby-dev \ ruby-bundlerRUN rm -rf /var/cache/apk/*
![Page 40: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/40.jpg)
Next: node.jsIf you love yourself, .dockerignore npm-debug.log. Seriously.But most importantly, cache node_modules:COPY package.json . RUN npm install --production COPY . .
This way, only run npm install if package.json changes.
![Page 41: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/41.jpg)
Java!Multi-stage builds are your friend:FROM maven:3.5-jdk-8 as BUILDCOPY --from=BUILD
Like, Golang, this let’s you build an artifact in one stage, and simply run the binary in the second stage, resulting in more minimal final images.
More on multistage builds up next.
![Page 42: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/42.jpg)
Multi-stage buildsFROM ubuntu AS build-envRUN apt-get install make ADD . /src RUN cd /src && make
And for the second Dockerfile, copy from #1:
FROM busybox COPY --from=build-env /src/build/app /usr/local/bin/app EXPOSE 80 ENTRYPOINT /usr/local/bin/app
![Page 43: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/43.jpg)
Tools are here to help
![Page 44: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/44.jpg)
With great containers comes great responsibility
• Document!• Automate where possible• AWS has a few tenants for
services: secure, resilient, scaleable
• Lean on (the right) tools for a helping hand
![Page 45: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/45.jpg)
Docker Security Scan
![Page 46: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/46.jpg)
Docker Security Scan
![Page 47: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/47.jpg)
Docker Image + System PruneDocker image prune:
$ docker image prune –a
Alternatively, go even further with Docker system prune:
$ docker system prune -a
![Page 48: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/48.jpg)
The importance of garbage collection• Clean up after your containers! Beyond
image and system prune:• Make sure your orchestration platform
(like ECS or K8s) is garbage collecting:• ECS• Kubernetes
• 3rd party tools like spotify-gc
![Page 49: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/49.jpg)
Looking forward to the future
![Page 50: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/50.jpg)
But wait, there’s always more!• Always new and better things coming
• Linux and Windows Server• Official image are multi-platform• Always new and better minimal
images and operating systems coming out for containers
![Page 51: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/51.jpg)
So what did we learn?One takeaway: less layers is more.•Share layers where possible•Choose or build your base wisely•Not all languages should build the same•Keep it simple, avoid extras•Tools are here to help
![Page 52: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/52.jpg)
Useful linksDocker image documentationDocker scratchatsea sample appArun Gupta on smaller Java imagesElton Stoneman Windows DockerfilesAlpine (the base image from the examples)Running Linux containers on WindowsDocker garbage collectionImage cleanup in Amazon ECSImage cleanup in Kubernetesspotify-gc
![Page 53: Creating Effective Docker Images](https://reader033.vdocuments.net/reader033/viewer/2022061307/5a6476f67f8b9afc4d8b469b/html5/thumbnails/53.jpg)
Thanks!@abbyfuller