applied microservice security - goto conference · applied microservice security adrian mouat goto...

AppliedMicroserviceSecurity

AdrianMouat

GOTOAmsterdam2017

AppliedMicroserviceSecurity

Howtobuildanddeployamicroservicesecurely

Withthemajorcaveatthatnothingontheinternetissecure

Andthatbestpracticesarestillevolving

ExampleApplication

Architecture

"Bad"IdentidockDocker leFROMpython

RUNpipinstallFlaskuWSGIrequestsredisWORKDIR/appCOPYapp/appCOPYcmd.sh/

EXPOSE90909191

CMD["/cmd.sh"]

EvilNo.1Noversionnumbersforsoftware

Breaksrepeatabilityandprovenance

Whichversion?Manypackagesusesemver

MAJOR.MINOR.PATCH

Toospeci candriskmissingsecurityupdates

Toocoarseandriskbreakingchanges

ConsiderMAJOR.MINOR

"Versioned"IdentidockDocker le

FROMpython:3.6

COPYrequirements.txt/requirements.txtRUNpipinstall-r/requirements.txtWORKDIR/appCOPYapp/appCOPYcmd.sh/

EXPOSE90909191

CMD["/cmd.sh"]

requirements.txtappdirs>=1.4,<1.5certifi==2017.4.17chardet>=3.0,<3.1click==6.7Flask>=0.12,<0.13idna==2.5...

Aside:TotalRepeatabilty

Currentlynotpossiblewithdockerbuild

Alsopackagescanbeaproblem

Canrunownmirrore.g.

Bazel

"Buildtoolsmustallowustoensure

consistencyandrepeatability"

SiteReliabilityEngineering

https://www.aptly.info/

EvilNo2.NotSettingaUserIdentidockisrunningasroot

Changetolessprivilegeduser

IdentidockDocker lewithUser

FROMpython:3.6

RUNgroupadd-ridentidock&&useradd-r-gidentidockidentidock

COPYrequirements.txt/requirements.txtRUNpipinstall-r/requirements.txtWORKDIR/appCOPYapp/appCOPYcmd.sh/

USERidentidock

EXPOSE90909191

CMD["/cmd.sh"]

ChangingUseratStart-up#!/bin/shset-e

if["$1"='redis-server'-a"$(id-u)"='0'];thenchown-Rredis.execgosuredis"$0""$@"fi

exec"$@"

gosusudoforcontainers

su-execinAlpine

https://github.com/tianon/gosu

$dockerrun-itdebian-with-sudosudo-unobodypsauxUSERPID%CPU%MEMVSZRSSTTYSTATSTARTTIMECOMMANDroot10.00.0410963048?Ss+20:050:00sudo-unobodynobody70.00.0175002068?R+20:050:00psaux

$dockerrun-itdebian-with-gosugosunobodypsauxUSERPID%CPU%MEMVSZRSSTTYSTATSTARTTIMECOMMANDnobody10.00.09084800?Rs+20:060:00psaux

Would-beEvilNo3.NotVerifyingDownloads

Doesn'toccurinthisDocker le

EssentialforProvenance

ENVREDIS_DOWNLOAD_URLhttp://download.redis.io/releases/redis-3.2.9.tar.gzENVREDIS_DOWNLOAD_SHA6eaacfa983b287e440d0839ead20c2231749d5d6b78bbe0e0ffa3a890c59ff26...wget-Oredis.tar.gz"$REDIS_DOWNLOAD_URL";\echo"$REDIS_DOWNLOAD_SHA*redis.tar.gz"|sha256sum-c-;\...

https://github.com/docker-

library/redis/blob/master/3.2/Docker le

ImageNamingandMetadata

Don'ttagyourimages"latest"

Addmetadataforimageprovenance

https://github.com/opencontainers/image-

spec/blob/master/annotations.md

Docker leFROMpython:3.6

...

CMD["/cmd.sh"]

#https://github.com/opencontainers/image-spec/blob/master/annotations.mdARGCREATEDARGREVISIONARGNAMELABELorg.opencontainers.image.created=$CREATED\org.opencontainers.image.revision=$REVISION\org.opencontainers.image.name=$TAG\org.opencontainers.image.source="git@github.com:amouat/identidock.git"

BuildScriptTAG=identidock:v2.0.1dockerbuild-fDockerfile_labelled\--build-argCREATED="$(date--rfc-3339=s)"\--build-argREVISION="$(gitrev-parseHEAD)"\--build-argTAG=$TAG\-t$TAG.

PushingandPullingSecurely

Notaseasyasitsounds

DockerContentTrust

Digests

DockerContentTrust

TurnonwithexportDOCKER_CONTENT_TRUST=1

Imagescanthenbe"signed"

Pulledimagescheckedagainstpublisherspublickey

Pushingimagesrequirescreationofsigningkeys

"TOFU"

Requiresnotaryserver

ProbablyDockerHub

Digests

Immutablecontent-basedhashofimage

Canpullbydigest

dockerpulldebian@sha256:72f784399fd2719b4\

cb4e16ef8e369a39dc67f53d978cd3e2e7bf4e502c7b793

DigestsTAG=myregistry.com/identidock:v2.0.1dockerbuild-fDockerfile_labelled\--build-argCREATED="$(date--rfc-3339=s)"\--build-argREVISION="$(gitrev-parseHEAD)"\--build-argTAG=$TAG\-t$TAG.

#Testing...

dockerpush$TAG

DIGEST=$(dockerinspect-f'{{index.RepoDigests0}}'$TAG)

#dockerserviceupdate--image$DIGESTidentidock#kubectlsetimage...

TheNo1.Vulnerability?Runningout-of-datesoftware

Don'tRunVulnerableSoftware

Keeppackagesuptodate

Useasecurityscanner

KeepPackagesup-to-dateUsetooling

npmoutdated,piplist--outdated

Auto-builds&hooks

watchtower

SecurityScanning

ScanningServicesClair

Opensource

Designedtointegrateintowork ow

DockerSecurityScanning

Neuvector

Twistlock

AquaSecurity

Integrateintowork owMosttoolsareAPIbased

scanautomaticallyonpush

DockerCompose

version:"3"

services:proxy:image:nginx:1.13volumes:-./default.conf:/etc/nginx/conf.d/default.confports:-"80:80"

identidock:image:amouat/identidock:2.0environment:ENV:PROD

dnmonster:image:amouat/dnmonster:1.0

redis:image:redis:3.2

Read-onlyFS

$dockerrun--read-onlydebiansh-c'echo"x">/file'sh:1:cannotcreate/file:Read-onlyfilesystem

Read-onlyFSCanmountvolumesforspeci c les

dockerrun-d-p80:80--read-only\--tmpfs/var/cache/nginx/--tmpfs/run\nginx

Minimaldistrodebian123MB

alpine5MB

AdvantagesSmallerattacksurface

Easiertodistribute

DisadvantagesSmallerpackagemanager

muslvsglibc

Lessdebuggingtools

Nobash

Smallersetofmaintainers?

DockerComposeAlpineversion:"3"

services:proxy:image:nginx:1.13-alpine

...

redis:image:redis:3.2-alpine

Aside:BinaryonlycontainersStaticallycompilecode

Go,C,Rust...

Placeintoscratchimage

Super-minimal

Aside:Aside:UnikernelsTheLinuxkernelislarge

Lotofitisuneeded

Floppydrivers?

Multitenancy

Mergekernelandapplication

runonH/Worhypervisor

NetworkSegregationRedisanddnmonsterdon'ttalktoeachother

Sotheyshouldn'tbeableto!

NetworkSegregationservices:proxy:...networks:-frontend

identidock:...networks:-frontend-database-backend

dnmonster:image:amouat/dnmonster:1.0networks:-backend

redis:image:redis:3.2-alpinenetworks:-database

networks:-database-frontend-backend

LimitingResourcesMemoryismostimportant

CPUsharedbydefault

LimitingResources...redis:image:redis:3.2-alpinedeploy:resources:memory:200M

networks:-database...

Aside:Capabilities&SeccompLimitsystemcalls

Aside:LinuxSecurityModules

AppArmor

SELinux

HostSecuritySameasbefore

Keepup-to-date

Sticktowhatyouknow

docker-bench

Aside:ContainerDistrosRancherOS

CoreOS

Atomic

LinuxKit

Aside:SecureKernelsGRSecurity

PaX

SecretsPasswords,tokens,keys

Cangettrickywithms

SecretsEnvironmentvariableswork

butkindaicky

Swarm&Kuberneteshavesolutions

Vault

MonitoringEssentialwithmicroservices

lotsofsolutions

Prometheus

Checklist

MustKeepsoftwareupdated

Runasunprivilegeduser

Establishprovenanceandrepeatability

ShouldRunwithread-onlyfs

Scanforvulnerabilities

Enforcenetworksegregation

Runminimalcontainerdistro

CouldUsevaultforsecrets

Restrictcapabilitiesandresources

Runaminimalhostdistro

Runasecurityenhancedkernel

ConclusionDon'ttrytodoeverythingatonce

Easywins

Containersaddsecurity