container orchestration wars (2017 edition)

© 2017 Mesosphere, Inc. All Rights Reserved. 1

CONTAINER ORCHESTRATION WARS( 2017 Edition )


KARL WHO?

WHERE?- Mesosphere- Pivotal

WHAT?- DC/OS- Kubernetes- CloudFoundry- BOSH

- github.com/karlkfi

- twitter.com/karlkfi

- linkedin.com/in/karlkfi

- karl.isenberg.us


INFRASTRUCTURE EVOLUTIONPLATFORM SPECTRUM

CONTAINER PLATFORM LAYERSCONTAINER PLATFORM CAPABILITIESCONTAINER PLATFORM COMPARISON

HOW DO I CHOOSE?


INFRASTRUCTUREEVOLUTION


MONOLITHAT SCALE

ONLINELATENCY ROUTED

MULTI-REGIONLOAD BALANCED

MULTI-ZONEREPLICATED

AUTO-SCALEDDATA REPLICATION

DATA SYNCHRONIZATION

AWS Simple IconsBy Amazon Web Services, Inc., CC BY-SA 3.0

https://aws.amazon.com/architecture/icons/

https://aws.amazon.com/architecture/icons/


MICROSERVICESAT SCALE

Wheel of DoomFrom A Journey into Microservices by Hailo

https://sudo.hailoapp.com/services/2015/03/09/journey-into-a-microservice-world-part-3/


INFRASTRUCTUREEVOLUTIONAPPLICATION

PROVISIONING

ON

BARE METAL


“Ultimately, utility cloud providers have exposed how difficult it is to properly operate data centers — and reminded all of us that the

ability to expertly operate infrastructure is what really fuels the consumption of open

source infrastructure.”

Brian Stein(Rackspace VP - 2017)



PROVISIONING

ON

VIRTUAL INFRASTRUCTURE PLATFORM

(IaaS)


“The goal of Cloud Foundry is to put more of the controls back in the hands of developers so they can self-provision, so there aren’t a lot of roadblocks in

their way. But it gives a lot of guardrails.”

Chip Childers(Cloud Foundry Foundation CTO - 2017)



PLATFORM(PaaS / aPaaS)

ON

INFRASTRUCTURE PLATFORM

(IaaS)


“...traditional “PaaS” roles have now been taken over by containers… The piece that is left for PaaS is the part that was always the

most important part of PaaS in the first place, and that’s the opinionated developer

experience.”

Brendan Burns(Kubernetes Cofounder - 2017)



CONTAINERORCHESTRATION

ON


(IaaS)



CONTAINERPLATFORM

(CaaS)

ON


(IaaS)



CONTAINERPLATFORM

(CaaS)

ON

BARE METAL


“If your PaaS can efficiently start instances in 20ms that run for half a second, then call it serverless.”

Adrian Cockcroft(AWS VP - 2016)



FUNCTIONPLATFORM

(FaaS)

ON


(IaaS)



FUNCTIONPLATFORM

(FaaS)

ON

CONTAINERPLATFORM

(CaaS)



FUNCTION PLATFORM(FaaS)

ON

CONTAINER PLATFORM(CaaS)

ON

INFRASTRUCTURE PLATFORM(IaaS)


PLATFORMSPECTRUM


PLATFORM SPECTRUM

INFRASTRUCTUREPLATFORM

( IaaS )Virtual Machines

DisksNetworksFirewalls

SOFTWAREPLATFORM

( SaaS )Whatever

You Want

( to pay for )

FUNCTIONPLATFORM

( FaaS )Actions

/tmpTriggers

Gateways

CONTAINERPLATFORM

( CaaS )Containers

VolumesIPs & Ports

Load Balancers

APPLICATIONPLATFORM

( PaaS / aPaaS )Apps/tmp

80/443Routes

Abstraction

Flexibility Velocity

Low Level High Level


PLATFORM EXAMPLES

INFRASTRUCTUREPLATFORM

( IaaS )OpenStack

vSphereAzure Stack VMs

AWS EC2GCE

Azure VMs

SOFTWAREPLATFORM

( SaaS )

BYO

SalesforceOracle

SAP

FUNCTIONPLATFORM

( FaaS )OpenWhisk

FissionIron.io

LambdaGCF

Azure Functions

CONTAINERPLATFORM

( CaaS )Kubernetes

DC/OSDocker Datacenter

GKEECSACS

APPLICATIONPLATFORM

( PaaS / aPaaS )CloudFoundry

OpenShiftWaveMaker RAD

HerokuPCF

Jelastic

HOSTED


CONTAINERPLATFORMLAYERS


CONTAINER ORCHESTRATION

Apps

Container Runtime

Container Orchestration

- User workloads

- Distributed container management

- Local container management

Infrastructure

ServicesJobs

- Container agnostic infrastructure


CONTAINER PLATFORM

Apps

Container Storage

Container Runtime


System Services

- User workloads

- System management & service enablement

- Distributed container management

- Local container management

- Container aware infrastructureContainer Networks

Infrastructure

ServicesJobs

- Container agnostic infrastructure


DISTRIBUTED OPERATING SYSTEM

Apps

Container Storage

Container Runtime


System Services

Container Networks

Infrastructure

ServicesJobs

Security&

Multitenancy

Logging&

Metrics&

Monitoring


CONTAINERPLATFORMCAPABILITIES


CONTAINERS- Resource Isolation- Resource

Constraints- Process Tree- Environment

Isolation- Shell / Exec

IMAGES- Build- Layers- Download- Cache- Publish- Prune

NETWORKS- Container- Bridge- Host- Virtual- Overlay- Remote- User-defined- Port Mapping

DATA VOLUMES- Ephemeral- Host- Backup / Restore- Copy In / Out- Shared

RUNTIME CAPABILITIES


ORCHESTRATION CAPABILITIES

SCHEDULING- Placement- Replication/Scaling- Readiness Checking- Resurrection- Rescheduling- Rolling Updates- Collocation- Daemons- Cron Jobs

RESOURCE MGMT- Memory- CPU- GPU- Ephemeral Volumes- Remote Persistent Volumes- Local Persistent Volumes- Ports- IPs (per container)

SERVICE MGMT- Labels- Groups/Namespaces- Dependencies- Load Balancing (L7)- VIPs (L3/L4 LB)- DNS- DNS Proxy- Secrets- Config Mgmt


OPERATIONAL CAPABILITIES

ADMINISTRATION- GUI- CLI- Metrics API- Logs API- Events API- Rolling Upgrades- Backups & Restores

MULTI-INFRASTRUCTURE- Multi-cloud- Multi-zone- Multi-region- Hybrid-cloud- Federation

SYSTEM SERVICES- Auto-Scaling- Package Management- Service Catalog- Service Brokers- Admin Proxy- API Gateway


PLATFORM CAPABILITIES

CONTAINER NETWORKS- Overlay- Routing- Network Address

Translation (NAT)- Firewalls- Access Control Lists- Quality of Service

CONTAINER STORAGE- Local Volumes- Remote Volumes- Block Storage- File System Storage- Object Storage

PLATFORM DATABASES- Lock Service- Key-Value Database- Relational Database- Time Series Database


SECURITY- User Accounts- Service Accounts- System/User Space- E2E Encryption- Non-root User Workloads- Audit Logging- Public Key Infrastructure- Certifications

MULTITENANCY- User Groups- Permissions- RBAC- ABAC- Resource Sharing

- FIFO- Fair- Quotas

- Branding- Quality of Service

PLATFORM CAPABILITIES


NON-FUNCTIONAL QUALITIES

SCALABILITYPerformance, Responsiveness, Efficiency

AVAILABILITYFault Tolerance, Robustness, Reliability, Resilience, Disaster Recovery

FLEXIBILITYFormat Support, Interoperability, Extensibility, Container Runtimes

USABILITYFamiliarity, Maintainability, Compatibility, Debuggability

PORTABILITYHost OS, Cloud, Bare-Metal, Hybrid

SECURITYEncryption Quality, Vulnerability Process, Fast Patching, Backporting


CONTAINERPLATFORMCOMPARISON


Docker

Kontena

ECS

Nomad

Kubernetes

Rancher Cattle

DC/OS

OpenShift

TODAY’S CONTESTANTS


CloudFoundry GKE

Heroku

Tectonic

Juju

Triton

HONORABLE MENTIONS

© 2017 Mesosphere, Inc. All Rights Reserved.

Placement

Replication/Scaling

Readiness Checking

Resurrection

Rescheduling

Rolling Updates

Collocation

37

SCHEDULINGCONTAINER PLATFORM COMPARISON (06/2017)

Kubernetes DC/OS ECS Docker Nomad Cattle Kontena

Included New/External/Partial/Experimental

OpenShift

Daemons

Cron Jobs


RESOURCE MANAGEMENTCONTAINER PLATFORM COMPARISON (06/2017)

Memory



CPU

GPU

Disk Space

Ephemeral Volumes

Remote Persistent Vols

Local Persistent Vols

Ports

IPs (per container)

OpenShift


SERVICE MANAGEMENTCONTAINER PLATFORM COMPARISON (06/2017)

Labels



Groups/Namespaces

Dependencies

Load Balancing (L7)

VIPs (L3/L4 LB)

DNS

DNS Proxy

Secrets

Config Mgmt

OpenShift


HOW DO I CHOOSE?


KUBERNETES- Huge community- Solid API- Some assembly required- Multitude of vendors/installers

OPENSHIFT- Application platform based on Kubernetes- Always trailing Kubernetes releases- No assembly required- Open core, enterprise platform

DC/OS- Runs native applications (non-Docker)- Specialized in data services- Ambitious scope (on-prem AWS)- No assembly required- Open core, enterprise platform

DOCKER- Huge community- Fast moving API- Integrated orchestration and runtime- Recent pivot from runtime to orchestration- Open core, enterprise platform

HEAVY HITTERS


EC2 CONTAINER SERVICE (ECS)- Hosted-only solution- Tight integration with AWS services- Closed platform

RANCHER CATTLE- Gateway to Kubernetes, Mesos, and Docker- Open platform, enterprise support

NOMAD- Provisioner with orchestration features- Runs native applications (non-Docker)- Tight integration with Vault and Consul- Some assembly required- Open platform, enterprise support

KONTENA- Simple to set up- No assembly required- Open core, enterprise platform

LITE ORCHESTRATION


CONSIDERATIONS- Which is more important to you: velocity or flexibility?- Do you want an opinionated application platform?- Do you need to support Big Data initiatives and pipelines?- Do you want a hosted solution?- Are you willing to build out your own integrations?- Do you need on-prem & hybrid capabilities?- Do you want to avoid infrastructure lock-in?- Are you already invested in a specific infrastructure?- Are you already invested in a specific operating system?- Do you need federation and multi-region support?- Do you want multitenancy or is multi-instance good enough?- How important are seamless automated rolling upgrades?- How many nines do your customers need?- How important is reverse compatibility & API stability?- Do you need to support non-Docker workloads?

HOW DO I CHOOSE?


KARL ISENBERGkarlkfi

container orchestration wars (2017 edition)

Software