aws re:invent 2016: introduction to container management on aws (con303)
Post on 16-Apr-2017
702 Views
Preview:
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Brandon Chavis
December 2, 2016
CON303
Introduction to Container
Management on AWS
Containers on AWS
Hypervisor Containers
Server
Guest OS
Bins/LibsBins/Lib
s
App2App1
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
“Okay, let’s run our containers on AWS.”
Choose your own adventure!Start!
You ==
Wizard?
Legacy
Apps?
Big
Data?
Cluster >
10000?
Love
Docker
CLI?
Cluster >
1000?
Cluster >
200?
Huge IT
team?
No
Yes
No
Yes
Inspiration:
http://adrianotto.com/wp-
content/uploads/2016/02/ch
oose-adventure-
1024x574.png
No Yes
Yes
YesNoYesNoYes
Yes
AWS Container Ecosystem
• Foundation
• Monitoring
• CI/CD
• Security
• PaaS
AWS Customers Have Options
Today we’ll cover:
• Amazon ECS
• CoreOS Tectonic (Kubernetes)
• Mesosphere DC/OS (Mesos)
• Docker Datacenter
Container Orchestration
Instance Instance Instance
OS OS OS
Container Runtime Container Runtime Container Runtime
App Service App App Service Service
Container Orchestration
Container Orchestration
myJob: {Cpu: 10Mem: 256}
Orchestrator
Schedule
Run “myJob”
Container Orchestration
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orc
he
str
ationService Management
Availability
Lifecycle
Discovery
Container Orchestration
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orc
he
str
ation
Scheduling
Placement
Scaling
Upgrades
Rollbacks
Container Orchestration
Instance/OS Instance/OS Instance/OS
App Service App App Service Service
Service Management
Scheduling
Resource Management
Orc
he
str
ation
Resource
Management
Memory
CPU
Ports
Schedulers
Cluster Machines
Cluster StateInformation
Monolothic Two-Level Shared State
No Concurrency Pessimistic Concurrency(offers)
OptimisticConcurrency
(transactions)
Scheduling Logic
Kubernetes
Replication Controller
API Server
Kubernetes Master
Kubelet KubeProxy Docker
ContainerContaine
r
Pod Pod
Kubelet KubeProxy Docker
Container
Container
Pod Pod
Kubernetes Cluster
etcd
KubernetesContainer orchestration or the OS for distributed compute
Upstream Kubernetes
● Container orchestration
● Horizontal scale
● High availability
● Service discovery & load
balancing
● Basic command line
operations
kubelet
default addons
(DNS)
controller manager
proxy
api server
scheduler
etcd
system kubelet
OS
Docker
Rkt
flannel
Kubernetes
Storage & Compute
apps/container/microservices
OS
Tectonic Extends
Upstream Kubernetes
● Installer
● Management console
● Painless updates
● Cluster scaling
● Disaster recovery
● Alerts and logging
● Security (integrated)
● Integration across
environments
Extending Kubernetes for the Enterprise
Security Mgmt
Kubernetes
CoreOS Linux
Cloud Integration
Update
Container Registry
Storage & Compute
apps/container/microservices
Tectonic
Kubernetes Management
● Single console across
environments
● AWS Auto Scaling
● Monitoring and logging
● Audit log
● Alert configuration
● View compute usage over
time and isolate by
namespace/cluster
Extending Kubernetes for the Enterprise
Security Mgmt
Kubernetes
CoreOS Linux
Cloud Integration
Update
Container Registry
Storage & Compute
apps/container/microservices
Tectonic
Operating System
● Packaged with CoreOS
Linux, tested against
Docker and rkt
● Rolling upgrades of OS
Extending Kubernetes for the Enterprise
Security Mgmt
Kubernetes
CoreOS Linux
Cloud Integration
Update
Container Registry
Storage & Compute
apps/container/microservices
Highlighted AWS Features
• Production-ready installs: Built from our learnings from kube-aws,
the Tectonic AWS installer installs a highly available self-hosted
Kubernetes cluster in your environment
• Managed upgrades on AWS: CoreOS applies CoreUpdate
functionality to Kubernetes.
• Disaster recovery: Managed backups and restore of Kubernetes
clusters via etcd to s3
Kube-AWS
• github.com/coreos/kube-aws
• kube-aws is a templating engine for AWS Cloudformation templates
• All assets (CloudFormation, userdata for instances) are declarative
templates that can be checked into git and version controlled
• Uses Amazon KMS to encrypt all secrets before putting them into assets
files. Secrets are unlocked once the machines boot into their IAM roles
• Spreading of workers and control plane nodes across AZs
Customer Use Case
Planet Labs launches and manages a fleet
of earth-imageing sattelites
Manages 100 million images and supports
tens of terabytes of data every day
Building a whole-earth dataset
Before:
• Operational overhead from
30 million jobs and 15,000
instances
• Challenges with
inconsistent packaging
and deployments across
teams
After:
• Kubernetes + CoreOS on
AWS
• Containers allowed
consistent app packaging
• Less rigid dependencies
• Less focus on infrastructure
Customer Use Case
Use Tectonic if:
• You want a Kubernetes experience tailored for
AWS
• You want to purchase support for Kubernetes
• You may want to consider another container
runtime (i.e. rkt)
Kube-aws:
Docker Datacenter (DDC)
Docker Universal Control Plane
Integrated
Security
Docker EngineContainer runtime, orchestration, networking, volumes, plugins
Docker Trusted Registry
Operating
Systems Config Mgt Monitoring LoggingCI/CD ..more..Images Networking Volumes
VirtualizationPublic Cloud Physical
Docker Datacenter
Universal Control Plane:
• Cluster manager
• Etcd for state management
• Swarm for scheduling
• LDAP integration, service
discovery, load balancing
built in
Docker Trusted Registry
• Private repository
• Image Signing
• LDAP/RBAC support
• S3 backend support
Docker Datacenter (DDC)
UCP
Manager
UCP
Manager
UCP
Manager
Internal distributed store
DTR
Replica
Worker
Raft consensus group
Image Registry
BYO TCP Load Balancer
UCP
WorkerUCP
Worker
UCP
Worker
UCP
Worker
push / pull
Admin / UserDeploy / manage
LDAP/AD
Monitoring
Logging
External CA
Image Storage
DTR
Replica
Worker
DTR
Replica
Worker
• Swarm Mode (Docker 1.12) Support
• Point and click UI to manage nodes,
services, containers, and networks
• Highly Available ( 3, 5, or 7 Controllers)
• Secure access control with LDAP/AD
support and granular RBAC
• Docker Content Trust : Image Signing
and Runtime Enforcement
UCP Manager
CS Docker Engine
Swarm Mode Manager Node
Monitoring
Web UI
Log Aggregator
Access Control
Auth Server
Docker Swarm
Docker Datacenter: UCP
DDC Quickstart – AWS Architecture
Availability Zone Availability Zone
Private subnet
Public subnet Public subnet
Private subnet
EC2 instances
EC2 instances
Elastic Load Balancing
Elastic Load Balancing
InternetgatewayVPC NAT
gateway
VPC NAT gateway
Internet gateway
DDC Quickstart – AWS Architecture
Availability Zone Availability Zone
Private subnet Private subnet
Swarm NodesSwarm Nodes
DTR Controller
DTR Controller
DTR Controller
UCP Controller
UCP Controller
UCP Controller
Docker Datacenter:Customer Use CaseADP leverages Docker Datacenter on AWS to deliver Security And Scale For Both Legacy
And Microservices Applications
Challenge
Refactor legacy monolithic applications to microservices
Disparate systems at scale - hundreds of products, hundreds of thousands of clients
Solution
Leverages Docker Datacenter on AWS - UCP, DTR, CS Engine
Benefits
Adopt hybrid strategy – mix of big and small containers for any application creating an
evolutionary path forward to microservices on the Cloud.
Swarm spans across public and private infrastructure and across applications, allowing the
swarm to the abstraction layer between physical compute and the application teams.
Apps running in hardened containers, image signing, and multiple DTRs ensure a secure
environment
“Docker’s CaaS approach will enable us to drive transformation across the entire application
lifecycle from development to operations. With Docker, we will be able to ensure application
portability, whether it is between dev and ops or between the datacenter and the cloud.”
Use Docker Datacenter if…
• The “Docker Native” stack is important to you
• Your development workflow is built around Compose and the
Docker CLI
• You want Commercial Support for the Docker Engine
DDC Quickstart:
MesosMaster
Marathon
ZooKeeper
Mesos + Marathon
Mesos Slaves
Long Running Tasks
Jobs
Coordination & Configuration
● Collection of services to power your apps
● Service installation and lifecycle management
36
DC/OS Universe
Mesosphere Enterprise DC/OS
Large install base on premise and in the cloud
● Container & big data operations
● Security, fault tolerance & high availability
● Open core & production proven at scale
Container orchestration
Datacenter aggregation
Turnkey lifecycle mgt
User Interface & CLI
SecurityAdvanced
NetworkingMonitoring &
OperationsCompliance
Elastic
● Supports hybrid deployments
Mesosphere DC/OS
Mesosphere DC/OS
Big Data
Analytics
Stateful
Service
Big Data and Stateful Apps
Mesosphere Enterprise DC/OS
Container
App
Container
App
CaaS PaaS
● Uses 66% less infrastructure
● Simplified operations
● Turnkey install of datacenter-wide services
● Easier to experiment with new tech (e.g., Spark)
Mesosphere DC/OS
Mesosphere DC/OS: Customer Use Case
• Production event handling service
• Deployed to 3 AWS Regions
• 40 seconds to deploy new builds
• Infrastructure Density: 66% less instances
• Cost Reduction: 57% less spend
Use DC/OS If….
• You prefer technology that has been around longer than your
toddler
• You want to take advantage of the “DC/OS Universe”
• You want to also manage big-data applications with your
orchestration tool
DC/OS on AWS:
DockerTask
Container Instance
Amazon ECS
Container
ECS Agent
ELB
Internet
ELB
User / Scheduler
API
Cluster Management Engine
Task
Container
DockerTask
Container Instance
Container
ECS Agent
Task
Container
DockerTask
Container Instance
Container
ECS Agent
Task
Container
AZ 1 AZ 2
Key/Value Store
Agent Communication Service
Amazon ECS
Amazon ECS
Container Management
at Any Scale
Flexible Container
Placement
Integration
with the AWS Platform
Components of ECS
Task
Actual containers running on Instances
Task Definition
Definition of containers and environment for task
Cluster
Fleet of EC2 instances on which tasks run
Manager
Manage cluster resource and state of tasks
Scheduler
Place tasks considering cluster status
Agent
Coordinate EC2 instances and Manager
Cluster, Scheduler, Task Scheduler
ManagerCluster
Task Definition
Task
Agent
ExpediaOne of the world’s leading travel companies
• Primer – Internal deployment tool
• Supports various applications
• Creates GitHub repository, pipeline,
and monitoring by one click
• Based on ECS Optimized AMI,
configured by AWS
CloudFormation
• Zero-Downtime Instance
Replacement
http://www.slideshare.net/AmazonWebServices/deep-dive-on-microservices-and-amazon-ecs-64033400
Continuous Delivery to ECS with Primer
ECS Production Clusters – Serving 200 applications
14 instances: 56 apps (+ 19 canaries) 17 instances: 78 apps (+ 25 canaries)
35 instances: 107 apps (+ 23 canaries) 5 instances: 7 apps (+ 4 canaries)
Charts produced with c3vis: github.com/ExpediaDotCom/c3vis
Amazon PersonalizationDistributed neural network learning on multiple GPUs
• From Apache Spark, run CPU
and GPU tasks transparently
• CPU: Amazon EMR
• GPU: Amazon ECS
• Package GPU libraries with
Docker image
• DSSTNE runs workloads in
parallel across hundreds of
GPUs
https://blogs.aws.amazon.com/bigdata/post/TxGEL8IJ0CAXTK/Generating-Recommendations-at-Amazon-Scale-with-Apache-Spark-and-Amazon-DSSTNE
Use ECS if:
• You want a managed service that scales with you
• You want to leverage native AWS integrations: IAM Roles, ALB, VPC, Cloudwatch Logs, Autoscaling, ECR
• You want to build around AWS tools: API/CLI/SDK/CloudFormation
• You want to leverage the strong ECS partner ecosystem
Conclusion:
• AWS has a rich ecosystem and supports every
major orchestration framework
• Whether you choose to use Amazon ECS or an
AWS Partner solution, our goal to provide the best
experience possible
• Container workloads: Think AWS
Thank you!
top related