![Page 1: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chad Schmutzer, Solutions Architect
Hubert Cheung, Solutions Architect
David Kuo, Solutions Architect
Andy Mui, Solutions Architect
November 2016
Deploy a Deep Learning Framework
on Amazon ECS
CON314
![Page 2: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/2.jpg)
What to expect from this workshop
• Workshop goals
• Overview of ECS + ECR
• Overview of AWS CloudFormation
• Overview of EC2 Spot Instances
• Hands on workshop
• Wrap-up
![Page 3: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/3.jpg)
Workshop Goals
![Page 4: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/4.jpg)
Demonstrate ECS value
• Increase infrastructure utilization
• environment isolation
• placing mixed applications in same environment
• easy deployment
• We just chose to use MXNet as an example application
to containerize
![Page 5: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/5.jpg)
What’s MXNet?
• MXNet is an open-source deep learning framework that
allows you to define, train, and deploy deep neural
networks on a wide array of devices, from cloud
infrastructure to mobile devices. It is highly scalable,
allowing for fast model training, and supports a flexible
programming model and multiple languages.
• http://mxnet.io/
![Page 6: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/6.jpg)
ECS + ECR
![Page 7: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/7.jpg)
ECS Benefits
Cluster management
made easy
Flexible scheduling Integrated and
extensible
Security Performance at scale
![Page 8: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/8.jpg)
ECS Architecture
Docker
Task
Container instance
Amazon
ECS
Container
ECS agent
ELB
Internet
ELB
User /
Scheduler
API
Cluster Management Engine
Task
Container
Docker
Task
Container instance
Container
ECS agent
Task
Container
Docker
Task
Container instance
Container
ECS agent
Task
Container
AZ 1 AZ 2
Key/Value Store
Agent Communication Service
![Page 9: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/9.jpg)
ECS Common Use Cases
Applications and services
• Configuration and deployment
• Microservices
Batch processing
![Page 10: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/10.jpg)
What’s ECR?
• Amazon EC2 Container Registry (Amazon ECR) is a
fully-managed Docker container registry that makes it
easy for developers to store, manage, and deploy Docker
container images. Amazon ECR is integrated with
Amazon EC2 Container Service (Amazon ECS),
simplifying your development to production workflow.
• Learn more: https://aws.amazon.com/ecr/
![Page 11: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/11.jpg)
How does ECS use ECR?
Amazon ECR
virtual private cloud
VPC subnet
VPC subnet
ECS Task Definition
![Page 12: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/12.jpg)
AWS CloudFormation
![Page 13: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/13.jpg)
Template CloudFormation Stack
JSON formatted file
Parameter definition
Resource creation
Configuration actions
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Framework
Stack creation
Stack updates
Error detection and rollback
CloudFormation – Components & Technology
![Page 14: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/14.jpg)
CloudFormation Benefits
Templated resource
provisioning
Infrastructure
as code
Declarative
and flexible
Easy to use
![Page 15: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/15.jpg)
CloudFormation Use Cases
Stack replication Infrastructure
scale out
Blue-green
deployments
Infrastructure
as code
![Page 16: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/16.jpg)
Why do customers use CloudFormation?
Developers/DevOps teams value CloudFormation for its ability to treat
infrastructure as code, allowing them to apply software engineering principles,
such as SOA, revision control, code reviews, integration testing to
infrastructure.
IT Admins and MSPs value CloudFormation as a platform to enable
standardization, managed consumption, and role-specialization.
ISVs value CloudFormation for its ability to support scaling out of multi-tenant
SaaS products by quickly replicating or updating stacks. ISVs also value
CloudFormation as a way to package and deploy their software in their
customer accounts on AWS.
![Page 17: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/17.jpg)
EC2 Spot Instances
![Page 18: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/18.jpg)
On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
Amazon EC2 Consumption Models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot
Bid for unused
capacity, charged at a
Spot price which
fluctuates based on
supply and demand
For time-insensitive
or transient
workloads
![Page 19: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/19.jpg)
With Spot, the rules are simple
Markets where the price of compute changes based on
supply and demand
You’ll never pay more than your bid. When the market exceeds your bid you get 2 minutes to
wrap up your work
![Page 20: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/20.jpg)
$0.27 $0.29$0.50
1b 1c1a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C3
$1.76
On
Demand
$0.88
$0.44
$.22
$0.11
Show me the markets!
Each instance family
Each instance size
Each Availability Zone
In every region
Is a separate Spot Market
![Page 21: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/21.jpg)
Spot Fleet
![Page 22: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/22.jpg)
Spot Fleet helps you
Launch Thousands of Spot Instanceswith one RequestSpotFleet call.
Get Best PriceFind the lowest priced horsepower that works for you.
or
Get Diversified ResourcesDiversify your fleet. Grow your availability.
And
Apply Custom WeightingCreate your own capacity unit based on your application
needs
![Page 23: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/23.jpg)
Diversification with EC2 Spot Fleet
Multiple EC2 Spot Instances
selected
Multiple Availability Zones
selected
Pick the instances with similar
performance characteristics, e.g.,
c3.large, m3.large, m4.large,
r3.large, c4.large.
![Page 24: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/24.jpg)
Workshop: Image Classification
![Page 25: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/25.jpg)
Overall Architecture
Client Internet
Amazon
S3
Amazon ECR
VPC subnet VPC subnet
instances instances
![Page 26: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/26.jpg)
Lab 1: Getting Started
CloudFormation
Amazon
S3
Amazon ECR
VPC subnet VPC subnet
instances instances
![Page 27: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/27.jpg)
Lab 2: Build MXNet on a Docker Container
Amazon ECR
virtual private cloud
VPC subnet
VPC subnet
ECS Task Definition
![Page 28: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/28.jpg)
Lab 3: Launch MXNet with ECS
Client Internet
Amazon
S3
Amazon ECR
VPC subnet VPC subnet
instances instances
SSH
![Page 29: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/29.jpg)
Lab 4: Image Classification Demo
![Page 30: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/30.jpg)
Wrap-up
• ECS makes cluster management easy
• ECS has flexible scheduling
• ECS has enables strong security posture
• What other applications can you containerize to deploy
and manage at scale?
![Page 31: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/31.jpg)
Thank you!
![Page 32: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/32.jpg)
Remember to complete
your evaluations!
![Page 33: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/33.jpg)
Related Sessions
• MAC306 - Using MXNet for Recommendation Modeling
at Scale
• CON301 - Operations Management with Amazon ECS
• CON302 - Development Workflow with Docker and
Amazon ECS
• CON401 - Amazon ECR Deep Dive on Image
Optimization
![Page 34: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/34.jpg)
Appendix
![Page 35: AWS re:Invent 2016: Workshop: Deploy a Deep Learning Framework on Amazon ECS (CON314)](https://reader031.vdocuments.net/reader031/viewer/2022030317/587125fd1a28abe4448b623d/html5/thumbnails/35.jpg)
Estimated Workshop Costs