aws elastic container service (ecs) with a ci pipeline overview
TRANSCRIPT
![Page 1: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/1.jpg)
Deploying a (Dockerized) application onto
EC2 Container Service (ECS)
Wyn Van Devanter
@wynv
![Page 2: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/2.jpg)
EC2 Container Service (ECS)
• Manages clusters of EC2 instances for running containers
• Takes care of installing, operating and scaling a cluster management infrastructure
![Page 3: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/3.jpg)
EC2 Container Service (ECS)
• Container management with API
• Centralized service to get cluster state
• Schedules the placement of containers across your cluster based on resources needs, isolation policies, and availability requirements
• Can integrate your own scheduler
• Integrated with other EC2 and AWS services and features
![Page 4: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/4.jpg)
Key terms
• Cluster – logical grouping of EC2 instances, each with ECS container agent installed
• Container Instance - EC2 instance with container agent that runs containers in ECS
• Task Definition (like docker-compose.yml) – blueprint for the app to run in ECS; describes containers to run for app, and where environment variables can go
• Task – running instance of a task definition
• Service – keeps 1+ tasks running
• Elastic Container Registry (ECR)
![Page 5: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/5.jpg)
![Page 6: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/6.jpg)
ECS Clusters
• Run containers across instances in availability zones in a region
• Cluster contains EC2 instances (called container instances). ECS-optimized AMIs are available.
• Instances contain the ECS container agent • Sends information about the instance's current running tasks and resource
utilization to Amazon ECS
• Starts and stops tasks whenever it receives a request from Amazon ECS
![Page 7: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/7.jpg)
Running apps on ECS
• Define task definitions to specify what container images to run across your clusters
• Container images are pulled from container registries (i.e. ECR)
• Services can be used with a task definition to keep a desired number of app instances running
![Page 8: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/8.jpg)
Task definitions
• To prepare your application to run on Amazon ECS, you create a task definition.
• JSON file that describes one or more containers run to form your application; blueprint • Similar to the Docker Compose file and can be generated from one
• Specify various parameters for your application, such as: • which image(s) to use
• which ports should be opened
• what data volumes should be used
• where the container(s) should be placed
![Page 9: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/9.jpg)
Tasks
• An instantiation of a task definition on a container instance within your cluster
• The Amazon ECS task scheduler is responsible for placing tasks on container instances
• Restarted as needed
![Page 10: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/10.jpg)
Services
• You can define a service that runs and maintains a specified number of tasks simultaneously
• Manages running your application
• Allows scaling more instances (containers) of your app as needed
![Page 11: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/11.jpg)
![Page 12: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/12.jpg)
![Page 13: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/13.jpg)
![Page 14: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/14.jpg)
ECS setup
• Setting up IAM accounts for ECS management and deployment
• Setting up ECS for application deployment
• Setting up CI for automatic build and deployment
![Page 15: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/15.jpg)
Setting up IAM accounts for ECS managementand deployment• IAM account for ECS management (i.e. creating cluster, load balancer)
• IAM account for ECS deployment (i.e. updating services and task definitions)
• Groups
![Page 16: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/16.jpg)
Setting up ECS for application deployment
• Create cluster
• Create EC2 Container Registry (ECR) for app
• Set up load balancer for app
![Page 17: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/17.jpg)
Setting up ECS for application deployment
• Create a task definition for the app first environment (i.e. dev/staging or production)
• Create a service to run the task definition
• Repeat for each environment, setting their environment variables
![Page 18: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/18.jpg)
Setting up CI for automatic build and deployment
• Set environment variables for app in CI (i.e. cluster name, task definition base name, AWS account ID)
• Link to repository with app w/.travis.yml & deployment bash scripts
![Page 19: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/19.jpg)
![Page 20: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/20.jpg)
Deploying
• TravisCI detects check-in
• Builds app via Docker & pushes to ECR
• Deploys using task definition
![Page 21: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/21.jpg)
App infrastructure
• API app in a container (ASP.NET Core)
• PostgreSQL container, data mounted via volume
• EC2 Container Service (ECS) 2-node cluster with ECR, load balancer, AWS Certificates and Route 53
• Travis with bash scripts for CI/CD
![Page 22: AWS Elastic Container Service (ECS) with a CI Pipeline Overview](https://reader033.vdocuments.net/reader033/viewer/2022051301/5a64c45a7f8b9a310b8b47d5/html5/thumbnails/22.jpg)
Thanks! @wynv
o Docs: Overview & Key Concepts http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
o Docs: Core Setup Steps, http://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html
o Reference architecture for ECS w/deployment pipeline (CloudFormation), https://github.com/awslabs/ecs-refarch-continuous-deployment/blob/master/templates/deployment-pipeline.yaml
o ECS IAM Policies, http://docs.aws.amazon.com/AmazonECS/latest/developerguide/IAM_policies.html
o Sample app ready for ECS with task definition, https://github.com/excellalabs/ecs-deploy-setup