architecting for the cloud: hoping for the best, prepared for the worst
TRANSCRIPT
![Page 1: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/1.jpg)
AWS Loft: Behind the scenes with Cotap
Architecting for the Cloud:
Hoping for the best, prepared for the worst.
![Page 2: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/2.jpg)
![Page 3: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/3.jpg)
![Page 4: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/4.jpg)
![Page 5: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/5.jpg)
![Page 6: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/6.jpg)
![Page 7: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/7.jpg)
![Page 8: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/8.jpg)
![Page 9: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/9.jpg)
![Page 10: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/10.jpg)
![Page 11: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/11.jpg)
Infrastructure as Code
![Page 12: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/12.jpg)
Infrastructure as Code
● Current state
● Past decisions
● Tracking the evolution
![Page 13: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/13.jpg)
● CloudFormation
● Design -> JSON
● Version Control!
Infrastructure as Code
![Page 14: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/14.jpg)
Infrastructure as Code
![Page 15: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/15.jpg)
Infrastructure as Code
![Page 16: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/16.jpg)
Infrastructure as Code
![Page 17: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/17.jpg)
![Page 18: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/18.jpg)
![Page 19: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/19.jpg)
Rule #1
All changes have to be under Version
Control
![Page 20: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/20.jpg)
Design for automation
![Page 21: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/21.jpg)
Design for automation
● AutoScalingGroups
● Hardware: CloudFormation
● Software: Configuration management
● Cattle not Cats
![Page 22: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/22.jpg)
Rule #2
No instances should be launched manually.
![Page 23: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/23.jpg)
Monitoring & Alerting
![Page 24: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/24.jpg)
Monitoring & Alerting
● Cost ofo Interruptions
o Waking somebody up
● Channels
● Self-healing infrastructure
● External monitoring
● Page only when critical
![Page 25: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/25.jpg)
Monitoring & Alerting
Situation Channel Page
Disk full 60% Chat, Email ✗
Disk full 90% Chat, Email, PagerDuty ✓
Chef not running for > 30m Chat, Email ✗
Redis not running for > 3 x 5s Chat, Email, PagerDuty ✓
ElasticSearch N-1 Chat, Email ✗
ElasticSearch N-2 Chat, Email, PagerDuty ✓
![Page 26: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/26.jpg)
Monitoring & Alerting
● Cost ofo Interruptions
o Waking somebody up
● Channels
● Self-healing infrastructure
● External monitoring
● Page only when critical
![Page 27: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/27.jpg)
Platform to fail
![Page 28: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/28.jpg)
Platform to fail
● Easy creation of temporary “Stacks”
● Branches can get their own hardware
● Clients can talk to a branch
● QA happens on Sandbox
● Exact copy of Production
● Scale up/down based on needs
● Different Region (us-east-1)
![Page 29: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/29.jpg)
Platform to fail
![Page 30: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/30.jpg)
Platform to fail
● Easy creation of temporary “Stacks”
● Branches can get their own hardware
● Clients can talk to a branch
● QA happens on Sandbox
● Exact copy of Production
● Scale up/down based on needs
● Different Region (us-east-1)
![Page 31: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/31.jpg)
All changes have to go through Sandbox.
Rule #3
![Page 32: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/32.jpg)
Rule #4
Production is just a more powerful Sandbox
![Page 33: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/33.jpg)
Disaster Recovery
![Page 34: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/34.jpg)
Disaster Recovery
● Multi-AZs
● Traffic routing
● Multi-Regions (S3 too)
● AutoScalingGroups Min:1 Max:1
● Off-site backups (VPN + Disks)
● RPO + RTO
![Page 35: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/35.jpg)
Security
![Page 36: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/36.jpg)
Security
● MFA
● Public key distribution
● Root key rotation
● Private/Public Subnets
● ACLs/Security Groups
● Update AMIs
● Trusted Advisor!
![Page 37: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/37.jpg)
Security
![Page 38: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/38.jpg)
Scaling
![Page 39: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/39.jpg)
Scaling
● Preemptive
● Automatic
● Vertically
● Horizontally
● Bottlenecks
![Page 40: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/40.jpg)
Scaling
![Page 41: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/41.jpg)
Cost Control
![Page 42: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/42.jpg)
Cost Control
● Tagso Role
o Environment
● Cost explorer
● Threshold alerting
● Share monthly
● Export to CSV
● Right-Scale (ASG)
![Page 43: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/43.jpg)
Cost Control
![Page 44: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/44.jpg)
Cost Control
![Page 45: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/45.jpg)
Cost Control
● Tagso Role
o Environment
● Cost explorer
● Threshold alerting
● Share monthly
● Export to CSV
● Right-Scale (ASG)
![Page 46: Architecting for the Cloud: Hoping for the Best, Prepared for the Worst](https://reader035.vdocuments.net/reader035/viewer/2022062419/55a19dc81a28ab555c8b45fc/html5/thumbnails/46.jpg)
4 rules of 5 nines.
● All changes have to be under VC
● No instance should be launched manually
● All changes are deployed to Sandbox first
● Production is just a more powerful Sandbox