aws user group sydney - atlassian 5-10-16
TRANSCRIPT
Micros Atlassian’s Internal PaaS on AWS
JASON UMIKER • @JASONUMIKER
PaaS Overview
•‿• > <
PaaS Overview
CloudFormation RDS S3 DynamoDB
EC2
Over
services running right now
500
• Big cool statistic
• 2,569 • Add-Ons in Marketplace
Todo: better image
Why do we need a Platform?
Data Centre
JIRA & Confluence Cloud Architecture
R a c k
Cloud instance in OpenVZ container
(approx. 1500 per rack)
Breaking up the Monoliths
* *
Data Centre
R a c k
BlobStore service
PaaS
Breaking up the Monoliths
> <
Data Centre
R a c k
PaaS
_ _ - - /oo\
o o
. .
Todo: better image
What does the platform do?
+ SD { }Artifact Descriptor
Deploying a Service
Deploying a Service name: Micros Workshop
description: A sample app
links:
binary:
type: npm
artifactId: '@atlassian/micros-workshop'
version: 0.0.1
healthcheck:
uri: /healthcheck
source:
url: 'ssh://[email protected]/x.git'
owners:
organization: Cloud Platform Engineering
+ SD { }Artifact Descriptor
https://<service>.atlassian.io
Deploying a Service
Compute: CloudFormation stack with: • Route 53 DNS record • Elastic Load Balancer • EC2 instances in Autoscaling group
Resources: • S3, RDS, DynamoDB • Redis, Memcached, SQS…
Logging & Monitoring
AWS CloudWatch
https://<service>.atlassian.io
. .
cloudymccloudface.atlassian.io
> micros service:deploy
my-service
-f service-descriptor.yml
Deploying a Service
• Big cool statistic
• 2,569 • Add-Ons in Marketplace
Lessons learned: Balancing consistency & flexibility
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Structured Log Data
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
https://<service>.atlassian.io
Common Upgrade flow
v1 v2
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Encourage: Resource isolation
Resource Isolation ServiceB.atlassian.io
Datastores
ServiceA.atlassian.io
Datastores
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Enforce: Common service metadata & cost allocation
Encourage: Resource isolation
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Enforce: Restricted ssh access
Enforce: Common service metadata & cost allocation
Encourage: Resource isolation
Where to favor rules & consistency?
Enforce: Common logging & monitoring infra
Encourage: Structured log data
Enforce: Standard upgrade flow
Enforce: Limited set of persistent data stores
Enforce: Statelessness & disposable compute nodes
Enforce: Restricted ssh access
Encourage: Resource isolation
Enforce: Common service metadata & cost allocation
12 Factor (not pets)
Nodes as cattle (not pets)
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Services by
Runtime
Docker
55% JVM 25%
Node.js 15%
Python 5%
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Enable: Integration with 3rd party services
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt: Education & on-boarding
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt: Education & on-boarding
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Open the platform for contributions
Contributions &
Self-hosted PaaS
components
Bootstrapping Service
Deployment orchestration
service
Chaos Monkey
Log Analysis Service
PaaS
PaaS
PaaS
PaaS
PaaS
Simple Workflow integration
Service cost estimation
Data Pipelines Integration
Service-to-service auth
…
Where to bend the rules / favor flexibility?
Enable: Service-level technology stack flexibility
Adapt to underlying platform limitations
Enable: Integration with 3rd party services
Adapt: roadmap to inbound requests & technology trends
Open the platform for contributions
Adapt: Education & on-boarding
Organisation wide: Disaster Recovery
Compliance
Infra improvements: Container Clusters
Serverless
Promises of Service Decomposition
Choice of tech stacks
Scalability Independent deployments
Resilience End-to-end ownership
Your platform should enable them – with heart and balance.
PaaS
Healthy PaaS —> Happy Microservices!
Balance consistency with flexibility
Encourage / enforce “12 factor” & “nodes as
cattle”
Keep the barrier to entry low
Tips for a healthy, balanced PaaS & a happy microservice ecosystem.
Questions?
Reuters. Chickens perch on the roof of a hennery to escape rising floodwaters after Typhoon Utor hit Maoming, Guangzhou province August 15, 2013. Typhoon Utor hit China's southern Guangdong and Guangxi provinces before easing to a tropical storm.
Image Credits
https://design.atlassian.com/
Courtesy of Scott Monday: - https://au.pinterest.com/pin/146085581635905884/ - http://scottmonday.com/
Copyright: velvetocean / 123RF Stock Photo – http://www.123rf.com/profile_velvetocean
Copyright: feedough / 123RF Stock Photo – http://www.123rf.com/profile_feedough'
Copyright: iimages / 123RF Stock Photo – http://www.123rf.com/iimages_feedough