building a saas with coreos, docker, and etcd

@RossKukulinski

Building a SaaS with CoreOS and etcd

CoreOS Fest May 4, 2015

@RossKukulinski

Ross Kukulinski Yodlr Founder / CEO

BayNode Co-Organizer

iojs/nodejs Evangelist

Soccer Fanatic

ross@getyodlr.com

@RossKukulinski

The internal tool that wasn’t internal anymore

@RossKukulinski

Our Goals• Reduce application complexity

• A/B test multiple versions of apps

• Consistent app from dev → test → staging → prod

• Scalable

• Fault tolerant

• Minimize time spent doing ‘devops’

@RossKukulinski

Docker

@RossKukulinski

Our Goals• Reduce application complexity

• A/B test multiple versions of apps

• Consistent app from dev → test → staging → prod

• Scalable

• Fault tolerant

• Minimize time spent doing ‘devops’

@RossKukulinski

How do you ship Docker containers?

@RossKukulinski

Linux for Massive Server Deployments

@RossKukulinski

Goals• Reduce application complexity

• Run multiple versions of the same app

• Consistent app from dev → test → staging → prod

• Scalable

• Fault tolerant

• Minimize time spent doing ‘devops’

@RossKukulinski

After further deliberation, we’re going big and switching to CoreOS/Docker entirely. We feel its the right direction long-term for us and application development in general.

- Me, internal memo September 2014

Lightbulb! by Matthew WynnMonitor photo by Don Fulano

@RossKukulinski

1. etcd is a databaseSo treat it like one

@RossKukulinskiImage: CoreOS.com

Give etcd space

@RossKukulinski

2. etcd is a databaseFor your real-time infrastructure state

@RossKukulinski

Service Availability

REST API microservice Sidekick

Sidekick Pattern: 1. Get ip:port of microservice (docker inspect) 2. Health checks (containerized integration tests) 3. Publish service instance details to etcd

etcd1 & 2 3

@RossKukulinski

Service Discovery

haproxy confd

Confd: 1. Query etcd for ip:port of upstream services 2. Update configuration from template 3. Reload load balancer process

etcd

github.com/kelseyhightower/confd

2 & 3 1

@RossKukulinski

Dynamic external services

DNS & Cloud Load Balancers

lb-config

lb-config: 1. Query etcd for domain/ip:port of internal lbs 2. Update DNS & external load balancers

etcd12

github.com/yodlr/rax-lb-config

@RossKukulinski

3.

@RossKukulinski

3. etcd is a databaseWith super important data: Protect it!

PSA: $private_ipv4 is probably not what you think it is

@RossKukulinski

4. Sometimes you just need a GUI(Docker & CoreOS do have awesome CLI tools though)

github.com/yodlr/CoreGI

Also check out purpleworks/fleet-ui

@RossKukulinski

5. Fleet unit templates aren’t enough

rest-api@.service rest-api@{1..3}.service —> 3 instances of the service

@RossKukulinski

rest-api-{{env}}-{{tag}}@.service

github.com/yodlr/fleet-templater

@RossKukulinski

rest-api-prod-86a951a9@.service

github.com/yodlr/fleet-templater

@RossKukulinski

App deploys before

Kubernetes

@RossKukulinski

Rolling Deploys as a Service

github.com/yodlr/fleet-rolling-deploys

@RossKukulinski

Closing thoughts• Containers are awesome

• We <3 CoreOS

• You can get really far with fleet & etcd

• Kubernetes is so close to primetime

• Remember: etcd is a database, treat it like one

@RossKukulinski

Thanks!

ross@getyodlr.com