Yelp is (always) hiring!Talk to a Speaker or Recruiter if

interested!

Obligatory Welcome Slide!

SF Devops Meetup Organizers Thank You!

Yelp’s Mission:Connecting people with great

local businesses.

A Behind the Scenes Tour of Yelp's SOA Architecture

What Are We Going To Talk About?

● YelpSOA Configs - linchpin of our stuff

● Service Discovery (Smartstack)+ Docker

● Mesos-based Deployment (PaaSTA)

● Dynamic Monitoring (Sensu)

First - Terminology

● Services ~= “microservices?”○ I don’t care how small your services are.

● SOA == Service Oriented Architecture○ Opposite of a monolithic app architecture. Usually

accompanied with teams (reverse Conway’s law?)

Yelp’s Architecture● Still has a significant amount of code in the

monolith○ it “pays the bills”○ Treated mostly like any other service

● Still though, lots of teams, lots of services○ Years of pushing for “the SOA dream”?

● In AWS and Datacenter in 3 “super” regions (yes, hybrid)

● We think we have some good“behind the scenes ideas”

What Would I Do If I Did It Again?

1. Have an idea2. Make a git repo

YelpSOA Configs: The Big Idea

● Let’s just use config files to be the source of truth about what our SOA infrastructure looks like instead of being in databases, zookeeper, or just from “whoever ran the deploy command last”

● Be declarative instead of imperative● “Just a git repo”

YelpSOA Configs: Things We WON'T do

● Let’s try not to be implementation specific● Let’s try not to do service discovery● No secrets in the config● All configs in one repo, no dev/prod split

Architecture: Good Ol Rsync + Cron

Plain-ole git repo

Normal Commits

Rsync Pulls

Machine generated files added

Read-Only

Slaves

Most Servers At Yelp

Most Servers At Yelp

Most Servers At Yelp

Read-Only

Slaves

Read-Only

SlavesMost Servers At Yelp

Architecture: Mostly Doesn’t Matter● Git repo - Works fine

● Gitolite for ACLs - Sure

● Pre-receive hooks for validation - Cool

● Cron Jobs to Deploy the Git repo - What else?

● Rsync Fanout to distribute the files - Heh, fine

● Yelp Puppet-Cron makes this easy and high-frequency: https://github.com/Yelp/puppet-cron

Demo: Seeing == Believing

What does it look like when you have your services metadata in an easy-to-reach place?

Example Ideas of Things to Put In There

● team data (ownership)● general service metadata (what is it?)● monitoring data (who should get alerts?)● deployments data (where should it run?)● routing data (where should it be

advertised?)

Conclusion / Big Ideas● Every team has people with enough privileges to edit

them● They are configs about the services, not for the

services (they don’t exist in the code repo)● Super fast deployment. No code, only config.● Describe the concepts, not the implementation● Build the metadata and the tools will follow!

A. To describe how cool Yelp's SOA configs areB. To persuade viewers buy Yelp’s SOA configs enterprise product offeringC. To Inspire viewers to build their own SOA configs - based on some of these ideas presentedD. To bore viewers about how Yelp rsyncs YAML files around with cron jobs.

Reading Comprehension Question:(what is the purpose of this talk?)

A. To describe how cool Yelp's SOA configs areB. To persuade viewers buy Yelp’s SOA configs enterprise product offeringC. To Inspire viewers to build their own SOA configs - based on some of these ideas presentedD. To bore viewers about how Yelp rsyncs YAML files around with cron jobs.

Reading Comprehension Question:(what is the purpose of this talk?)

● Helps puppet-cron type for high-frequency crons:https://github.com/Yelp/puppet-cron

● Gitolite for ACLS and hooks: http://gitolite.com/● rsync + cron

Tools used:

Questions