Parity
Keeping development andproduction environments in sync
for fun, profit, and sanity
Hi, I’m Geoff
I’m the CTO at Cortexhttp://cortexintel.com
Developing in Ruby since 2008
Formerly operator ofFive Tool Development
What is parity?
Parity is consistency in the configuration of your application’s environments (production, staging, development, test, etc.)
Why is this important?
• When your tests pass, you should be confident that your code works.
• When you have parity, you'll find many more bugs and configuration idiosyncrasies in development, before you get to a QA or (gasp) production phase.
Achieving parity• Make the time gap small: a developer may write
code and have it deployed hours or even just minutes later.
• Make the personnel gap small: developers who wrote code are closely involved in deploying it and watching its behavior in production.
• Make the tools gap small: keep development and production as similar as possible.
Source: http://12factor.net/dev-prod-parity
Setup for parity• Use the same backing services and support
software in all environments
• At Cortex our bootstrap script (modified from Thoughtbot’s suspenders-created script) installs PostgreSQL, Redis, node.js, and Bower if those packages are not already installed. It then sets up pow, foreman, tunnelss, so we can serve the app at http://cortex.dev and https://cortex.dev
Backing Services• Database (Postgres, MySQL, Mongo, etc)
• Queues (Sidekiq, Resque, DelayedJob)
• Storage (local disk, AWS, CloudDrive, SkyDrive)
• Search (solr, Elasticsearch)
• Caching (Rails In-memory, Memcached)
The adapter fallacy: ActiveRecord
• ActiveRecord promises portability, but most applications will eventually implement behavior that isn’t consistent across different databases in the form of SQL literals or queries that return different results
• Rails encourages the use of SQLite in development, but there’s no reason not to use the database you’re going to use in production
Tools for parity• Foreman - process manager
• Setup script
• pow - app/DNS/proxy with nice “.dev” domain names
• Tunnelss (fork of tunnels) - route SSL to another port locally
• Parity - easy management of Heroku environments
Foreman
• Process manager that works well locally as a stand-in for Heroku
• Runs web and background processes like Sidekiq, DelayedJob, or Resque
• Easy to configure alongside pow
Parity gem and Heroku
• Name your app folder big-red-machine
• Name your Heroku remotes big-red-machine-staging and big-red-machine-production
• Use parity aliases to execute commands easily on Heroku
• migrate, rake, log, tail, log2viz, console, config, open
Moving data with parity
• production backup - takes a snapshot on production
• staging restore production - copies the latest production backup to staging
• development restore production - copies the latest production backup to local dev
Rails environment setup
• Make differences between your environments explicit
• Move universal configuration to application.rb or initializers
• Have environments inherit from a base environment (staging and demo from production), then override the few items that are different
Differences in development
For your own sanity, some items might be smart candidates to break parity in your development environment:
• Eager loading
• Asset compilation
• SSL
Windows?
• If you deploy to a POSIX system (Linux, BSD, etc.), you should be doing development or at least your testing in a similar environment
• Use Vagrant or other virtualization tools to run a facsimile environment locally
Configure with ENV Never store credentials or secrets in source control
Environment can be used for configuration beyond credentials!
• S3 configuration
• CDN setup
• DB connection pool size
Local ENV• Rails 4.1 introduced secrets.yml, a standard
way to store local credentials
• dotenv/dotenv-rails can load credentials from Heroku configs (downloaded with heroku config:pull)
• Figaro performs similar functionality to Rails 4.1’s secrets.yml (YAML format, one file for all environments)
Careful what you mock
You should not need a network connection to run your tests locally
In integration tests, don’t stub or mock gems that interface with external services. Record a real response with the vcr gem, then replay that on later tests (make sure to strip out credentials after recording!)
Fog and Amazon• Use Fog to abstract Amazon S3 so that local
development and test machines don’t need the network
• Allows common interface for local and external storage
• In our CI environment, we make a real connection to S3 to confirm that our local setup is not hiding a real problem
Strategies for parity
• Use the same backing services everywhere!
• Manage packages with a package manager such as Homebrew or apt-get
• Use the same application server in development as production. If you’re using Unicorn in production, don’t use WEBrick in development
OSX protip: brew services
Can’t remember how to turn on PostgreSQL? How to restart Redis? Use brew services
• brew services list
• brew services start postgresql
• brew services restart redis
Strategies for parity
Avoid Rails.env.production? checks. Use tools like custom_configuration
# config/environments/production.rbBCX::Application.configure do config.x.resque.jobs_inline = trueend
# config/initializers/resque.rbif Rails.configuration.x.resque.jobs_inline Resque.inline = trueend
Make it easy on testers
• My app serves locally at http://cortex.dev
• My laptop is on the IP address 10.0.1.37
• My tester can reach my machine athttp://cortex.10.0.1.37.xip.io
• When you’re not on the same subnet, use ngrok or Finch
Strategies for deployment
• Keep your PRs as small as is reasonable
• Only merge to master code you’re ready to ship
• Deploy early and often (or… continuously!)
• Automate your deployment process
• Reduce the number of things you can forget
Make deployment easy
• You should be able to do most deploys withONE (1) command
• Reduce the number of things you can forget through automation
• Don’t trust that you or your teammates will follow a checklist every time
Protect yourself
Automating deployment means:
• You don’t forget to run migrations
• You don’t forget to update reference data
• You don’t forget to recompile or sync assets
• You don’t forget to restart the server
How we deploy
• Alias or script: push-to-staging, push-to-production, push-to-demo
gp staging $(git symbolic-ref --short -q HEAD):master --force && staging migrate
• Always rebuild assets with bower
• Always run migrations
• Always apply production data updates
Heroku Buildpacks
• Tim Pope’s Ruby buildpack tracks with Heroku’s buildpack, but always runs migrations without requiring a second connection and app restart
• We use qnyp/heroku-buildpack-ruby-bower to automatically build our assets from Bower whenever we deploy, which helps avoid issues where we forget to sync assets
Questions?
Further Reading• Heroku's 12 Factor App site:
http://12factor.net/
• parity gem: http://github.com/croaky/parity
• Foreman as process manager, Pow as DNS server:http://robots.thoughtbot.com/foreman-as-process-manager-pow-as-dns-server-and-http
• Beyond the default Rails environments:http://signalvnoise.com/posts/3535-beyond-the-default-rails-environments
Thanks!
I love talking about Ruby and Rails, so feel free to reach out to me with questions!
email: [email protected]
Github: geoffharcourt
Twitter: @geoffharcourt