dcrug: achieving development-production parity
DESCRIPTION
Thursday, June 12th 2014 Discussing strategies in Rails development for keeping multiple application environments as consistent as possible for the best development, testing, and deployment experience.TRANSCRIPT
![Page 1: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/1.jpg)
Parity
Keeping development andproduction environments in sync
for fun, profit, and sanity
![Page 2: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/2.jpg)
Hi, I’m Geoff
I’m the CTO at Cortexhttp://cortexintel.com
Developing in Ruby since 2008
Formerly operator ofFive Tool Development
![Page 3: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/3.jpg)
What is parity?
Parity is consistency in the configuration of your application’s environments (production, staging, development, test, etc.)
![Page 4: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/4.jpg)
Why is this important?
• When your tests pass, you should be confident that your code works.
• When you have parity, you'll find many more bugs and configuration idiosyncrasies in development, before you get to a QA or (gasp) production phase.
![Page 5: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/5.jpg)
Achieving parity• Make the time gap small: a developer may write
code and have it deployed hours or even just minutes later.
• Make the personnel gap small: developers who wrote code are closely involved in deploying it and watching its behavior in production.
• Make the tools gap small: keep development and production as similar as possible.
Source: http://12factor.net/dev-prod-parity
![Page 6: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/6.jpg)
Setup for parity• Use the same backing services and support
software in all environments
• At Cortex our bootstrap script (modified from Thoughtbot’s suspenders-created script) installs PostgreSQL, Redis, node.js, and Bower if those packages are not already installed. It then sets up pow, foreman, tunnelss, so we can serve the app at http://cortex.dev and https://cortex.dev
![Page 7: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/7.jpg)
Backing Services• Database (Postgres, MySQL, Mongo, etc)
• Queues (Sidekiq, Resque, DelayedJob)
• Storage (local disk, AWS, CloudDrive, SkyDrive)
• Search (solr, Elasticsearch)
• Caching (Rails In-memory, Memcached)
![Page 8: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/8.jpg)
The adapter fallacy: ActiveRecord
• ActiveRecord promises portability, but most applications will eventually implement behavior that isn’t consistent across different databases in the form of SQL literals or queries that return different results
• Rails encourages the use of SQLite in development, but there’s no reason not to use the database you’re going to use in production
![Page 9: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/9.jpg)
Tools for parity• Foreman - process manager
• Setup script
• pow - app/DNS/proxy with nice “.dev” domain names
• Tunnelss (fork of tunnels) - route SSL to another port locally
• Parity - easy management of Heroku environments
![Page 10: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/10.jpg)
Foreman
• Process manager that works well locally as a stand-in for Heroku
• Runs web and background processes like Sidekiq, DelayedJob, or Resque
• Easy to configure alongside pow
![Page 11: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/11.jpg)
Parity gem and Heroku
• Name your app folder big-red-machine
• Name your Heroku remotes big-red-machine-staging and big-red-machine-production
• Use parity aliases to execute commands easily on Heroku
• migrate, rake, log, tail, log2viz, console, config, open
![Page 12: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/12.jpg)
Moving data with parity
• production backup - takes a snapshot on production
• staging restore production - copies the latest production backup to staging
• development restore production - copies the latest production backup to local dev
![Page 13: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/13.jpg)
Rails environment setup
• Make differences between your environments explicit
• Move universal configuration to application.rb or initializers
• Have environments inherit from a base environment (staging and demo from production), then override the few items that are different
![Page 14: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/14.jpg)
Differences in development
For your own sanity, some items might be smart candidates to break parity in your development environment:
• Eager loading
• Asset compilation
• SSL
![Page 15: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/15.jpg)
Windows?
• If you deploy to a POSIX system (Linux, BSD, etc.), you should be doing development or at least your testing in a similar environment
• Use Vagrant or other virtualization tools to run a facsimile environment locally
![Page 16: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/16.jpg)
Configure with ENV Never store credentials or secrets in source control
Environment can be used for configuration beyond credentials!
• S3 configuration
• CDN setup
• DB connection pool size
![Page 17: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/17.jpg)
Local ENV• Rails 4.1 introduced secrets.yml, a standard
way to store local credentials
• dotenv/dotenv-rails can load credentials from Heroku configs (downloaded with heroku config:pull)
• Figaro performs similar functionality to Rails 4.1’s secrets.yml (YAML format, one file for all environments)
![Page 18: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/18.jpg)
Careful what you mock
You should not need a network connection to run your tests locally
In integration tests, don’t stub or mock gems that interface with external services. Record a real response with the vcr gem, then replay that on later tests (make sure to strip out credentials after recording!)
![Page 19: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/19.jpg)
Fog and Amazon• Use Fog to abstract Amazon S3 so that local
development and test machines don’t need the network
• Allows common interface for local and external storage
• In our CI environment, we make a real connection to S3 to confirm that our local setup is not hiding a real problem
![Page 20: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/20.jpg)
Strategies for parity
• Use the same backing services everywhere!
• Manage packages with a package manager such as Homebrew or apt-get
• Use the same application server in development as production. If you’re using Unicorn in production, don’t use WEBrick in development
![Page 21: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/21.jpg)
OSX protip: brew services
Can’t remember how to turn on PostgreSQL? How to restart Redis? Use brew services
• brew services list
• brew services start postgresql
• brew services restart redis
![Page 22: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/22.jpg)
Strategies for parity
Avoid Rails.env.production? checks. Use tools like custom_configuration
# config/environments/production.rbBCX::Application.configure do config.x.resque.jobs_inline = trueend
# config/initializers/resque.rbif Rails.configuration.x.resque.jobs_inline Resque.inline = trueend
![Page 23: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/23.jpg)
Make it easy on testers
• My app serves locally at http://cortex.dev
• My laptop is on the IP address 10.0.1.37
• My tester can reach my machine athttp://cortex.10.0.1.37.xip.io
• When you’re not on the same subnet, use ngrok or Finch
![Page 24: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/24.jpg)
Strategies for deployment
• Keep your PRs as small as is reasonable
• Only merge to master code you’re ready to ship
• Deploy early and often (or… continuously!)
• Automate your deployment process
• Reduce the number of things you can forget
![Page 25: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/25.jpg)
Make deployment easy
• You should be able to do most deploys withONE (1) command
• Reduce the number of things you can forget through automation
• Don’t trust that you or your teammates will follow a checklist every time
![Page 26: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/26.jpg)
Protect yourself
Automating deployment means:
• You don’t forget to run migrations
• You don’t forget to update reference data
• You don’t forget to recompile or sync assets
• You don’t forget to restart the server
![Page 27: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/27.jpg)
How we deploy
• Alias or script: push-to-staging, push-to-production, push-to-demo
gp staging $(git symbolic-ref --short -q HEAD):master --force && staging migrate
• Always rebuild assets with bower
• Always run migrations
• Always apply production data updates
![Page 28: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/28.jpg)
Heroku Buildpacks
• Tim Pope’s Ruby buildpack tracks with Heroku’s buildpack, but always runs migrations without requiring a second connection and app restart
• We use qnyp/heroku-buildpack-ruby-bower to automatically build our assets from Bower whenever we deploy, which helps avoid issues where we forget to sync assets
![Page 29: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/29.jpg)
Questions?
![Page 30: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/30.jpg)
Further Reading• Heroku's 12 Factor App site:
http://12factor.net/
• parity gem: http://github.com/croaky/parity
• Foreman as process manager, Pow as DNS server:http://robots.thoughtbot.com/foreman-as-process-manager-pow-as-dns-server-and-http
• Beyond the default Rails environments:http://signalvnoise.com/posts/3535-beyond-the-default-rails-environments
![Page 31: DCRUG: Achieving Development-Production Parity](https://reader033.vdocuments.net/reader033/viewer/2022061218/54b7aa754a795993718b4943/html5/thumbnails/31.jpg)
Thanks!
I love talking about Ruby and Rails, so feel free to reach out to me with questions!
email: [email protected]
Github: geoffharcourt
Twitter: @geoffharcourt