meetup fairfax lean agile engineering practices ci/cd continuous integration continuous delivery
DESCRIPTION
Meetup Fairfax Lean Agile Engineering Practices CI/CD Continuous Integration Continuous Delivery. Ramakrishnan Srinivasan, AOL ( [email protected] ) Thursday, September 18, 2014 *Everything I say reflects my own opinion only. CI/CD: We will cover. Why CI/CD? - PowerPoint PPT PresentationTRANSCRIPT
Meetup
Fairfax Lean Agile Engineering Practices
CI/CDContinuous IntegrationContinuous Delivery
Ramakrishnan Srinivasan, AOL ([email protected])
Thursday, September 18, 2014
*Everything I say reflects my own opinion only.
CI/CD: We will cover Why CI/CD? The development process & its goals Industry & business demands Discuss DevOps, CI, CD and available software The CI/CD setup Opscode Chef terminology & basics Chef Project structure & show a few screens References Q & A
We will speak to Operations folks Quality Assurance analysts Business managers & analysts Software engineers
DevOps Its about increasing efficiencies in the
development/QA/deployment process Its about increasing agility and output of the total system Emphasizes collaboration of Dev, QA and Ops QA personnel need to learn some software development in
order to automate testing Operations folks will find their tasks to be much easier There is an increasing demand for these skills
Typical workflow in s/w shop Developer works on a story to satisfy specific requirements Code is checked in to the repository Developer initiates a build Upon a successful build, QA team is notified QA installs the build in their environment QA tests the delivered functionality Bugs are reported & resolved Once cleared, its installed in production
The problems & motivation Code breaks. Defects are inevitable. No code change, but an external system has changed -
breaking our functionality We forget to run our unit tests before checking in code We want to know about issues sooner rather than later We want to be faster to market, deliver rapidly and often We want to automate deployment in a configurable way
Business demand: Efficiency We are constantly striving to improve productivity and
efficiency Developers
Make functions generic Write unit tests Code with a view to make debugging easy Only checkin code when
you are actually done, or you are sure of not breaking existing functionality
Efficiency .. 2 Testing: Automate where possible Goals
Reduce costs Decrease number of defects Increase productivity Spend more time building the product and creating value for the
customer Spend less time on functions/process that can be automated Do more with less
CI: A solution to improve efficiency Write unit tests with sufficient coverage for the important
parts Trigger a code build on every checkin or maybe every 5
minutes Run unit tests as part of the build Write integration tests for both UI and backend Run integration tests frequently Tests must be high quality & cover major functions Detect issues early, and as they happen
CD: Continuous Deployment Code and tests must be deployed before it can run CD is really part of the CI solution Adap.tv does several deployments to production on a daily
basis
CI Practices – Martin Fowler Maintain a Single Source Repository Automate the Build Make Your Build Self-Testing Everyone Commits To the Mainline Every Day Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment Reference: http://martinfowler.com/articles/continuousIntegration.html
CI/CD: Benefits Technical
Continuous delivery Standardized environments Less complex problems since they are identified quickly Faster resolution
Business Faster delivery of features More stable, controlled, and predictable operating environments Focus on adding business value rather than fixing issues Increased process effectiveness Increased agility
CI/CD: Summary Automate testing as part of the build Automate deployment to staging, test or production Automate rollback Deploy continuously to production Incremental new features initially enabled for small set of
users, then turned on for a wider audience
Software alternatives for CI/CD Plenty of alternatives available Opscode Chef Puppet Ansible Salt Cobbler
Here are a few: http://alternativeto.net/software/chef/
Software used in my project need to install Ruby Berkshelf – used to manage dependencies in Chef Chef Server on the server chef-client Jenkins SonarQube git
Servers used in a CI/CD setup Build server Deployment server Chef Server for provisioning & deployment Chef Client on each participating node Jenkins for scheduling and coordinating automated build
pipelines SonarQube: continuously analyze and measure code quality A single server can be used for many of the above Of course, servers VMs can be started up on demand in the
cloud
Chef topology
Chef basics/terminology - I Environment: settings, properties maintained in each
environment like Dev, QA and Production Bootstrap: first time setup Server: hub for configuration data Client/Node: a computer/laptop/server that can run chef-
client Workstation: its configured to use the knife command knife command: Used to activate Chef functionality
Chef components
Chef basics/terminology - II Based on the Ruby language Attributes: properties, key-value pairs Resources: commands, e.g. to run a shell script Templates Recipes Cookbooks Databags: JSON data accessible in a Chef environment that
can be encrypted. Libraries Overview: http://docs.getchef.com/chef_overview.html
Chef Attributes key-value pairs default values can be set Some values can be overridden in specific environments e.g. Connecting to Salesforce Dev, QA and Production instance URLs can be overridden in
respective environments Reference: https://docs.getchef.com/
chef_overview_attributes.html
Attribute locations
Example project Java Web application Apache Tomcat MySQL or Oracle Dev, QA and Production environments CI/CD pipeline
CI/CD Pipeline build the project execute unit tests for each build, create new linux RPM package deploy to CI server run smoke tests automated backend integration tests automated UI testing jobs (Selenium and CasperJS for
automated UI testing) deploy to production if delivery criteria are met
Chef Resources Template Package Directory bash, csh Service Cron Deploy Reference: http://docs.getchef.com/chef/resources.html
Templates these are static text files
with variables embedded in them
Node attributes can be referenced
to dynamically generate environment-specific files like scripts and config files
Recipes & cookbooks basic Ruby syntax the programming part of Chef has standard programming constructs like loops, conditionals, functions,
etc. a Recipe resides in a Cookbook - collection of recipes that belong together uses resources (like bash and service) to generate and install software download new package versions only when available Just enough Ruby for Chef Reference: https://docs.getchef.com/essentials_cookbook_recipes.html
Data bags global variables Chef-wide stored as JSON data Can be encrypted Can be accessed by recipes Reference: http://docs.getchef.com/
essentials_data_bags.html
Libraries Contain Ruby functions Provides the ability to execute arbitrary Ruby code Use to define custom resources, connect to a database, etc. Use to define template helper modules Reference: https://docs.getchef.com/
essentials_cookbook_libraries.html
Chef Project structure Project
attributes environments
Ruby classes Contain Ruby hashes (KV pairs) like a JSON dictionary or associative array Different format than attributes
templates (static text, config files) data_bags (encryptable JSON data) libraries (any Ruby code, helper functions)
Chef Development lifecycle Setup project (one-time) Create recipes, organized inside cookbooks Test cookbooks locally using Test Kitchen (good option) or
Vagrant Run cookbooks/recipes Check the target server If, e.g. a file is not in place as expected, fix recipe and repeat Once its working, push to git (or other repository) Run a recipe:
chef-client -o recipe[my_cookbook::my_recipe]
References Learn Chef: http://learn.getchef.com/ Test Kitchen: https://github.com/test-kitchen/test-kitchen Vagrant: https://www.vagrantup.com/ ThoughtWorks Treehouse blog ThoughtWorks: ThoughtWorks: http://www.thoughtworks.com/continuous-
integration How DevOps is killing the Developer Etsy Their story Continuous delivery with product/feature flags:
Continuously deliver with confidence Etsy deploys over 50 times a day Adap.tv engineering blog
Thank you!!! Questions Discuss Feedback