Continuous Integrationwith Jenkins, Github & Grunt

- Integrate any changes within a matter of hours into the product

- Collection of tools and practices to make that possible

- Originating from XP, now an essential practice in Agile SD

What is CI?

RPM/Spectos Cloud

● LAMPP● MEAN● Ubuntu● AWS, EC2● Jenkins● JIRA● Github

Technology Stack

Motivation

- RPM: Frontend + 10 backend services- New features usually affect at least frontend

+ one (maybe 2 or 3) backends- How to know everything works well with

each other?- Who determines when we can ship to the

customer?

Practices

● Single Source Repository● Automate the build● Self-Testing● Commit often● Build every commit

Practices (cont.)

● Fix broken builds ASAP● Keep builds fast● Easy access to latest version● Everybody can see what’s happening● Automate deployment

Source Code Repository

- Not one “single” repo but one per project

- It’s all on https://github.com/Spectos (private, sorry)

Source Code Repository

- Mainline branch:

Source Code Repository

Sources

Docs

Dependencies

Tests Cronjobs

IDE Settings

Build Script

Config

Build Automation- Using Grunt: http:

//gruntjs.com/ - Javascript taskrunner,

based on Node.js- Simple syntax (JS)- Easy to wrap around

other tools (Cake Shell, bash)

- Lot of plugins

Build Automation- Database backups,

migrations- Generate locales (ng)- Run tests (mocha.js or

Cake shell)- Deploy via shipit.js (like

Capistrano)- Init work queues, clear

caches, etc.

Build Automation

// start development server

grunt s

// run all tests

grunt test

// deploy to stage

grunt shipit:stage deploy

Testing

Testing

- Automated tests for each functionality added (TDD is recommended)

- Using PHPUnit for PHP or mocha for Node.js apps

- Part of every build- Automated frontend tests with Selenium

(WIP)

Commit Often

- On your local: multiple times per day, easy to experiment and undo changes (git reset --hard ftw.)

- Remote: Pull & rebase often to avoid merge conflicts

- Push every feature when you consider it done so it can be integrated & tested

Fix Broken Builds ASAP- Emails are not very

effective- Desktop notifications,

sounds- Integrations (Slack)- Broken build is top

priority!

Speed

- Full commit build not > 10 min

- Currently: ~6 min for full ecosystem

- Component builds (one per service)

Speed - How To?

- Avoid expensive stuff in tests (I/O, large datasets)

- Run things parallel- Consider build

pipeline (not necessary yet)

Build Triggers

- Jenkins- Using Github hooks- Every commit pushed to

dev/master will trigger a build

- Production deploy: manual

Easy Access

githubmaster

development

production

FDB-123 rpm-dev.spectos.com/FDB-123/current

rpm-dev.spectos.com/dev/current

rpm-stage.spectos.com

rpm.spectos.com

Feature Branch

Branch Builds (Frontend only)

Improvements

- Travis style builds (.travis.yml)- Dynamic provisioning for build workers- Containers (Docker, …)- Frontend tests (Selenium, Robot)- Test reports, coverage reports, Sonar- Blue/Green deployment

Benefits- Reduced risk: find problems early, no unpredictable

integration phase (“Blind Spot”)- Helps to ensure code & functional quality- Debugging: smaller changesets, bugs don’t pile up- (Almost) no manual tasks, repeatable, less error-prone- Precondition for Continuous Deployment => rapid

feedback cycles, more focused development- Bottom line: less bugs, more features = happier

developers & happier customers

References/Further Reading

- Martin Fowler: http://www.martinfowler.com/articles/continuousIntegration.html

- Duvall, Matyas, Glover: “Continuous Integration: Improving Software Quality and Reducing Risk”

http://spectos.comhttp://jhenning.me

Thank You!