5 steps to detecting issues earlier in your release cycles
Post on 09-Jan-2017
84 Views
Preview:
TRANSCRIPT
#beyondshi+le+ @perfectomobile
What We’ll Cover
• Our Climate Is Changing • Why Are Top Performers Winning? • Less Time Fixing, More Time CreaEng • Fast Feedback in 5 Steps
• Speak the Same Language • Write Stable Tests • Break Up the Monoliths • Test More in Every Build • Provide Context with Results
• Q & A
#beyondshi+le+ @perfectomobile
Our Presenters
Paul Bruce Developer Evangelist
@paulsbruce
Nick Sanjines Systems Engineer @NickSanjines
#beyondshi+le+ @perfectomobile
The Effect of “Climate Change” on App Development
Infrastructure is cheap
Releasing is easy
Faster to create apps
Automate everything
Release more frequently
People expect great apps
Maintenance is costly
At risk more o+en
Higher stakes for business
#beyondshi+le+ @perfectomobile
Top Performers
● Releasing is run-‐rate more o+en
● Manage change
early recovery
● Small batch sizes less lead Eme
● Less unplanned work
more new work
State of DevOps 2016, Puppet Labs
#beyondshi+le+ @perfectomobile
Less Cme fixing, more Cme creaCng,
From this: ● Unclear goals ● Technical debt ● Fire fighEng ● Release risk
To this: ● User saEsfacEon ● Be^er decisions ● Higher confidence
Defects
Backlog
Features
Time
#beyondshi+le+ @perfectomobile
How do we spend less Cme on re-‐work?
• Catch defects earlier • Simplify diagnosis and fixing • Automate repeEEve tesEng • Foster fast feedback
“As an engineer, you should constantly work to make your feedback loops shorter in +me and/or wider in scope. ” — @KentBeck
Source: IBM System Science InsEtute
#beyondshi+le+ @perfectomobile
5 Steps to DetecCng Issues Earlier
1. Speak the Same Language
2. Write Stable Tests
4. Test More in
Every Build
3. Break Monoliths Apart
5. Provide Context with
Results
#beyondshi+le+ @perfectomobile
Why speak the same language?
• Simplifies collaboraEon on tesEng • Enables tests to reuse code arEfacts • Makes code and tests equal ciEzens
Selenium
App Code Java, Javascript, C#, etc.
#beyondshi+le+ @perfectomobile
How do we write stable tests?
• Reduce technical gaps in object idenEficaEon • Clean setup / teardown procedures • Be as hermeEc as possible • Soak new tests before checking them in
#beyondshi+le+ @perfectomobile
Job Total CI Eme budget
Test Time (avg.)
Scope
Commit (VCS) 15-‐30 mins <100ms Unit, API, IntegraEon
Hourly 20-‐40 mins Varies New Tests, Key Smoke (BVT)
MulEple/day 30-‐60 mins <2s All Smoke, Key FuncEonal, Some Data
Nightly 2-‐7 hrs <120s All FuncEonal, All Data
Weekend 10-‐48 hrs all All Tests incl. Performance
Don’t leave all the regression tesCng to the end!
An anecdotal example, your own mileage will vary
#beyondshi+le+ @perfectomobile
What does “more” mean in terms of your audience?
Digital Test Coverage Index
Q4 2016, 5th EdiEon
bit.ly/DTC-‐index-‐5
#beyondshi+le+ @perfectomobile
5 Steps to DetecCng Issues Earlier
1.Speak the Same
Language
2. Write Stable Tests
4. Test More in
Every Build
3. Break Monoliths Apart
5. Provide Context with Results
top related