20141113 devoxx2014 jochim van dorpe testing in agile
DESCRIPTION
Testing in an agile context - Beyond the urban legend Talk from #Devoxx 2014 on November 13 by @jochimvandorpe on software testing in an agile context? What does it mean for testing methodology when switching from waterfall to agile? Have testers become redundant or rather an essential part of a software development team? How do you keep the pace when functional requirements are changing every two weeks? This presentation is giving no-nonsense advice from hands-on testing experience in a large ICT organisation. #agile # testing #Java #DevoxxTRANSCRIPT
![Page 1: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/1.jpg)
@ Smals_ICT#DV14 # AgileContextTesting @Smals_ICT#DV14 #AgileContextTesting
Beyond the urban legend
–
Testing in an agile contextJochim Van Dorpe
![Page 2: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/2.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
In-house ICT shared services for e-government
• Focus on social security and e-health services
• For federal, regional, local & European institutions
• Based in Brussels
• Software development, ICT-operations & staffing
• >1700 people
Introduction: What is Smals?
![Page 3: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/3.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: Who Am I?
(QA) ( (lead) (technical) test(er)
analyst/coördinator/manager) (automator/engineer)
I am a tester
![Page 4: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/4.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: What do I have to tell?
![Page 5: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/5.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
What is testing?
It's not about proving that the software works, it's about finding bugs
![Page 6: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/6.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Why is testing necessary?
To prevent:
• Death
• (Serious) injury
• Loss of business
• Loss of reputation
![Page 7: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/7.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Why do we test?
![Page 8: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/8.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
The test project in waterfall
![Page 9: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/9.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Agility
Agility is the continuous
delivery of prosperity (value) to
stakeholders of a system in a
sustainable and balanced
manner -Schalk Cronjé (@ysb33r)-
![Page 10: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/10.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Agile fundamentalists are like …
Did you even read
our manifesto?
![Page 11: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/11.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Agile manifestoOf course …
• Individuals & interactions over
processes and tools
• Working software over
comprehensive documentation
• Customer collaboration over
contract negotiation
• Responding to change over
following a plan
![Page 12: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/12.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Urban legends
• Agile doesn’t need testers
• Agile = de facto good code, built-in quality
• Testing is dead
• Agile = TDD, ATDD, BDD
• Testers should be able to code
• Agile hasn’t time for testing
• 100% automation
• Faster, shorter, better!
• Developers and Testers are like oil and water
• You only need to unit test
• User Acceptance Testing is no longer necessary
![Page 13: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/13.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
So…
• The Agile manifesto shouldn’t stand between you &
good quality
• No documentation, few documentation or poor
documentation is no excuse for “not testing”
• Don’t put an Agile stamp on your team, just to avoid
the things you dislike, but find a way of doing them
so you like them …
![Page 14: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/14.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
No difference at all?
![Page 15: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/15.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Many ways to be agile …
![Page 16: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/16.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
… but I’m no salesman
![Page 17: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/17.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Principles of testing in an agile
context
-Elisabeth Hendrickson (@testobsessed)-
![Page 18: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/18.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
1. Testing moves the project forward
Shifting focus:
No longer the last line of defense
![Page 19: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/19.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
2. Testing is not a phase
Integrated & continuous:
develop & test together
![Page 20: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/20.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
3. QA = responsability of the team
![Page 21: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/21.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
4. Reduce feedback latency
No long gaps between development
& testing feedback
![Page 22: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/22.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
5. Don’t let bugs crawl around
Deal with bugs as soon as possible
![Page 23: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/23.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
6. Test should represent requirements
Don’t ASSUME, it makes an ASS of
U & ME
SCHOOLBUS
![Page 24: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/24.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
7. Reduce overhead
Think high level: don’t waste time on
too much detail
![Page 25: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/25.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
8. Don’t believe in fairy tales
If something
seems to good to be true, it probably is
![Page 26: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/26.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
9. Done means done
Done means implemented and tested
![Page 27: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/27.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Concrete practices
![Page 28: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/28.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: This will drown your testers:
Manual tests
Acceptance tests
System &
integration tests
Comp.
tests
![Page 29: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/29.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: Make sure your testers can
cope with the pace
Component tests
System &
Integration tests
Acc.
Tests
Manual
testsManual tests
Acceptance tests
System &
integration tests
Comp.
tests
![Page 30: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/30.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate Testing, not only tests
Test
execution
ch
ec
kin
g
Load
Data-
sets
Reset
DB
Continuous integration
Logging
resultsGenerate
reports
Live
dashboards
Calculating
‘metrics’
Use the tools that fit your purpose
AND NOT THE OTHER WAY AROUND!
![Page 31: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/31.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
B. Explore
Leave some room for
exploratory testing
![Page 32: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/32.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
C. Diversify
• …in the test levels
• Unit-tests only won’t find
everything
• System test only, and you’ll
drown
• … in the way of testing
• Automation is good, automation
is fun, but in the end, an
automated test isn’t better than a
human based test (manual test)
![Page 33: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/33.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
D. Test the tests
• GREEN == GOOD ?
• DO: Review tests
• DONT: No test coverage fairy tales
• DONT: Mock lasagna
• DONT: I should adapt the test cases so it’s green again
![Page 34: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/34.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
E. Crowdsource …
• Crowdsource in your team to find
the good & passionate tester
• Accept that there are learning curves
• Testing is a craftmanship
• A passionate tester will tend to be more
effective than the chinese-volunteer-
tester
• Encourage collaborative testing
![Page 35: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/35.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
… or ask a testjumper
• Ask a ‘test jumper’ to join the
team
• May spend some hours as a consultant
• Or months as an ordinary tester
• Coaches or contributes himself
• Encourages testability
• Help the developers think
productively about testing
![Page 36: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/36.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
The test project in agile
![Page 37: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/37.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Metrics such as:
• # of bugs found
• Bugs per functionality
• Bugs per 1000 lines of code
• DDP (defect detection percentage)
• We can however show our QA-efforts
• QA-survey
• Test case progress
• Fixed test cases
![Page 38: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/38.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Qualification (good / bad) of the devs
• 100% green charts
• Upfront fixed number of test cases
• Automated self-generating auto-code-correcting
tests
![Page 39: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/39.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
conclusion
There are no agile testers,
but we need testers who
adapted their mindset to an
agile context
![Page 40: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/40.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 1/2• Books:
• Lisa Crispin, Janet Gregory (2009). Agile Testing: A Practical Guide for Testers and
Agile Teams
• Lisa Crispin, Janet Gregory (2014), More Agile Testing: Learning Journeys for the
Whole Team
• Blogs:
• Huibschoots.nl
• LisaCrispin.com
• http://testobsessed.com : Agile testing overview
• Http://www.satisfice.com : Test jumpers: one vision of Agile testing
• pascaldufour.wordpress.com
• agile-and-testing.chriss-baumann.de
![Page 41: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/41.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 2/2• Presentations:
• 40 agile methods in 40 minutes by Craig Smith
• Agile testing quadrants explained by @RubyTester
• Certification:
• CAT-training: Certified Agile Tester
• ISTQB foundation level add-on: Agile tester
• Conferences:
• Agile testing day(s)
• More references:
• Word clouds generated on wordle.net
• Images found with google image search
![Page 42: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.net/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/42.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Questions