[rakuten techconf2014] [g-4] beyond agile testing to lean development
DESCRIPTION
Rakuten Technology Conference 2014 "Beyond Agile Testing to Lean Development" James O. Coplien (Gertrud & Cope)TRANSCRIPT
Beyond Agile Testing to Lean Development
James O. Coplien Gertrud & Cope
Processes and Tools over
Individuals and Interactions
Usability Testing
System Testing
Exploratory Testing
Test Testing
Why doesn’t unit testing work?
We can’t test things of business value
Not knowing what to test, we test many things that don’t need testing
Automated crap is still crap
Unit tests can’t test things of business value
Can you demonstrate that Method X of Object Y will ever be invoked in practice?
If a unit test fails, how does it affect ROI?
What’s amazing is that we continue to test… Paths that will never be exercised Methods that are never used Naive method invocation sequences
Most unit tests are unnecessary
We test scenarios that will never be exercised in the real world
We test methods that will never be invoked
We supply data that a real application would never have to process
Automated crap is still crap
Design requires intelligence: intelligence cannot be automated
Autonomation instead of automation
Testing is about exploratory inference
Get manual procedures working first and then automate the mundane parts
Automated testing is not testing
Nokia Test Score 2: Testing
No dedicated QA - 0
Unit tested - 1
Feature tested - 5
Feature tested as soon as completed - 7
Software passes acceptance testing - 8
Software is deployed - 10
Weinberg’s Law of Decomposition
Weinberg’s Law of Decomposition
Used
Int
erfa
ce
Used Interface
Must unit-test!
Must unit-test! Must
unit-
test!
Weinberg’s Law of Decomposition
Use Size
Test Size
Wasted testing effort
Part III: You Will Have Bugs!
Scenarios we test in the lab
Scenarios exhibiting
faults
Scenarios invoked in the field
Testing waste
Scenarios we test in the lab
Scenarios exhibiting
faults
Scenarios invoked in the field
Type 2 Muda
Type 1 Muda
Type 4 Muda
Type 1: Defects
Type 2: Stuff customers
don’t ask for
Type 4: OverprocessingType
1 Muda (field)
Removing tests leaves field faults
undetected!
Why do we unit test?
Because we can
It doesn’t require teamwork
We don’t have to talk with the business
We get to interact with tools instead of people
Our decisions are emotional, not rational
Reduction of WasteType 1: Better testing (e.g., going beyond contextual testing to exploratory & experience-based testing) (and better yet, better analysis and design, where the real payoffs are)
Type 2: Don’t test dead code
Type 4: Remove redundant and tautalogical tests, and tests that give no information
Type 1 in the field: Ship your tests!
Don’t take the tests out when you ship!Remember, you will have bugs in the field!
Deployment is your largest test driver
Ship your tests as part of the code
IN case of a failure, it saves you from Type 9 muda (unsafe execution)
Be Customer-Focused
Assertions and Design by Contract!
How Adding ASSERTS reduces work
0
500000
1000000
1500000
2000000
2500000
0 1 2 5 10 20 50 100
15 bugs p=0.01 15 bugs p=0.0210 bugs p=0.01 10 bugs p=0.025 bugs p=0.01 5 bugs p=0.02
number of ASSERTS
Wor
k, T
sing
lech
ecks
Overload Magazine,
October 2014
Defensive Programming
… is largely the incorporation of tests in the code
Long known for high-reliability systems
The value of asserting the obvious
Design by Contract
Assertions make code readable!
- (void) prepareForSegue: (UIStoryboardSegue*) segue sender: (id) sender { assert(segue != nil); if ([[segue identifier] isEqualToString: @"SegueAvatarPickerToPickOneLoginPlayer"]) { assert((UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone)); // The field parentViewController_ (instance variable) is // maybe null if we are in iPhone mode and doing a real // segue, instead of a popover as we do on the iPad. So // get the right object from the segue record. KnowsyPlayerPickerCommonViewController *parentViewController = [segue destinationViewController]; [parentViewController setKnowsyData: knowsyData_]; } else { // iPad - no special processing - should be no popover assert(parentViewController_ == nil); } }
Part IV: Beyond agile testing, Lean testingWhen a test finds a bug, it’s about fixing the process — not the bug
The long-term payoff is higher!
No bug-tracking systems: we fix problems NOW.
The tester should be the team’s conscience
Other Lean-isms
Deming: Quality is inversely proportional to the number of testers
GM kept around its cars for 1 - 6 weeks for test and repair: Toyota drove them off the end of the assembly line onto the car carriers.
Conclusion
Be humble about what tests can accomplish
It’s not about testing, but value
Design tests to assess properties of business value
Go beyond unit testing to defensive programming and design-by-contract
As agile testers, focus on process first and product second