specification by example and agile acceptance testing
DESCRIPTION
Specification by example and agile acceptance testing, presentation given to HSBC developers on 21/09/09 for more info see http://specificationbyexample.comTRANSCRIPT
Specification by Example and Agile Acceptance
Testing Bridging the communication gap in
software projectsGojko Adzic
[email protected]@gojkoadzic
http://gojko.net
Why should you care (PM/BA)?
• Developers will actually read the specifications
• They will understood the stuff correctly
• They will not skip parts of the specs
• You can track the development progress
• Save time on acceptance/smoke testing
Why should you care (dev)?
• Requirements will be unambiguous and without functional gaps
• Business analysts will really understand those special cases you mentioned
• You will have automated tests to guide development
• It will be easier to take-over and hand-over code
Why should you care (QA)
• Finally stop those guys from making the same mistakes over and over
• Avoid doing the same stuff all the time
• Build quality in from the start
• Verify business rules by a click on a button
http://www.flickr.com/photos/lambdachialpha/157986473/
http://www.flickr.com/photos/mulesafpilot/3513588967
http://www.defenseimagery.mil/assetDetails.action?guid=68cf92e35ce13e6c7c9c066f0b48b6daaa9bf8d8
An experiment with four active battalions in US Army
“Commander expectations matched actions in only 34% of the cases”
L.G.Shattuck, 2000http://www.au.af.mil/au/awc/awcgate/milreview/shattuck.pdf
http://www.flickr.com/photos/mataniere/3107073262
The process is very much like a telephone game
How many points are there?
How many points are there?
“Just-in-case code is the biggest source of waste
in software development”
Mary and Tom Poppendieck
Lean Software Development
B2 bomber crashed and $2bn went up in flames
"the aircraft actually performed as it was designed. In other words, all the systems were
functioning normally."
Maj. Gen. Floyd L. Carpenter
http://www.foxnews.com/wires/2008Jun05/0,4670,B2Crash,00.html
http://www.flickr.com/photos/biolog/3457774800
You can't help a lot when the party is already over...
F-16 design team was asked to do the impossible -
a cheap 2.5 Mach airplane! “When asked […] why they need Mach
2 - 2.5, the answer was to be able to escape from combat. Their solution was […] providing acceleration and
maneuverability, not maximum speed.”
http://97-things.near-time.net/wiki/Seek%20the%20value%20in%20requested%20capabilities
Refuse requirements that are a solution to an unknown problem!
http://www.flickr.com/photos/sylvancatharsis/3783608640/
One of the most effective ways of testing requirements is with test cases very much like those for testing the completed system
Donald Gause and Gerald Weinberg
Exploring Requirements - 1989!
As formality increases, tests and requirements become indistinguishable.
Robert C. Martin and Grigori Melnik
Tests and Requirements, Requirements and Tests: a Mobius Strip
IEEE Software January/February Issue 2008
Key practices• Discuss real-world examples to build a
shared understanding of the domain
• Use those examples as an acceptance criteria
• Automate acceptance tests
• Focus the development on those tests
• Use the tests as a live specification to facilitate change
Better names• “Acceptance testing” is
misleading!
• Behaviours
• Executable specifications
• Examples
Jim Shore: “Describe-
Demonstrate-Develop”
A very useful way to think about acceptance tests in practice
http://www.jamesshore.com/Blog/How-I-Use-Fit.html
Iteration flow (just a suggestion)
Step 1:Building a shared
understandingof the domain
Specification workshops
• Business people, developers and QA in the same room
• Transfer the knowledge• Ensure that we all understand
the same thing
Inconsistencies and gaps are easy to spot
when you write the rules down!
Real-world examples help flush out
incorrect assumed rules find real business rules!
People have think at a more detailed level
and can't brush questions off…
People approach the same problem from
different perspectives, so this avoids
groupthink!
Step 2:Select a formal set of acceptance tests and
automate them
The Toyota Way1. Check at the source2. Inexpensive Verifications3. Test to prevent defects,
not to discover them
Save time on manually (acceptance/smoke)
testing
Verify business rules with the click of a button
Automate tests, but still keep them human-readable
And you can add pictures as well….
This is a specification:
It is also a test
This as well
And this
And this
Given a stock of prices 0.5,1.0
When the stock is traded at 2.0
Then the alert status should be OFF
When the stock is traded at 5.0
Then the alert status should be OFF
When the stock is traded at 11.0
Then the alert status should be OFF
A good acceptance test is• Focused on a single thing (rule, step..)
• Specification not a script
• Self-explanatory
• Uses the domain language
• SMART– Specific
– Measurable
– Achievable
– Relevant
– Time-bound
Step 3:Providing focus for
development
No just-in-case code
Developers will have to code exactly what was
specified… not just the rules they see
Automated testreports show where
we are…
When all the tests are green, the job is done
Step 4: Keeping in touch with
changes
Live documentation
As relevant and reliable as executable code, but much easier to read!
Previous examples help you ensure to
discuss all important edge cases.
Automated tests show straight away when
something is obsolete or broken
[tests became a]
“significant and valuable business resource”
Rick Mugridge, Doubling the Value of Automated Tests,Google Tech Talks 09/2006
Recap (PM/BA)
• Developers will have to read the specifications to implement tests
• Discussion makes sure that everyone understood the stuff correctly
• To make all tests green, they cannot skip parts of the specs
• You can track the development progress
• Save time on acceptance testing with automated verifications
Recap (dev)
• Discuss and suggest examples until the gaps and inconsistencies are flushed out
• Make sure that business analysts understood special cases by suggesting them as examples and discussing them
• Acceptance tests are a live specification/documentation for the code
Recap (QA)
• Suggest examples and discuss them to cover mistakes that people make over and over
• Automated tests help you avoid doing the same stuff all the time
• Build quality in from the start by suggesting tests that prevent problems
• Verify business rules by a click on a button
Where next?
http://gojko.net http://agiletesting.org.uk http://acceptancetesting.info
27 Nov – Agile specifications and testing exchange
30 Nov – Agile acceptance testing workshop
4 Dec – Building software that matters