acc presentation for qa club kiev
DESCRIPTION
TRANSCRIPT
Balancing Your Test Effort
Planning test with Google’s approach
Nikita Knysh
Ciklum, August 17, 2011
The Speaker
• Nikita Knysh, 30 y.o., ~11 years in IT
• Background: IT education, webdev, support+lead, PM, TW, BA+lead, ISTQB FL
• Now: 4 years with Scanjour, Test TL
Agenda
• The Challenge• The Directions• The Method• The Tools• Probs & Cures
The Challenge
• What to test if not enough time?• What to test if have buffer?• Much test, few bugs• No overview of test needs• Not enough sync between roles
The Directions
• Discuss• Summarize• Prioritize
… team-wide & easily
So what’s the right approach then??
The Method: Origin
ACC (Attribute-Component-Capability)
James A. Whittaker,
Test Director at Google,
2010
Microsoft and Google use this.
Can we?
The Method: Concept
• List product’s selling points (Attributes)• Break down the product into tech
Components. • Break down the product based on
WHAT it does (Capabilities).
We get a model that reflects all
the vital views on the product!
The Method: ACC Modeling
• ACC list
• Time to give it some numbers!
The Method: Giving it Numbers
• Complexity
increases risk of human mistakes during code development and maintenance and therefore risk of introducing bugs
• Use full scale (1 to 5)• Track averages
The Method: Giving it Numbers
• Complexity factors
code logicbusiness logic
how much UI?
how much error handling?
how easy to debug?quality of requirements
tech expertise
unit test coverage
auto test coverage
code ownership
architecture
The Method: Giving it Numbers
• Frequency of Use
how often the capability is called by user or automatically and therefore how often failures caused by defects in code will most likely occur
The Method: Giving it Numbers
• User Impact
damage dealt to user and / or system intent should the capability fail completely or severely
The Method: Outcome
• Testing Needs
= Complexity * Frequency * Impact
Now we know where the risk is.
Now we know where we need more test.
and…
Our knowledge is based on cumulated vision of the whole team!
…and it is extremely easy to overview!
The Method: Outcome
• ACC list, now with numbers
The Method: Outcome
• Matrix view of capability count
The Method: Outcome
• Matrix view of testing needs
The Tools: How We Do It
• ACC items are TFS work items• ACC linked to TCs and bugs for metrics• Excel book for each model• Two-way sync between TFS and Excel• Instant update: DWH cube is avoided• Pivot tables and charts
make the beauty
The Tools: How It Looks
• Model overview in Excel
That easy?
Really?
Probs & Cures
• “No time for modeling!”. Include into DoD.
• “It shows what we knew!”
Be strong! Don’t manipulate!
+True for any document.
More Probs & Cures
• Can’t test individual capabilities. Reconsider product breakdown.
• Hard to bind test cases and bugs to capabilities. Reconsider model’s detail level. Think starting from ACC, not test cases.
Thank you!
• Questions & Answers