definition of quality (xp days ukraine)
DESCRIPTION
Доклад о внедрении способов оценки программных проектов и программных продуктов.TRANSCRIPT
Definition of Quality
Natalya Rukol, Quality Lab.
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
About speaker• 10+ years in IT• 9 years in quality
management• PM, TM, CEO• She is cool!
Introduction
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
About speech1. Why to
measure2. What to
measure3. How to
measure
Introduction
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Don’t measure!Why measure?
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
No visible failDon’t measure!
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Meet your fateDon’t measure!
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Make a real job!Don’t measure!
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Kill a weak teamDon’t measure!
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Leave it:Don’t measure??
Projects are usually dead without measurements
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
What to measure?
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
What to measure?Product quality
• Make clear decisions
• Visualize your goals
Project status
• Control• React in time
Team efficacy
• Better planning
• Improvement strategy
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Product qualityWhat to measure
How it looks
What’s inside
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
External qualityProduct quality
• Customers’ satisfaction• Business
results• Refunds (N &
%)• Support &
Hotline statistics
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
External qualityProduct quality
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
ExamplesProduct external quality
Metric 6.5 Release 7.0 Release Status
Customers’ satisfaction 4.4 4.2
Refunds 115 109
Refunds % 0,06% 0,11%
Support time per client, hours 6 8
Support tickets amount per month
4112 3640
Support tickets in % (bugs/questions/praises)
27/61/12 19/79/2 ?
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
ExamplesProduct external quality
Metric 6.5 Release 7.0 Release Status
Customers’ satisfaction 4.4 4.2
Refunds 115 109
Refunds % 0,06% 0,11%
Support time per client, hours 6 8
Support tickets amount per month
4112 3640
Support tickets in % (bugs/questions/praises)
27/61/12 19/79/2 ?
OK, and what???
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
How to useProduct external quality
• Track changes while it’s not too late• Investigate
WHY problems?• Ask boss for
increase
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Too late…Product external quality
Your software is
shit!
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Let’s go insideProduct quality
• Functionality (what?)• Non-functional
requirements (how?)• Usability• Performance• Etc.
• Stability (when?)
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Measure insideProduct quality
What How to measure
Functionality (What?)
• Requirements accepted
• Users scenarios execution
Non-functional requirements (How?)
• Load results• Performance results• Usability scenarios pass
ratio
Stability (When?) • Tests pass rate• Builds stability statistics• Bugs
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Internal qualityProduct quality
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
ExamplesProduct internal quality
Metric 7-11/09 14-18/09 20-24/09
Tests pass rate (% OK/Fail/Bugs) 70/12/18 68/13/19 69/12/19
Bugs total / criticals 612 / 28 654 / 19 602 / 24
Bugs average severity 3,7 3,2 3,8
Builds stable / unstable 11 / 0 9 / 4 6 / 7
Features accepted 19/21 17/21 20/22
Usability scenarios passed 9/10 n/a 8/10
Load results 18K 18K 6K
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Not relevantProduct internal quality
Your software is still shit!
100% pass rate, no
criticals, blah blah
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
ConsequencesProduct quality
How it looks
What’s inside
Customers satisfaction
• What do they like?• What they use
mostly?• What they don’t like?• Pain features?
• Critical bugs• Testing pass rate• Usability scenarios• Features accepted
Refunds • Why refunds?• Collect critics
• Critical bugs• Features accepted
Support tickets • What bothers most? • Usability scenarios
1. You have to find your own consequences! 2. First find, then prove!
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
ConsequencesProduct quality
Internal quality• Design• Code• Testing
External quality• Sales, Refunds• Satisfaction• Support cost
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Why external first?Product quality
• CLOKs & bugs are useless w/o customers & sales• No way to
forget priorities• Think big!• Tech depends
on business, not vice versa!
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
How to measure?
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Do it right!How to measure
• Pre-analysis• Introduction• Measurement• Control• Continuous
improvements
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Pre-analysisHow to measure
• Collaborate with the whole team• Remember The
Great Goal (TGG)• Find your
strategy together
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
Soft introductionHow to measure
• Don’t force• Share
responsibility• Find enough
time• Provide with
enough resources • Inside out
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
DisciplineHow to measure
• Automate all the measurements• Show your
respect• Grow a habit• Stand it for 2
months
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
PDCAHow to measure
• Plan, Do, Check, Act• Post-Mortems• Accept mistakes
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
ResultsQuality Definition:
• Clear view• Achievement
s• Team
collaboration• Karma
increase
XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality
What’s nextQuality Definition:
Contact with any questions:
http://natalyarukol.ru
http://nrukol.moikrug.ru
http://software-testing.ru
Skype: natalya.rukol
mailto: [email protected]
Additional materials:
http://natalyarukol.ru/definition-of-quality