qa 2010 stki
DESCRIPTION
מצגת המראה מגמות ומספרים המתייחסים לשוק הבדיקות והבטחת האיכות בישראלTRANSCRIPT
QA Trends
Galit FeinEVP & Senior Analyst
Office of the CIO Strategies
www.galitfein.blogspot.com
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI Do not remove source or attribution from any graphic or portion of graphic 2
Agenda
QA & Testing Services Israeli Market
Factors Influencing QA Staffing Level
IT Trends Influencing QA Staffing Level
Pricing Models
Galit Fein’s work Copyright 2010 @STKI 3Do not remove source or attribution from any graphic or portion of graphic
QA & Testing Services Israeli Market
2008240.0 -10.42% 215.0 13.95% 245.0 2.04% 250.0 2.00% 255.0
20.0 -10.00% 18.0 38.89% 25.0 40.00% 35.0 20.00% 42.0 Project Management Office
2009 2010 2011 2012Q/A & Testing
Software TestingOutsourcing:
An IdeaWhose Time Has
Come?
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 4Do not remove source or attribution from any graphic or portion of graphic
QA & Testing Main Players
Matrix
MALAM/TEAM
NESS
Tesnet
Tescom
Mertens Hoffman
Calanit Carmon
Taldor
Top Group
One1
Netwise
EL AD Systems
Category 1
Category 2
Category 3
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 5Do not remove source or attribution from any graphic or portion of graphic
QA Staffing Level
The skills of a capable QA group are essential to
improving application development & maintenance:
• Early discovery of inconsistencies
• Errors reduce
• Rework prevention
Yet, the appropriate level of QA
staffing is often difficult to determine
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 6Do not remove source or attribution from any graphic or portion of graphic
Factors Influencing QA Staffing Requirements
When ITO requires higher level of QA staffing:
Orgs that produce their own SW other than orgs that
implement commercial SW packages
Orgs relying on SW for mission-critical functions
Highly regulated industries
Orgs that produce SW as a part of their services to customers
Companies devoting their efforts to steady process
improvement (Six Sigma/ CMMI…)
Galit Fein’s work Copyright 2010 @STKI 7Do not remove source or attribution from any graphic or portion of graphic
QA & Testing Ratios
1.70%25 percentile
3.40%Median
5.30%75 percentile
Per FTE
425 percentile
6Median
1875 percentile
• SW developers per QA FTE
• QA unit* ratio to total IT
*Includes employees who are part of a dedicated QA or testing function. It does not include employees who perform testing as part of their other job responsibilities, such as application programmers who also perform their own unit testing
Source: ComputerEconomics
- Do some orgs have lower needs & are they more efficient in their use of QA staff ?
OR- Are they understaffed in this position?
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 8Do not remove source or attribution from any graphic or portion of graphic
QA Staff Ratio / total IT
3.90%
2.20%
0.70%
Finance
High Tech
Government
Source: ComputerEconomics
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 9Do not remove source or attribution from any graphic or portion of graphic
Find The Right Balance
Too often QA is viewed as an obstacle to getting work done
Desire to
meet
deadlines &
enhance
systemsNeed to
ensure
quality,
reduce
waste &
enforce
standards
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 10Do not remove source or attribution from any graphic or portion of graphic
IT Trends Influencing QA Staffing Requirements
New regulations
Cloud computing
Agile
Outsourcing
Office of the CIO
Agile
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 11Do not remove source or attribution from any graphic or portion of graphic
New Regulations Requirements
In recent years, the quality & integrity of information
systems are being examined more closely because of
regulations that address the security and privacy of
information
QA function becomes even more important in light of
these requirements
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 12Do not remove source or attribution from any graphic or portion of graphic
Office of the CIO Israeli Trends
After many years of talking OTC is actually happening!
Financial crisis & budget reduction in 2009 forced better ITG:
• Increased focus on IT costs - efficiency increasing
• New regulations requested better transparency
Most ITOs have expanded their OTC/ PMO department:
• IT budget control & mng, PPM, rewrite methodologies & WF definition,
tools evaluation, HR - resource allocation & mng, QA, GRC…
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 13Do not remove source or attribution from any graphic or portion of graphic
The PMO goals are:
Keep all this in the right balance:
• Scope of Work
• Budget
• Time
• Quality
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 14Do not remove source or attribution from any graphic or portion of graphic
SW metrics
Bugs - How many bugs (including severity)
• When the bug was open, when it was closed
• Number of bugs correspondingly to closing date of the version
• When was the bug found(dev, functional test, regression, prod = escaping bugs)
Test coverage (what part of SW was tested)
Percentage of automatic tests (from the complete project = automatic +
manual)
Progress of project compared to plan
How many (and percentage of) passed builds
Development of new functionality vs. effort to correct bugs from prod &
from tests
STKI recommends: change your metrics once in a while
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 15Do not remove source or attribution from any graphic or portion of graphic
SW Measurement Insights
Bugs are the most easy metric to measure & track
Sometimes too much effort is spent on bugs (example:
minor memory leak) while important functionality is
delayed
How much effort should developer put in “non-dev”
missions (like detailed progress report)?
Should maintenance be part of the general development
team or should maintenance have a special “team”?
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 16Do not remove source or attribution from any graphic or portion of graphic
Agile Project Management
Analy
sis
Code
Test
Implem
ent
Analy
sis
Code
Test
3 6 9 12
36
912
Source: Cutter
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 17Do not remove source or attribution from any graphic or portion of graphic
Agile SW Development Israeli Trends
Agile is catching up in ITOs
Clients see the benefits of Agile in SMPs
Large Agile projects (50 people) are
more challengeable mainly when there is high
interdependencies between the SW modules
Israeli ISV’s report that sometimes their clients demand Agile
Agile requires change in the hierarchical structure that is
extremely difficult to large conservative organizations
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 18Do not remove source or attribution from any graphic or portion of graphic
Outsourcing
While Professional Services market significantly dropped
last year, the QA & testing PS remained stable
QA - rapidly growing sourcing segment
Acknowledging the advantage & cost saving (up to 50%)
by local nearshore
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 19Do not remove source or attribution from any graphic or portion of graphic
Israeli Nearshore
Israeli ITOs do not offshore QA, except global orgs
Israeli nearshore existing clients have broaden their
scope
Added Value Professional Services :
• Best Practices – ISTQB, ISO 27001
• Automation Testing Tools
ISTQB
ISOCMMI
Six
Sigma
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 20Do not remove source or attribution from any graphic or portion of graphic
Israeli Nearshore Main Players
Matrix Global (Talpiot)
Ness
Malam Team
Ness
Aman – Tehila
Manpower
Galil Software
Many more…
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 21Do not remove source or attribution from any graphic or portion of graphic
Pricing Models
Traditionally QA was priced on a fixed-time/fixed-price
basis (most tenders still are)
Today:
There is a real difficulty to evaluate QA scope of work,
therefore most contracts priced on time & material basis
Israeli ITOs prefer in-house responsibility & QA PS, both
• On premises
• Off premises – nearshore
Your Text hereYour Text here
Galit Fein’s work Copyright 2010 @STKI 22Do not remove source or attribution from any graphic or portion of graphic
Developers vs. Testers
Advantage:
Already knows the system
- Driven by “delivery”
Disadvantage:
Must learn the system
- Seek the problems
- Driven by “hours”
But But