Driving Business Adaptability with SQAM: An Emerging Market

Driving Business Adaptability with Software Quality Analysis & Measurement:

An Emerging Market

Melinda Ballou Program Director, IDC

Application Life-Cycle Management & Executive Strategies Service

Feb-12 © 2012 IDC 2

Define and Understand SQAM

& Trends Driving Adoption

Evaluate SQAM Survey Results

Key Strategies Moving into 2012/13

Questions?

Agenda

Feb-12 © 2012 IDC 3

Industry Highlights: Disruptive Trends Driving SQAM Adoption

Diverse deployment demands for mobile, cloud, embedded drive corporate need for architectural impact analysis for application portfolio, business dynamism is enabled by software quality analysis

Organizations re-invest, seeking to do more with fewer resources with financial and staffing constraints; leveraging efficient approaches to restore and sustain high performing, timely, business-critical software.

Complex sourcing/off-shoring plus use of open source need strong teaming, effective code management, testing, and metrics enabled by SQAM; Services driven environment (SaaS/cloud, Devops emergence)

Global economic competition and local compliance across geographies demand quality, change and portfolio management, adaptability and rigor

Flexible development paradigm with services creation increasingly drive technology and business collaboration – strong agile emergence

Emerging security issues (as driver) and virtualization/cloud (as enabling technology) for SQAM adoption; ad hoc approaches unsustainable

End-user experience and business impact challenges of rich Internet, mobile, embedded, with social media collaboration/community opportunities

Feb-12 © 2012 IDC 4

• Software Quality Analysis and Measurement: software tools that

enable organizations to observe, measure, and evaluate software

complexity, size, productivity, and risk (including technical &

structural quality, non-functional testing)

• Architectural assessment of design consequences (on software

performance, stability, adaptability, and maintainability)

• Static analysis and dynamic analysis

• Quality metrics for complexity, size, risk, and productivity to establish

baselines and to help judge project progress and resource capabilities

• Application portfolio evaluation through understanding the impact of

architectural flaws and dependencies

• In-phase prevention of additional software problems not easily

observable through typical ASQ tools.

SQAM Definition Evolving Beyond Traditional ASQ

Feb-12 © 2012 IDC 5

Total Revenue for SQAM in 2010 = $356.3M

13.6% growth projected for 2011 to reach $406M

2008 = $279.1M; 2009 = $309.5M

Expected growth to $714M by 2015

CAGR for the forecast period (‘11-’15) is 14.9%

Top Five Vendor 2010 Revenues (narrow range):

CAST & Coverity @ around $39M

HP @ $38M & Parasoft $37M & IBM @ $36.5M

SQAM numbers for 2011 currently in process

SQAM Share and Forecast Summary

Feb-12 © 2012 IDC

SQAM Forecast Comparing the 2007 and 2011 Models

0%

5%

10%

15%

20%

25%

30%

0

100

200

300

400

500

600

700

800

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Gro

wth

M

illi

on

s o

f $

2011-2015 Forecast 2011-2015 Growth

6

Feb-12 © 2012 IDC 7

“Quality Gap”: High Cost of Failure

Poor Quality = Increased Business Risk

Lost Revenue

($$$$$)

Lost Customers

Lost Productivity

Increased Costs

Lower Profits Damaged Brand

Feb-12 © 2012 IDC 8

IDC SQAM Survey Demographics

200 companies (NA/SA 35.5%, EMEA 37%, Asia16.5%, CEMA 11%)

Majority very large organizations: – 5,000-9,999 employees (45.5%)

10,000-29,999 employees (32%); 30,000+ employees (21.5%)

IT employees: 100-299 (63.6%); 250-499 (17%); 500+ (19.5%)

Revenue: $2B-$3.9B (48%); $4B-$9.9B (29.5%); $10-$19.9B (9%);

$20B+ (12%); with around 95% currently using SQAM solutions

IT management 29%; IT ops 21%; App Owner 20%; Software dev 10%

Major industries: manufacturing, financial services, etc.

Self-described majority directors and managers (76%)

Key Drivers: Complex sourcing, business velocity, compliance, budget

Feb-12 © 2012 IDC

Demographics for User Role

9

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QS1. Which of the following statements best describes your involvement

with software quality analysis and metrics tools used in your organisation?

26.0%

35.5%

23.5%

15.0%

0.0%

0% 5% 10% 15% 20% 25% 30% 35% 40%

I use software quality analysis and metrics tools

I am responsible for business, cost, and vendor management issues related to application failures

and IT time to market

I influence or am involved in the purchasing process (recommending or sign-off) of software quality

analysis and metrics tools

I both use and am involved in purchasing of software quality analysis and metrics tools

None of the Above (neither use nor purchase)

Feb-12 © 2012 IDC

Team Distribution Broad Across Areas

10

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QA7. MEAN SUMMARY TABLE (INCLUDING 0) – How are your organization’s

software developers distributed among the following teams?

14.2

11.7

21.5

7.3

12.7

7.1

6.9

5.3

11.5

1.9

0 5 10 15 20 25

Architecture

Requirements

Development/Engineering/Modeling

Code analysis and assessment

Quality Assurance (QA)

Security

Software Change and Configuration Management

Release Provisioning and Operations

Maintenance

Other

Feb-12 © 2012 IDC

Complexity, Cost & Agility Drive Adoption

11

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC1. MEAN SUMMARY TABLE – How important to your organization are the following

factors as drivers in the adoption of software quality analysis tools.

2.1

2.2

2.3

2.3

2.0

2.3

2.5

2.9

2.3

2.0

2.2

2.3

1.0 2.0 3.0

Business consequences of poor quality code design (impact of production problems)

Increased costs due to constant application failures

Improvement in software development decision and planning process

Lowering of maintenance and performance costs and resource impact (detection and MTTR)

Internal and external customer satisfaction

Fit to existing systems and standards

Compliance initiatives (SOX, JSOX, Basel II)

Offshoring/Outsourcing oversight and management

Resource constraints (efficiency, productivity improvement and resource reallocation to innovation)

Security concerns

Business agility/speed of competitive response/compressed delivery cycle

Architectural complexity and increased resulting risk

Feb-12 © 2012 IDC

Resource Constraints Mid Complexity Create Challenges

12

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC3. Which of the following is the most significant challenge to the quality

of your organization’s software development today ?

18.5%

5.5%

8.0%

6.5%

8.5%

19.0%

12.0%

11.5%

2.0%

8.5%

0.0%

0% 5% 10% 15% 20%

Complexity

Outsourcing

Virtualization management

Multi-threaded software

Internal Staffing/Resources

Financial resources/Budget

Time to implement/Pace of change

Project prioritization

Poor architecture

None - No hurdle

Other (Please specify)

Feb-12 © 2012 IDC

Majority Plan SQAM Spending Increase

13

N = 200

Source: Custom Survey for CAST, IDC, December 2010

Combined totals across questions re: spending plans

79.5%

20.5%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Increase

Decrease

Feb-12 © 2012 IDC

Balanced SQAM Budget Split across Areas

14

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC7B. MEAN SUMMARY TABLE (INCLUDING 0)

How will your organization’s budget for software quality analysis and

measurement tools be distributed across the following functional areas in 2011 ?

20.2

18.7

20.0

18.9

21.4

0.9

0 5 10 15 20 25

Architectural Analysis & Risk Evaluation

Quality Metrics/Measurement

Application Portfolio Management/Application Portfolio Analysis/Software Dependencies

Code Analysis (Static & Dynamic)/Transactions/Sizing

Security

Other

Feb-12 © 2012 IDC

Context for SQAM Adoption

15

N = 200; Multiple Responses Allowed; Does not Sum to 100%

Source: Custom Survey for CAST, IDC, December 2010

QC9. Which of the following tools or approaches does your organization currently

employ in reviewing code and uncovering code problems for software as it is

designed and developed?

52.0%

29.5%

37.0%

32.5%

34.5%

50.5%

54.0%

28.5%

1.5%

5.0%

0% 10% 20% 30% 40% 50% 60%

Manual review/peer code review

Static analysis tools

Dynamic analysis tools

Application portfolio management tools

Architectural design tools

Unit testing

Functional testing

Virtualization for test labs and deployment

Other

None of the Above

Feb-12 © 2012 IDC

Positive Perspective on Defects

16

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC10. On average, how many architectural and other code problems requiring

patches are discovered in the 12-month period following release of the software

into production?

5.5%

19.0%

19.0%

16.5%

17.5%

3.0%

1.5%

18.0%

0% 5% 10% 15% 20%

None

1 to 10

11 to 25

26 to 50

51 to 150

151 to 500

More than 500

Don't Know

Feb-12 © 2012 IDC

Coupled with Increase in Challenges…

17

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC11. Over the past 2 years, has the amount of time an average developer in your

organization spent doing code analysis increased, decreased, or stayed the same?

4.0%

30.5%

49.0%

16.0%

0.5%

0% 10% 20% 30% 40% 50% 60%

Significantly Increased

Increased

Remained the Same

Decreased

Significantly Decreased

Feb-12 © 2012 IDC

… And High Optimism

18

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC12. How confident are you that your organization’s current code review process

identifies all potentially serious problems?

1.5%

9.5%

34.5%

31.0%

15.0%

8.5%

0% 5% 10% 15% 20% 25% 30% 35% 40%

Not at all Confident

2

3

4

Very Confident

Don’t Know

Feb-12 © 2012 IDC

Confidence Balance

19

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC14. How often does your organization’s quality analysis and measurement team find

problems, complexity and risks that were not found during code review?

3.5%

19.5%

42.0%

15.5%

5.5%

14.0%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Never

2

3

4

All the time

Don’t Know

Feb-12 © 2012 IDC

Decrease in Effort

20

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC15A. Over the past 2 years – Has the amount of time it takes to identify code

problems, fix them, rework and roll out new releases increased, decreased, or

stayed the same.

1.5%

20.0%

45.0%

21.0%

5.0%

7.5%

0% 10% 20% 30% 40% 50%

Significantly Increased

Increased

Remained the Same

Decreased

Significantly Decreased

Don’t Know

Feb-12 © 2012 IDC

More Decreases Expected

21

N = 200

Source: Custom Survey for CAST, IDC, December 2010

QC15B. In the next 2 years – Will the amount of time it takes to identify code problems,

fix them, rework and roll out new releases increased, decreased, or stayed the

same.

2.5%

14.5%

44.5%

25.5%

6.0%

7.0%

0% 10% 20% 30% 40% 50%

Significantly Increased

Increased

Remained the Same

Decreased

Significantly Decreased

Don’t Know

Feb-12 © 2012 IDC

Context for Purchasing

22

N = 200; Multiple Responses Allowed; Does not Sum to 100%

Source: Custom Survey for CAST, IDC, December 2010

QC16. Which of the following individuals in your organization

are involved with introducing and investing in software quality

analysis and measurement tools and processes?

45.5%

18.0%

60.0%

46.0%

22.0%

26.0%

12.0%

22.5%

28.5%

11.0%

12.5%

12.5%

2.0%

0% 10% 20% 30% 40% 50% 60% 70%

Senior IT executive (CIO, CSO, CTO)

VPs

Director of IT

Development Manager

Network Manager

Security Manager

Desktop Manager

Architect

Other IT Manager

Senior non-IT Executives (i.e. CEO, …

Line of Business Managers

Procurement or Purchasing …

Other

Feb-12 © 2012 IDC 23

• Survey provides context for current adoption patterns

• Challenges exposed – complexity, agility, security and

financial constraints play role

• Optimism increase for defect problems (overly exuberant or

more efficient?)

• Future plans and overall purchase increase laid out

• Survey supports findings for SQAM market growth

Survey Summary

Feb-12 © 2012 IDC 24

IDC Survey Calls to Action

• The challenges of increased complexity and high-end

development across diverse platforms increase code

problems, increase costs and drive debilitating consequences

resulting from defects pre- and post-deployment

• Companies must become better educated about the business

consequences and labor costs of poor software design since

optimism mask the need for change

• Organizations should evaluate SQAM tools to supplement

traditional ASQ along with appropriate process and

organizational approaches

• Across industries, poorly designed and problematic software

leads to brand perception impact above and beyond individual

problems – demand response

Feb-12 © 2012 IDC 25

Goals of Effective IT/Business Alignment

New Business Value

Reduced Exposure

Innovation: Maximize Upside

Through Technology- Enabled Business

Processes

Compliance: Minimize Downside

Through Risk Management

Feb-12 © 2012 IDC 26

IT and Business Challenges: Silos, Gaps

Today’s applications are high-visibility, and carry a high cost-of-failure -- customer self-serve, supplier/channel; key internal business applications

“Network effect” – failure in one leads to other failures

The need for SQAM as part of quality life-cycle is key since G2000 organizations are split across groups:

– Business/users stakeholders

– Architects, Designers and Developers

– QA professionals

– Operational staff

Must extend the Quality life-cycle across geographies, life cycle phases and groups

Feb-12 © 2012 IDC 27

Summary

Coordinate a Quality Life-Cycle approach that targets pragmatic approaches to SQAM from design through to deployment to obtain benefits

Evaluate your organization’s current strategies for design, application portfolio review, effective quality processes and automated tools adoption

Schisms between business, architects, development, testers and operations must be addressed -- IT groups and the business must build a common language, common metrics, and common tools and practices that include SQAM

Drive towards an effective quality strategy to help cut costs, increase efficiency and business agility, to sustain brand, address competitive challenges

Align Vendor SLAs with Long Term Value

with Steve Hall, author of "Managing Global Development Risk”

and Partner at ISG (formerly TPI),

a leading research, consulting and advisory services firm

Thursday, February 16th 11am-12pm EST (9:30pm IST, 5pm CET, 4pm UK, 4pm GMT, 8am PST)

Steve Hall will discuss the challenge of aligning vendor SLAs with long term business value.

He will provide details on how you can build healthier and transparent relationships with

vendors by incorporating application structural quality measurement and practical,

meaningful metrics to mitigate risk and maintain value from vendor relationships.

You’ll learn how you can avoid vendor lock-in, improve production support activities

and align metrics between vendors and project managers.

Upcoming Webinar

To learn more about CAST

Pete Pizzutillo

p.pizzutillo@castsoftware.com

www.castsoftware.com

blog.castsoftware.com

slideshare.net/castsoftware

Twitter: @OnQuality

To view the entire webinar including Q&A click here New IDC Research on Software Analysis and Measurement

Learn more about CAST

www.castsoftware.comblog.castsoftware.com

www.facebook.com/castonquality www.slideshare.net/castsoftware www.twitter.com/OnQuality