SwissQ Requirements Trends &Benchmarks Switzerland 2012Where are we now – where are we going to?

SwissQ Requirements Trends & Benchmarks 2012 2TABLE OF CONTENTS

3

4

5

6

7

8

9

10

11

12

13

14

15

EDITORIAL

TRENDWAVE 2012

KEY MESSAGES

PROJECTS

QUALITY

EFFORT

MATURITY LEVEL AND SUCCESS FACTORS

ORGANIZATION AND TRAINING

AGILE REQUIREMENTS ENGINEERING

CHALLENGES

TOOLS

FRAME OF SURVEY

TESTING AND AGILE

It leads to discontent with the RE if this prioritization is not or only insufficiently carried out (30 % believe the maturity level of their RE to be weak or very weak). Here lies the task of the requirements engineers (or the business analysts, product owners, etc.) to use appropriate methods to get the estimates from the stakeholders.

Another reason for insufficient RE is the inappropriate use of tools. Most res-pondents (>85 %) still use Microsoft Office for RE tasks. It is obvious that the traceability poses the biggest challenge here (see p.12). Integrated tools, so called application lifecycle management tools are increasingly important as proposed solutions. Sooner or later, the tools question for an efficient process support in RE will become a focus because of the increasing complexity and range of functions of applications.

As Heraclitus already noted, the world is in constant motion. With the SwissQ Trend Wave® (well-known from the Testing Trends & Benchmarks) you can see the changes in the RE market. Together with other results from this report they are a guide through this storm of changes. This report will therefore be published on a regular basis in the future. In that sense, SwissQ wishes you many interesting insights and hopes you enjoy the reading.

“The only constant in the universe is change.”

Over 2500 years ago, Heraclitus from Ephesus already hit the nail right on the head. In order for you to have an overview of these changes and to be able to systematically shape the process of change, SwissQ created the “Requirements Trends & Benchmarks Report“ at hand. The report is based on a survey completed by over 300 participants from the Swiss IT Community. In addition, we also personally interviewed numerous IT decision makers from various companies, branches, and regions about the current trends. From this developed a representative overview about the current state of Requirements Engineering (RE) in Switzerland in the year 2012 and an outlook on the most important future trends. Let yourself be surprised by the interesting results.

SwissQ Requirements Trends & Benchmarks 2012 3EDITORIAL

Also in Switzerland, only 35 % of all projects are finished in scope, in time and in budget. These results approximate to the international situation as published in the “Chaos Report“ by the Standish Group. The situation has improved slightly compared to previous surveys but still a majority of the projects are threatened by failure. The reasons are many and varied.

The systematic analysis of stakeholders for example – who, by the way, are a central source for requirements – seems to be a necessary evil instead of a success factor for many respondents. Around 1/3 does not invest any time into this analysis as the stakeholders are assumed to be a given. It is therefore not surprising that almost 70 % are not or only somewhat satisfied with the elicitation of requirements. The insight that the stakeholder analysis is important for the success of the project doesn‘t seem to be really established, it only came in in fifth place in the poll. So it seems only logical that ever changing or growing requirements for the system are named as the a reason for insufficient requirements by over 75 % of all respondents.

The missing business value of requirements – In addition to insufficient requirements – poses a problem for over 50 % of the respondents. This is very surprising especially since agile methods have been introduced in businesses long ago (75 % of all respondents have already worked with agile methods) and the focus on business value is a central element of agile projects. Meanwhile, tested techniques are on the market – such as for example “Priority Poker“ by SwissQ – which can efficiently prioritize requirements according to their business value.

SwissQ Requirements Trends & Benchmarks 2012 4TRENDWAVE 2012

INTRODUCTION – This topic has been identified and some companies are deploying initial implementations. However, it cannot be foreseen whether this trend will positively advance and whether testing will be considerably influenced.

GROWTH – This topic is more and more accepted and many companies are considering it. The first tools are being developed and consultancy firms offer services for the same. Often risks are associated due to limited implementation experience.

MATURITY – Most companies are working on the implementation or have already completed it. The knowledge of this topic is often widespread, resulting in sub-topics being raised.

DECLINE – The topic has already been implemented by most of the companies, with the exception of individual latecomers. Often, there is no more added value in acquiring further knowledge in these areas, since it will become obsolete shortly.

INTRODUCTION GROWTH MATURITY DECLINE

TIME

PRIO

RITY

Requirements Modeling

RE Processes/Roles

ALM Tools

RE Outsourcing Acceptance Test Driven Development (ATDD)

IIBA CBAP

IREB CPRE AL

Planguage

Business Value

Agile RE

Phrase Templates

IREB CPRE FL

RE PoolsUse Case Specifications

RE Mgmt Tools

MoSCoW Prioritization

RE Workshops

Reviews

4 5 6Requirements Engineering has a low priority in companies or is even regarded as a necessary evil according to almost half of all respondents.

The professional image of a Requirements Engineers / Business Analyst seems to be established on the market. This is also partly due to the training that has been standardized by IREB in the past five years.

More and more is being invested into the collaboration between Business and IT, and into the training and the standardization of requirements processes. All this at the cost of outsourcing and the building of organisational Requirements Engineering units.

7 8 9Over 36 % do not check their requirements for need whereas functionality and feasibility are checked by more than 80%.

Over 2/3 invest less than a day in the stakeholder analysis. This is surprising since the stakeholder analysis is generally considered an important success factor.

Misunderstandings in commu-nication and the ever changing requirements for the complete system are the key reasons for insufficient requirements.

2 31 Only 25 % of all respondents think their requirements engineering is good or excellent. The most important improvement measure named is the standardization of requirements processes and tools.

Top strategic goals in 2012 are Agile Requirements Engineering and Business Process Driven Requirements. Agility is on the rise here as well.

Modelling requirements and defining acceptance criteria are named as the most important success factors.

SwissQ Requirements Trends & Benchmarks 2012 5KEY MESSAGES

Project Type70 % of all projects are new developments or updates of an existing software.

Project SuccessJust over a third of all projects are finished with the expected functionality, within the expected timeframe and within budget.

SwissQ Requirements Trends & Benchmarks 2012 6PROJECTS

of the respondents describe the starting situation of their projects as satisfying or insufficient related to

>50 % Estimation Planning Definition of requirements Realistic expectations

12 %

39 %

31 %

10 %

8 % New development

Update of existing software

Migration

Implementation of standard software

Operation, support, maintainance, re-design

Project Size (in Swiss Francs)

up to 1 Mio0 %

20 %

40 %

more than 20 Mio

up to 20 Mio

51 %

39.2 %

10.8 %Project

stoppedProject

extended / rescheduled

Proj. finished with budget and / or time

overruns

Proj. finished with major functional changes

Project finished in

time, budget and

functionality

0 %

10 %

20 %

30 %

40 %

35.1 %

17.5 %18.1 %

25.1 %

4.1 %

Classic Mistakes in RequirementsLinguistic mistakes are still the most commonly named problem in requirements. It is very surprising that despite agile methods on the rise the missing business value remains a problem in (too) many cases.

Acceptance Criteria for Requirements

Reasons for Insufficient Requirements

SwissQ Requirements Trends & Benchmarks 2012 7QUALITY

Missing business value is a problem in more than

Requirements are checked for functional accuracy, feasibility and completeness.

of all projects.

In over

50 %

80 %of all respondents rarely or never check requirements for their need.

Over

of the projects, linguistic mistakes in requirements are a problem.

In

75 %

36 %

0 % 20 % 60 %40 % 80 % 100 %

Systematic mistakes: missing business value/

benefit for the project

Logical mistakes: inconsistency,

redundancy

Content mistakes: wrong facts,

incompleteness

Language mistakes: incomprehensibility,

ambiguousness, unquantifiability

25.5 %57.5 %

58.5 % 26.4 %

49.1 % 38.9 %

48.1 %46.2 %

17.0 %

15.1 %

12.0 %

5.8 %

0 % 20 % 60 %40 % 80 % 100 %

Misunderstandings in communication

Growing or changing requirements of whole system

Formulations too abstract (need more details, be more clear)

New insights (pilot operation, prototype, etc.)

Changes in boundary conditions (prioritization, etc.)

Feasibility wrongly assessed

Changes in stakeholder structure

65.1 %

56.5 %

22.6 %

23.1 %

29.2 %50.9 %

42.3 %49.0 %

45.4 %43.5 %

12.3 %

20.4 %

19.8 %

8.7 %

11.1 %

70.5 %26.7 %

73.6 %23.6 %

always often rarely/never

always often rarely/never

The Most Important Sources for REAs expected, customers and users are the most important source for requirements.

of the respondents use less than 15 % of the total project effort for requirements engineering.

50 %

SwissQ Requirements Trends & Benchmarks 2012 8

RE Effort in Proportion to Total Project EffortThere is no clear tendency when measuring the RE effort compared to the total project effort. Depending on the project, a lot or very little is being invested in RE.

Effort for Stakeholder Analysis2/3 of all respondents invest less than a day in the stakeholder analysis.

EFFORT

0 %5 - 10 %

15 - 20 %

10 - 15 %

20 - 30 %

30 - 50 %

above< 5 %

10 %

5 %

25 %

15 %

20 %

17.6 %

19.6 %

15.7 %

23.5 %

14.7 %

6.9 %

2.0 %

Sponsors/ Customers and Users

51 %Existing Product/

Software

21 %Regulations &

Legal Requirements

14 %

Designers & Developers

8 %

Others 6 %

6.3 %

37.3 %

30.9 %

25.5 %

No effort because it‘s a given

1-5 man-days

Less than 1 man-day

More than 5 man-days

RE effort proportional to total project effort

SwissQ Requirements Trends & Benchmarks 2012 9

SatisfactionThe process of eliciting and analysing requirements is barely satisfactory but the biggest problems seem to lie in managing them.

Measures for Quality ImprovementsWell trained employees and the establishment of standardized processes are the most important measures to improve the RE quality.

Maturity Level of RE in ProjectsOnlyl 1/4 of the respondents rate their RE as good or excellent.

Most Important Success FactorsThe modelling of requirements and the compiling of acceptance criteria are the most important success factors in RE.

MATURITY LEVEL AND SUCCESS FACTORS

22.7 % 8.2 %43.6 %25.5 %

Good/excellent Medium Weak Very weak

Planned Done Not plannedSatisfying Medium Unsatisfying

Compilation of acceptance criteria

Modeling of requirements

Clean stakeholder analysis Use of defined

RE processes

Structured reviews

Internal/further education and training

Establishment of standard RE processes

Establishment of internal templates/standards

Establishment of standard tools

Specific recruiting of RE/BA

Defined specialist career for RE/BA

Systematic IREB training

Employment of external specialists

Systematic IIBA training

0 % 20 % 60 %40 % 80 % 100 %

20.9 %50.9 %28.2 %

21.2 %36.5 %42.3 %

25.2 %38.3 %36.4 %

32.1 %34.9 %33.0 % %

35.2 %48.6 %16.2 %

58.4 %29.7 %11.9 %

69.2 %25.0 %

85.9 %7.1 %

49.5 %24.3 %26.2 %

Manage

Document

Verify

Elicit

Analyse

0 % 20 % 60 %40 % 80 % 100 %

18.2 %46.4 %35.5 %

20.2 %48.6 %31.2 %

27.5 %50.5 %22.0 %

45.9 %36.7 %17.4 %

31.8 %38.2 %30.0 %

TrainingsThe IREB® CPRE Foundation Level seems to be part of the standard repertoire of RE/BAs. The future focus lies with the IREB® CPRE Advanced Levels and the Business Analysis, as well as with Agile RE.

SwissQ Requirements Trends & Benchmarks 2012 10

Who Executes RE?The role of the Requirements Engineer is recognized and will be assigned with the appropriate tasks regardless of the company size.

Prestige of Requirements EngineeringAt least 2/3 of the respondents recognize the value of Requirements Engineering. But those 17% who believe that RE is a necessary evil or even completely superfluous show the need for development in this area.

ORGANIZATION AND TRAINING

I already have it It is planned No issueIn the not too distant future

IREB® CPRE (Foundation Level)

Agile Requirements Engineering

Project Management (IPMA, PMI, ...)

Certified Product Owner

Certified Scrum Master

Certified IT Process and Quality Manager

IREB® CPRE (Advanced Level Elicitation & Consolidation)

IREB® CPRE (Advanced Level Requirements Modeling)

IIBA CBAP (Certified Business Analysis Professional)

0 % 20 % 60 %40 % 80 % 100 %

18 % 17 %63 %

43 % 28 %19 %11 %

43 % 29 %27 %2 %

23 % 44 %12 %21 %

25 % 48 %6 %21 %

20 % 53 %8 %18 %

17 % 65 %6 %13 %

31 % 50 %17 %

42 % 37 %21 %

RequirementsEngineer

40 %Product

Manager /Product Owner24 %

ProjectLeader20 %

DeveloperTester12 % None

4 %

Strategic for company success

Important factor for reliable software

Low priority

Necessary evil

Unnecessary

2.7 %

8.2 %14.5 %

20.9 %

53.7 %

of the respondents already gained experiences with agile methods.

of the respondents have less than two years experience with agile projects.

3/4 2/3

SwissQ Requirements Trends & Benchmarks 2012 11

Trends in Agile Requirements EngineeringThe high rate of changes in the agile field poses big challenges to many an experienced requirements engineer. It does not go far enough to propagate the product owner as a solution, that would be simply hiding the old under the cloak of the new. Agile requirements engineering has to take into account the values and methods of the agile context. Approaches like the following belong to this:

Extreme Prioritization (according to business value)

Continuous planning

Backlog Management (Who‘s responsible? When is it being filled? Synchronisation with strategic projects, ...)

TDD and ATDD (Acceptance Test Driven Development)

Strong use of iterative RE (quick feedback cycles and adjustments)

More face-to-face communication

Level and sustainability of requirements documentation

Correct cutting of requirements (business value versus implementation in one sprint)

etc.

Nothing has changed with the fact though, that the client wants exactly what he imagined at the end of the project. For a “classic“ requirements engineer, these are known problems. It is now time to adjust the classic methods to the agile context in order for “good practices“ not to be lost and still be able to use the method with the light agile approach. SwissQ would be pleased to share its experiences from various agile projects with you.

Use of Agile TechniquesIterative planning, daily standups and the management of backlogs are widely used techniques in the agile field.

Project leader Unit manager

“We often develop a feature and then can‘t find a user / stakeholder for it!“

“The success of a SCRUM project depends on the personality of the product owner.“

AGILE REQUIREMENTS ENGINEERING

Iterative planning

Daily Standup

Backlog Management

Taskboard

Retrospectives

Burndown Chart

Definition of Done

Velocity Chart

On-Site Customer

Co-Location

Test Driven Dev. (TDD)

Kanban

Acceptance Test Driven Dev. (ATDD)

0 % 20 % 60 %40 % 80 % 100 %

20.3 %

15.9 %

11.1 %

89.6 %

57.8 %

38.1 %

34.8 %

26.6 %

82.1 %

80.6 %

75.8 %

67.2 %

72.7 %

Used

Planned

Not anymore

No issue

Where do Investments Occur?The training of employees is still a main priority. Closer collaboration between business and IT is the next big investment topic.

SwissQ Requirements Trends & Benchmarks 2012 12

ChallengesThe traceability (relationship of RE artefacts to preceding and following artefacts) seems to be the biggest challenge.

CHALLENGES

Investmentsincrease

Investments remain constant

Investmentsdecrease

Further training for employees

Closer collaboration between Business and IT

Standardization of internal RE processes

Elaboration / Definition of role of RE

Development of templates and guidelines

Employment of new RE employees

Establishment of specific RE Tools

Establishment of own RE sections / departments

Outsourcing of RE activities

0 % 20 % 60 %40 % 80 % 100 %

33.0 %

33.0 %

25.7 %

24.3 %

22.4 %

22.1 %

21.9 %

17.9 %

11.8 %

13.2 %

14.2 %

13.8 %

15.9 %

16.8 %

23.1 %

14.3 %

19.8 %

40.2 %

53.8 %

52.8 %

60.6 %

59.8 %

60.7 %

54.8 %

63.8 %

62.3 %

48.0 %

Traceability

55 %41 %

Elicitation ofrequirements in distributed

teams

41 %

Non-functional requirements

35 %

Management of many

requirements(>500)

31 %

Natural language Requirements vs. Use Cases

30 %

RE in agile projects

68 %of the respondents are using Microsoft Office as a requirements tool in agile RE.

SwissQ Requirements Trends & Benchmarks 2012 13

Tools in the Agile ContextThe situation is similar when it comes to tools in the agile context. Microsoft Office dominates with 68 %. Jira is in second place with 30 %, followed closely by HP QC/ALM and Open Source Tools.

Tools in PlaceMicrosoft tools are still dominating in the field of Requirements Engineering as more than 80 % of all respondents mentioned Microsoft Office as the most important RE tool. It is followed by far by a tool formerly dedicated to test management – HP QC/ALM – which developed into an Application Lifecycle Suite where you can also create, document, and manage requirements.

TOOLS

Microsoft OfficeAtlassian Jira

HP QC/ALM

Open Source

Microsoft TFS

PolarionInflectra Spira

Rally SoftwareOwn developments

Version One

Microsoft Office Suite (doc, xls, ppt)

Microsoft Visio

HP QC / ALM

Open Source

IBM Rational Requisite Pro

IBM Rational DOORS

Others

MS Team Foundation Server

Sparx Enterprise Architect

Own developments

Polarion

MKS Integrity

Serena Dimension RM

Wiki

microTOOL In-Step

Atlassian JIRA

0 % 20 % 60 %40 % 80 %

85 %

47 %

21 %

14 %

13 %

12 %

12 %

10 %

6 %

4 %

3 %

3 %

2 %

2 %

2 %

2 %

IT EmployeesA bit more than half of the respondents work in companies with more than 500 IT employees.

SwissQ Requirements Trends & Benchmarks 2012 14FRAME OF SURVEY

ResponsibilitiesMore than 50 % of the respondents describe their job with more than one role.

0 %

IT

Finance, Insurance

Manufacturing

Public and semi-public companies

Traffic and Transportation

Telecommunication

MedTech

Others

10 % 20 % 30 % 40 %

36.1 %

28.4 %

7.4 %

7.4 %

5.6 %

4.0 %

3.7 %

7.4 %

0 %

2001– ...

501 – 2000

251 – 500

51 – 250

11 – 50

1 – 10

5 % 10 % 15 % 20 % 25 % 30 % 35 %

33.0 %

13.6 %

17.6 %

15.4 %

14.2 %

6.2 %

Industrial SectorMore than 60 % of the respondents work either in the IT or in the financial sector. Compared to the last years their proportion has decreased, demonstrating that the subject has arrived in other industries too.

of the respondents mainly work in projects.

60 %of the respondents are line managers.

33 %

30 %

20 %

10 %

0 %

Test

Manag

er

Depa

rtmen

t / D

ivisio

n Man

ager

Requ

irem

ents

Engi

neer /

BA

Test

Engi

neer

Proj

ect M

anag

er

Teste

r

Requ

irem

ents

Manag

er

Softw

are

Engi

neer

Along with the first edition of the SwissQ Requirements Trends & Benchmarks Report, SwissQ publishes the already fourth edition of the SwissQ Testing Trends & Benchmarks Report and as well the first edition of the SwissQ Agile Trends & Benchmarks Report, in 2012. Do you want to know more? You can download the detailed reports with further analyses from www.SwissQ.it.

SwissQ Requirements Trends & Benchmarks 2012 15TRENDS & BENCHMARKS REPORTS 2012 FOR TESTING AND AGILE

Main Reasons for the Failure of Agile Projects

0 %

Lacking experience with agile methods

Corporate culture is not compatible with agile values

External pressure to follow a traditional approach

Lacking support of line management

Lacking interconnections between organizational units

Lacking / insufficient training / coaching

Lacking team motivation

Others

10 % 20 % 30 % 40 % 50 % 60 %

52 %

45 %

41 %

38 %

36 %

35 %

22 %

12 %

Trends & BenchmarksAgile 2012

Trends & BenchmarksTesting 2012

Costs increased

up to 10 % up to 20 % up to 50 % up to 80 % No statement possible

Cost Savings by Test Automation

10.2 %

33.3 %

2.8 %

22.6 %23.7 %

7.3 %

ABOUT US

SwissQ supports its clients in the development and implementation of IT-solutions and assures that the end users get the functionality they really need. This is achieved by unambiguously determining requirements and risk-based testing the implementation.

Our vision is to improve the added value of IT through requirements management and software testing. Along with providing high-quality services, we pursue this vision by establishing independent platforms, like the Swiss Testing Day and the Swiss Requirements Day, which facilitate the exchange of know-how and experiences.

In addition to that we help bright minds to expand their knowledge in our trainings.

© by SwissQ Consulting AG | Stadthaus-Quai 15 | Switzerland-8001 Zürichwww.SwissQ.it | info@SwissQ.it | Phone +41 43 288 88 40 | Fax +41 43 288 88 39

Twitter: @SwissQ | Facebook: swissqconsulting