software engineering

Risk Analysis and Management

Look Out!!

If you don't actively attack the risks, they will actively attack you.

-Tom Gilb

Risk Definitions

• Risk is the potential for realization of unwanted negative consequences of an event.

[Rowe, William D. An Anatomy of Risk 1988]

• Risk is the measure of the probability and severity of adverse effects.

[Lowrance, William W. Of Acceptable Risk 1976]

• Risk is the possibility of suffering loss, injury, disadvantage, or destruction.

[Webster's Third New International Dictionary 1981]

Risk Characteristics

• uncertainty - an risk may or may not happen

• loss - an risk has unwanted consequences or losses

What is Risk Management?

Making informed decisions by consciously assessing what can go wrong and the severity of its impact.

A comprehensive risk management strategy, as part of total quality management approach, aims at anticipating and eliminating all the causes of risk

Why Do Risk Management

– Eliminate surprises

– Anticipate issues - prevent problems

– Begin programs on the "right" foot and stay on track

– Reach business goals

MIS Risks

80

65

60

55

50

Creeping user requirements

Excessive schedule pressure

Low quality

Cost overruns

Inadequate conf. Control

System Software Risks

70

65

60

50

35

Long schedules

Inadequate cost estimation

Excessive paperwork

Error-prone modules

Cancelled projects

Commercial Software Risks

70

55

50

45

30

Inadequate user documentation

Low user satisfaction

Excessive time to market

Harmful competitive actions

Litigation expense

Defense Software Risks

90

85

75

70

45

Excessive paperwork

Low productivity

Long schedules

Creeping user requirements

Unusable software

Contract Software Risks

60

50

45

30

20

High maintenance costs

Contractor Vs client friction

Creeping user requirements

Legal ownershipUnanticipated acceptance criteria

End-user Software Risks

80

65

60

50

20

Non-transferable applications

Hidden errors

Unmaintainable software

Redundant applications

Legal ownership

Risk Analysis

– Risk Identification

– Risk Projection

– Risk Assessment

– Risk Management

Risk Identification

– Project Risks

– Technical Risks

– Business Risks

Project Risks

– Potential budgetary, schedule, personnel, resources, customer and requirements problems and their impact.

– Project complexity, size and structural uncertainty are determining factors.

Technical Risks

– Potential design, implementation, interfacing, verification and maintenance problems.

– Specification ambiguity, technical uncertainty, technical obsolescence and ``leading edge'' technology.

Business Risks

Market Risk : building a product that no one really wants.

Product Risks: building a product that no longer fits into the overall product strategy for the company.building a product that the sales force doesn't understand how to sell.

Management Risk: losing the support of senior management due to a change in focus or people.

Budget Risk: losing budgetary or personnel commitment.

Risk-item checklist

• Product size• Business impact• Customer characteristics• Process definition• Development environment• Technology to be built• Staff size and experience

Product size risks

• Estimated size of the product in LOC or FP?• Estimated size of product in number of programs, files,

and transactions?• Percentage of deviation in size of product from average

for previous products?• Size of database created or used by the product?• Number of users of the product?• Number of projected changes to the requirements for the

product?• Number of changes before delivery? After delivery?• Amount of reused software?

Business impact risks

• Effect of this product on company revenue?

• Visibility of this product by senior management?

• Reasonableness of delivery deadline?

• Number of customers who will use this product and the consistency of their needs relative to the product?

• Number of other products or systems with which this product must be interoperable?

• Sophistication of end users?

Business impact risks

• Amount and quality of product documentation that must be produced and delivered to the customer?

• Governmental regulatory constraints on the construction of the product?

• Costs associated with late delivery?

• Costs associated with a defective product?

Customer characteristics risks• Have you worked with the customer in the past?• Does the customer have a solid idea of what is required?

Will the customer agree to spend time in FAST meetings to identify project scope?

• Is the customer willing to establish rapid communication links with the developer?

• Is the customer willing to participate in reviews?• Is the customer technically sophisticated in the product

area?• Is the customer willing to let your people do their job that

is, will the customer resist looking over your shoulder during detailed technical work?

• Does the customer understand the SE process?

Process definition risks

• Does your senior management support a written policy statement that defines a process for software development?

• Are specific documentation formats defined?

• Are formal technical reviews of the SRS, design, test procedures and test cases and code conducted regularly?

• Is SCM used to maintain consistency among system / software requirements, design, code, and test cases?

Process definition risks

• Is more than 90 % of your code written in a high-order language?

• Are software tools used to support planning and tracking activities?

• Are CASE tools used to support the software analysis and design process, prototyping, test support, documentation?

• Are quality and productivity metrics collected for all software projects?

Technology risks

• Is the technology to be built new to your organization?• Do the customer’s requirements demand the creation of

new algorithms or input or output technology?• Does the software interface with new or unproven

hardware?• Does the software to be built interface with vendor-

supplied software products that are unproven?• Does the software to be built interface with a database

system whose function and performance have not been proved in this application area?

• Is a specialized user interface demanded by product requirements?

Technology risks• Do requirements for the product demand the creation of

program components that are unlike any previously developed by your organization? What percentage of components are new?

• Do requirements demand the use of new analysis, design, or testing methods?

• Do requirements demand the use of unconventional software development methods, such as formal methods, AI (artificial intelligence)-based approaches, or artificial neural networks?

• Do requirements put excessive performance constraints on the product?

• Is the customer uncertain that the functionality requested is “doable”?

Checklist for staffing risk

• Are the best people available? • Are enough (too many) people available? • Do the people have the right combination of

skills? • Have staff members received the necessary

training? • Is the staff committed for the entire project

duration? • Will some staff members only be part time? • Will staff turnover be low enough for continuity?

Risk Projection (Estimation)

Attempts to rate each risk in two ways:

– Likelihood that the risk is real.

– Consequences of the problems associated with the risk, should it occur.

Risk projection activities

• Establishing a scale that reflects the perceived likelihood of a risk: Boolean, qualitative, quantitative

• Delineating the consequences of a risk.

• Establishing the impact of the risk on the project and the product.

• Noting the overall accuracy of the risk projection.

Probability Quantification

impossible toimprobable

probable frequent value

Prob. value (0, 0.4) (0.4, 0.7) (0.7, 1)Performance ------------- -------------Cost ------------- -------------Schedule ------------- -------------support ------------- -------------

Risk Drivers

Performance Cost Schedule support

Requirements Requirements Resources Design

Constraints Personnel Need dates Responsibilities

Technology Reusable SW Technology Tools, env

Dev. approach Tools, env Requirements Supportability

Impact Assessment

Performance Cost Schedule support

Catastrophic

Critical

Marginal

negligible

Risk Assessment

frequent probable improbable impossible

Prob. value (0.7, 1) (0.4, 0.7) (0, 0.4) 0

catastrophic

critical

marginal

negligible

HIGH

MODERATE

LOW

NONE

Factors affecting Impact

Risks are weighted by perceived impact and then prioritized. Three factors affect impact:

• The nature of the risk indicates the problems that are likely if it occurs.

• The scope of a risk combines its severity with its overall distribution.

• The timing of a risk determines when and for how long the impact will be felt.

The importance of risk impact and probability is linked to their effect on management concerns.

Risk Assessment

Risks can be represented as a set of triplets of the form: [r,l,x] where

– r is risk

– l is the likelihood (probability) of the risk

– x is the impact of the risk.

Risk Assessment

During risk assessment the following actions occur:

– An examination of the accuracy of the estimates made during risk projection.

– A prioritization of the risks that have been uncovered.

– A preliminary examination of the ways to control and/or avert likely risks.

Risk Referent Level

• At a certain level of risk, or a combination of risks, a project will have to be terminated.

• A risk referent level has a single point, called the referent or break point, at which time the decision to proceed or terminate are equally acceptable.

• Cost, schedule, support and performance represent typical risk referent levels.

Risk Referent Level

Projected cost overrun

Pro

ject

ed s

ched

ule

over

run

Referent point

High risk area

Risk Assessment Checklist

• Define the risk referent levels for the project. Referents are stated as a probability of failure or the probability of success level for each individual risk or the system as a whole.

• A value should be agreed upon where it is decided that a project should not continue.

The system risk referent can be:

• an aggregate of individual risks, or one or more prioritized high impact risks

• Attempt to develop a relationship between each [r,l,x] and each of the referent levels.

• Predict the set of referent points that define a region of termination, bounded by a curve or areas of uncertainty.

• Try to predict how compound combinations of risks will affect a referent level.

Risk Evaluation Outcomes

Comparing the evaluated risk against its risk referent has three possible outcomes:

1.Acceptable : the evaluated risk is less than the referent.

2.Impossible : the evaluated risk is much greater than the referent.

3.Infeasible : the evaluated risk is greater than, but almost equal to, the referent.

Seven-stage Hierarchy

1. Crisis management

2. Fix on failure

3. Mitigation

4. Prevention

5. Elimination of root causes

6.Anticipation

7. Management of change7. Management of change

SEI Risk Management Paradigm

RMMP Outline

I. Introduction

II. Risk analysis

III. Risk management

IV. Appendixes

RMMP - Introduction

A. Scope and purpose of document

B. Overview

1. Objectives, 2. Risk aversion priorities

C. Organization

1. Management, 2. Responsibilities, 3. Job descriptions

D. Aversion program description1. Schedule, 2. Major milestones and reviews, 3. Budget

RMMP - Risk analysis

A. Identification1. Survey or risks, a. Sources of risk, 2. Risk taxonomy

B. Risk estimation1. Estimate probability of risk, 2. Estimate consequence of risk, 3. Estimation criteria, 4. Possible sources of estimation error

C. Evaluation1. Evaluation methods to be used, 2. Method assumptions and limitations, 3. Risk referents, 4. Results

RMMP - Risk management

A. RecommendationsB. Risk aversion optionsC. Risk aversion recommendationsD. Risk monitoring procedures

IV. Appendixes

A. Risk estimate of the situationB. Risk abatement plan