baxter - usingahp

8/7/2019 Baxter - UsingAHP

1/32

Peter Baxter, DISTRIBUTIVE MANAGEMENT

Hampton Roads INCOSE Decision Analysis Conference - Nov 2009

USING ME A SURE ME NT A ND THE S AATY

ME THOD TO CH O O S E T H E BEST

DECISION ALTERNATIVE


2/32

OBJECTIVES

Learn:

How to construct and weight evaluationcriteria.

How to make pair-wise comparisons ofalternatives.

How to expand the example for morecomplex, nested types of criteria.

How a measurement process can supportthe Saaty method.

D I S T R I B U T I V E

M A N A G E M E N T

2


3/32

WHAT IS THE AHP/SAATYMETHOD?


4/32

SIMPLE DEFINITION

Compute the matrix values for a and voila!

The Analytic Hierarchy Process (AHP)


M A N A G E M E N T

4


5/32

COMPARE THE SIZES

OF THESEC

IRCLES

A B C D E

A B C D E

A 1 7 9 5 3

B 1/7 1 3 1/5 1/7

C 1/9 1/3 1 1/7 1/9

D 1/5 5 7 1 1/3

E 1/3 7 9 3 1D I S T R I B U T I V E

M A N A G E M E N T

5


6/32

SCALE OF COMPARISON

Dont need an absolute scale

(if you already know the size of all but one)

Relative scale approximates difference


M A N A G E M E N T

6


7/32

SIMPLE EXPERIMENT

One test is worth 1000 expert opinionsD I S T R I B U T I V E

M A N A G E M E N T

7


8/32

WEBERS EXPERIMENT

In 1846 Weber found, for example, that people while holding in

their hand different weights, could distinguish between a

weight of 20 g and a weight of 21 g, but could not if the

second weight is only 20.5 g.On the other hand, while they

could not distinguish between 40 g and 41 g, they could

between 40 g and 42 g, and so on at higher levels. We need

to increase a stimulus s by a minimum amount s to reach a

pointwhere our senses can first discriminate between s and

s+s. s is called the just noticeable difference (jnd). The

ratio r = s/s does not depend on s.

Webers law states that change in sensation is noticed when the stimulus

is increased by a constant percentage of the stimulus itself. This law

holds in ranges where s is small when compared with s, and hence

in practice it fails to hold when s is either too small or too large.D I S T R I B U T I V E

M A N A G E M E N T

8


9/32

WHY IT WORKS

People are inconsistent at providing an absolutescale to evaluate objects.

How much does Rock A weigh?

How much does Rock B weigh?

There are better at comparing pairs of objects.

Does Rock A weigh more than Rock B?


M A N A G E M E N T

9


10/32

APPLYING THE TECHNIQUE

1. Select criteria(s) to evaluate

2. Define comparison scale

3. Perform pair-wise comparison

4. Check consistency

5. Calculate values


M A N A G E M E N T

10


11/32

APPLYING THE METHOD


12/32

EXAMPLE #1

SOFTWARE ESTIMATION

A controlled test where existing software sizes are known.

Ask 30 grad students to estimate the SLOC of common data

structures like stack, queue, list.

Estimate SLOC three ways:

1. Guess a number.

2. Compare to one reference structure using numeric

scale.

3. Compare to one reference structure using relative scale.

Plot results

From Establishing Software Size Using Pair-wise Comparison Method

by Eduardo Miranda


M A N A G E M E N T

12


13/32

EXAMPLE #1

SOFTWARE ESTIMATION


M A N A G E M E N T

13


14/32

EXAMPLE #1

OBSERVATIONS

Mirandas Observations

The high variability of the finger in the wind approach,

which is almost two to three times bigger than the

corresponding paired comparisons method.

The high correlation, r = .979, existing between the

relative sizes of modules independent of the estimation

method employed. This seems to corroborate the

premise that the human mind is better at establishingdifferences than at guessing absolute values.


M A N A G E M E N T

14


15/32

EXAMPLE #2

SYSTEM REQUIREMENTS

From an SEI study Requirements Prioritization Case Study

Using AHP by Nancy Mead.

Given nine security requirements, decide which one(s) have

greatest cost-benefit.

Benefits

By using AHP, the requirements engineer can

also confirm the consistency of the result. AHP can

prevent subjective judgment errors and increase the

likelihood that the results are reliable.


M A N A G E M E N T

15


16/32

EXAMPLE #2

APPROACH

Approach

1. Review requirements for completeness.

2. Apply pair-wise comparison for value.

3. Apply pair-wise comparison for implementationcost.

4. Calculate AHP matrix and diagram for value & cost.

5. Use resulting diagram for analyzing requirements.


M A N A G E M E N T

16


17/32

EXAMPLE #2

COMPARISON SCALE


M A N A G E M E N T

17


18/32

EXAMPLE #2

COMPARISON

Compare one attribute of each requirement to another

requirement using a relative scale.

#1 is same

as #1

#1 is greater

than #2

#2 is much

less than #8


M A N A G E M E N T

18


19/32

ABOUT CONSISTENCY

If: A > B > C

Then: C > A is wrong

AHP contains a technique to calculate the extent ofpair-wise consistency, which can then be

compared to a consistency tolerance.

AHP can also indicate which pair-wise comparison(like the one above) is inconsistent.


M A N A G E M E N T

19


20/32

EXAMPLE #2

COMPARISON SUMMARY


M A N A G E M E N T

20


21/32

EXAMPLE #2

FINDINGS

Client feedback

It may be beneficial to see the consistency matrix.

Understand weight of cost and value.

Difficult to understand the motivation of each

reviewer.


M A N A G E M E N T

21


22/32

OTHER EXAMPLES

From Saaty:

Evaluate the best city in China for Disney

to build a new theme park.

Determine optimum foreign relationspolicy for dealing with Iran.

Estimate market share of super retail

stores.

Selecting a school.


M A N A G E M E N T

22


23/32

MEASUREMENT AND AHP


24/32

MEASUREMENT AND AHP

Goal is for Measurement to support AHP estimation

Estimate

Plan

Monitor& Control

Capitalize

Measurement Process

Use AHP


M A N A G E M E N T

24


25/32

TYPICAL MEASUREMENT

PROCESS

Evaluate

Establish

Capability

Technical

and Management

ProcessesINFORMATION

NEEDS

ANALYSIS RESULTS

ANALYSIS

RESULTSAND

PERFORMANCE

MEASURESIMPROVEMENT

ACTIONS

Measurement Process

USER FEEDBACK

Plan Perform

Core Measurement Process

Experience

Base

MEAS-

UREMENT

PLAN

25


26/32

WHAT TO MEASURE?

Measurement Requirements = information needs

Based on your business needs.(not a pre-defined list)

Information needs are:

Input to the measurement process.

Provided by the management and technical process thatneed information to perform their jobs.

Become the requirements for measurement process.

Refined into measures and then resulting informationproducts are provided to the users.


M A N A G E M E N T

26


27/32

COMMON PROJECT

MEASURES

Cost

Staffing , Staff Hours

Functional / Requirements Size

SLOC

Defects

Estimation Project/Program Store

Factors estimate

using HP


M A N A G E M E N T

27

Usually there is opportunity for measurement to support AHP since

there is overlap between what is estimated and what is measured


28/32

MEASUREMENT GUIDE AHP

Measurement contains project attributes:

Lifecycle model

Principle architecture

Application domain

AHP estimation of projects with similar attributes.

Reference sizes should not differ by more than an

order of magnitude.


M A N A G E M E N T

28


29/32

AHPIN USE

Estimate total staff hours in person years using data from

five completed projects sharing similar attributes.


M A N A G E M E N T

29


30/32

RESOURCES

Requirements Prioritization Case Study Using AHP

by Nancy Mead Software Engineering Institute

Establishing Software Size Using the Paired

Comparisons Method by Eduardo Miranda

Relative Measurement and Its Generalization in

Decision Making Why Pairwise Comparisons are

Central in Mathematics for the Measurement of

Intangible Factors The Analytic

Hierarchy/Network Process by Thomas Saaty


M A N A G E M E N T

30


31/32

QUESTIONS


M A N A G E M E N T

31


32/32

Peter Baxter

Distributive Management

www.distributive.com

[email protected]


M A N A G E M E N T

32

baxter - usingahp

Documents