risk modeling and analysis (mitigating the planning fallacy)

Mitigating the Planning Fallacy

Gui Ponce de Leon, PhD, PE, PMP, LEED AP

GPM Boot CampNewark, NJMarch 8th, 2013

(Risked Schedules─The New Normal)

©2012-2013 Permission is granted to PMA Technologies 1

2

There are lies, there are damned lies, and then there are deterministic schedules

Attributed to Dr. Vivek Puri, PMA’s resident simulation guru

©2012-2013 Permission is granted to PMA Technologies

3©2012-2013 Permission is granted to PMA Technologies

Mitigating the Planning Fallacy

CPM Risk Modeling & Analysis

GPM® Risk Modeling & Analysis

NetRiskTM Synopsis

Summary & Take-Aways

PRESENTATION OUTLINE

Just What Is the Planning Fallacy?


Kahneman1 and his longtime colleague, Tversky, coined the term to describe plans that

Are unrealistically close to best-case scenarios

Could be improved by consulting the statistics of similar cases

Planning Fallacy aka Optimism Bias


The problem of optimism bias arises when various factors combine to produce a systematic underreporting of the level of project uncertainty

Bent Flyvbjerg2, the renowned Danish planning expert, notes

Planning Fallacy aka Optimism Bias (cont’d)


The failure to reflect the probabilistic nature of project planning, implementation and operation is a central cause of the poor track record for megaproject performance

Bent Flyvbjerg futher notes

Scheduling Strategies for Mitigating the Planning Fallacy


The outside view does not try to forecast specific uncertain events that may affect the activities

Rely on an outside view as advocated by Kahneman 1

Scheduling Strategies for Mitigating the Planning Fallacy


The schedule is dealt with as inherently stochastic in nature rather than being analyzed merely for risk (i.e., what-if exercise)

Work with risked schedules as the new normal2

Taking an Outside View Relative to Schedules


At a minimum, the database captures ‘normal’ durations

The schedule is built using a database of historical activity durations by project type/context

Ideally, distributional information (mean, mode, low/high) is included

Physical work durations factor production rates, e.g., steel tons/day, concrete CY/day, large bore pipe LF/day, etc.

Working with Risked Schedules

Is the New Normal


THE CONCEPT OFRISKED SCHEDULES


A base‐case schedule portraying how the project would evolve with activities at their normal durations is the initial focus

The baseline schedule selected reserves schedule margin sufficient to support the targeted probability(ies) of completion

Following risk modeling, risk analysis trials are conducted to investigate alternate probabilistic and baseline scenarios

STEP 1 STEP 2 STEP 3

Risked schedules are generated in the following sequence:


The baseline schedule is risk assessed periodically and when revised to reflect scope of remaining work and current risks

Going from deterministic to probabilistic planning/scheduling and back is a seamless exercise throughout the project life cycle

RISKED SCHEDULES


1955 1960 1965 1970 1975 1980 1985 1990 1995 2000

1958 Development and implementation of PERT by the US Navy Special Projects Office

1962 Robert McNamara endorses use of PERT/COST (forerunner to earned value) throughout the US DOD3

1966 Pritsker develops GERT for NASA as a method to analyze stochastic activity networks5

1986-87 Risk management becomes a separate knowledge area in the PMBOK in the 1986-87 update6

1990s Simulation morphs into schedule risk analysis

1995 Primavera releases Monte Carlo version 3.0 offering “enhanced risk analysis software for project management”

Schedule Risk in the 20th Century

1963 First application of Monte Carlo simulation to network-based schedules by Van Slyke4


1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

2000 Risk analysis is added to Pertmaster and Pertmaster supports full integration with Primavera

2004 The Third Edition of the PMBOK adds the ‘risk register’ as a primary output of the Identify Risks Process

2011 Schedule risk analysis is intrinsic to scheduling excellence in the Planning &

Scheduling Excellence Guide7

2012 Schedule risk analysis is codified as one of the nine scheduling best practices in

the GAO Schedule Assessment Guide8

2013 GPM® Risk is introduced at the NetPoint® User Conference in New Orleans

Schedule Risk in the 2000s

2008 Oracle acquires Primavera, and the Pertmaster software is renamed OPRA

2012 AACE International releases RP 64 on CPM Schedule Risk Modeling and Analysis9

Emerging Consensus on CPM Risk Analysis


A schedule risk analysis (SRA) is conducted to determine

the likelihood of completion dates

schedule contingency needed for an acceptable level of certainty

Emerging Consensus on CPM Risk Analysis (cont’d)


The baseline schedule includes contingency aka schedule margin to account for the occurrence of risks

Schedule margin supports the targeted likelihood of meeting completion dates

Emerging Consensus on CPM Risk Analysis (cont’d)


An SRA is performed on the schedule periodically as the schedule is updated



Minimal use of constraints on activities

PDM logic ties are used in only limited and well-understood circumstances

Modeling accepts ‘existence risks’ and ‘branching risks’



Risk drivers occur with the same probability on impacted activities

In each realization, all activities are on early dates, but for perhaps SNE dates

Cruciality is combined with criticality

Weather risks are modeled


GPM planned dates are favored over SNE constraint dates1Activities and activity nodes are encoded with stochastic rules2Any risk that, if occurring, impacts multiple activities, may occur with a different probability and impact on each activity

3

Risks resulting from common, contemporaneous decisions to start activities on dates later than early dates are modeled

4

Criticality and cruciality are combined to measure importance5

Adverse weather and weather-event risks are modeled6

GPM algorithms are extended for stochastic networks7

INTRODUCING GPM RISKMODELING & ANAYSIS

Planned Dates ILO SNE Constraints


SNE constraints can be replaced in simulation with GPM planned dates

In CPM modeling, the consensus is to limit SNE dates to external dependencies10

Unlike a constraint date, a planned date may shift to an earlier date in a realization if predecessors on logic chains leading to the planned-date activity are sampled at the right mix of lower durations

Because GPM is not fixated on early dates

1

Planned Dates ILO SNE Constraints (cont’d)


1

Pursue trial simulations with alternate, earlier SNE dates, or

Replace the SNE date with a variable-duration activity

In CPM risk, to mitigate SNE constraints, analysts may

Schedule Demonstrative, Base-Case Scenario


Constraint Date

Simulation Trial, Planned Date ILO SNE Date11


Planned Date

Not Risked

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

08/26/2012 09/09/2012 09/23/2012 10/07/2012 10/21/2012

Num

. of Iteratio

ns

Date

SNE Constraint Planned Date

Planned Dates vs. SNE Constraints Results


10/03/2012

10/08/2012

01/09/2012

12/20/20

11

01/09/20

12If the SNE date could be moved up to 01/06/2012, the P80 date improves to 9/27/2012

Stochastic Activity Modeling in GPM Risk


GPM activities are diagrammed with start and finish nodes

2

An ‘or’ node is realized when the sampled predecessor is realized

An ‘any’ node is realized when any merging predecessor is realized

Activity nodes are encoded with stochastic rules:

An ‘if’ node is realized if its predecessor is realized

Stochastic Activities in GPM Risk


2

Delay risks

Branching risks

Stochastic activities occur based on a probability of occurrence and, if occurring, are of uncertain duration and may symbolize:

GPM Prime Risks vs. CPM Risk Drivers


In GPM risk, an occurrence risk that may impact multiple activities may occur with a different probability and impact on each associated activity/group of activities

3

GPM Prime Risks vs. CPM Risk Drivers (cont’d)


The risk driver approach restricts the risk to always occur/not occur in a realization for all of the associated activities/group of activities

This is not always true, to wit: if bad soil is hit when excavating the SW part of a building, the bad soil risk may not occur when excavating the NE part

In CPM, a risk driver is a particular case of a prime risk because, if occurring, occurs with the same probability of occurrence and the samepercentage impact for all associated activities

3

Float Consumption12

Risks in GPM Risk


4

delaying the start of an eligible activity

within its float then existing when the activity is scheduled

Floating: event that occurs randomly and that involves

Float Consumption12

Risks in GPM Risk


4

delaying the start of a pacing-eligible activity

within its float then existing when the activity is scheduled

Pacing: event that occurs randomly and that involves

provided the ratio then-existing float/deterministic float exceeds a threshold

Float Consumption12

Risks in GPM Risk (cont’d)


4Modeling can control how often an eligible activity actually floats or paces during a realization by defining a likelihood factor

Float Consumption Risks


A floating or pacing risk occurs whenever an activity that floated or paced and that falls on the longest path would not otherwise have been critical but for the floating or pacing decision

The floating or pacing decision in effect caused a critical path delay

Floating/pacing decisions rely on predicted vs. actual durations

Float Consumption Risks (cont’d)


Floating/pacing risks cannot be modeled with CPM for two fundamental reasons:

The CPM scheduling algorithm defaults all activities to their earliest possible dates

No activity has (total) float in a CPM schedule during the forward pass calculations, which means that, unlike GPM, float does not exist in CPM when the activity is scheduled

Simulation Trial, Floating & Pacing Risks


Floating

Pacing

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

08/11/2012 08/25/2012 09/08/2012 09/22/2012 10/06/2012 10/20/2012

Num

. of Iteratio

ns

Date

PERT Early Dates Floating & Pacing

Demonstrating the CPM ‘Optimism Bias’


10/08/2012

09/28/2012

CPM Optimism BiasExclusion of floating & pacing events in risk modeling combines to produce a systemic overestimation of the true probability of accomplishing targeted completion dates

CPM Optimism BiasExclusion of floating & pacing events in risk modeling combines to produce a systemic overestimation of the true probability of accomplishing targeted completion dates

46%

09/23/2012

32%

Includes early‐dates and merge bias

Unbiased forecast

Includes early‐dates bias

Activity Criticality in GPM Risk


5

A shorter, certain-duration activity and a longer, uncertain-duration activity may have equal criticality because they both fall on the stochastic longest path

Williams addressed this conundrum in 1992 when he developed his cruciality index13 that correlates sampled activity duration and realized project duration

Criticality index, while measuring the likelihood of activities falling on the stochastic longest path, fails to account for the correlation between activity duration and project duration

If an activity duration is certain, cruciality = zero

Activity Priority─A New Metric


To solve the dilemma between criticality and cruciality, some CPM risk analyzers combine the two by multiplying criticality x cruciality

This formula tends to downplay the criticality index

Activity priority equals criticality index + criticality index x cruciality index

The comparable statistic in GPM risk is ‘activity priority’

Activity Priority─A New Metric (cont’d)


GPM risk treats 66% confidence level

(2:1 odds of the value of cruciality being correct) as a default confidence threshold

Activity priority = criticality indexif cruciality has a significance confidence level below a:

Default Confidence Threshold

Criticality & Priority Indices Demonstrative


SYNOPSIS

NETRISK


NetPoint module that allows users to work with deterministic and probabilistic GPM &

CPM schedules seamlessly

Offers a full gamut of risk management processes, including:


‘Risk Manager’ interface, which acts as a single streamlined window that works dynamically with the canvas rather than obscuring it

Interface for defining a fully-customizable risk breakdown structure

Fully-customizable probability & impact matrix with tolerance thresholds

Risk identification through a risk register

Full range of activity and risk correlations, including floating and pacing

Automated risk removal process for sensitivity tornado analysis

Full gamut of GPM (quantitative) schedule risk analysis

Wide range of simulation data mining that is fully customizable and that interface with MS Excel

Risked Trial Runs─P80 Date Comparisons


33 33

15

30

24

38 38

15

37

31

43 43

33

43

37

0

5

10

15

20

25

30

35

40

45

Project Ready for Commissioning Elevator Install Complete Perm Power Available Start Process Installation

Uncertainty Only

Uncertainty + Planned Date

Uncertainty + Floating/Pacing

On the floating path

Very sensitive to floating

GPM & CPM Risk Software Comparison


GPM Risk as Embodied in NetRisk CPM Risk as Embodied in OPRA

Schedule view relies on time-scaled LDM networks Minimalist schedule view that relies on logic GANTT charts

Planned dates can be used to model SNE constraint dates SNE constraints cannot reflect the network stochastic nature

A risk occurs with unique probability/impact per activity-risk pair A risk occurs with the same probability on all impacted activities

Longest path, sampled path & shortest path logic constructs Longest path & sampled path logic constructs

Floating and pacing risks are modeled as random risks Neither floating nor pacing risks can be modeled

Automated risk removal for risk sensitivity analysis Manual, one-by-one risk removal for risk sensitivity analysis

Multiple simulations within the same file for easy comparisons One simulation per file complicates comparison of results

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

08/11/2012 08/25/2012 09/08/2012 09/22/2012 10/06/2012 10/20/2012

Num

. of Iteratio

ns

Date

PERT Early DatesFloating & Pacing OPRA

NetRisk & OPRA Distribution Functions


10/08/2012

09/28/2012

46%

09/23/2012

32%

Includes early‐dates and merge bias

NetRisk discretizes continuous distributions by dividing the range using proper (mathematical) rounding rules, which explains the slight difference in the distribution curves

Unbiased forecast

Includes early‐dates bias

NetRisk & OPRA Criticality Tornado Diagrams


1%

1%

1%

1%

20%

21%

21%

20%

20%

27%

27%

27%

58%

58%

58%

58%

58%

77%

77%

100%

100%

0%

0%

0%

0%

20%

20%

20%

20%

20%

32%

32%

32%

52%

52%

52%

52%

52%

71%

71%

100%

100%

0% 20% 40% 60% 80% 100%

FDN Permit

Piping/HVAC/FS Rough‐In

Start Exc, FDN

Comp Exc, FDN

Shops, R & A, Delivery

SOG, Pour & Seal Decks

Power/Lighting/Low Voltage

Steel, Joists, Decking

Bid/Award Steel

BOD Process Equip

Equipment Procurement

Gather Equip Quotes

Substation Fab/Delivery

Subtation Installation

Permit Set

Comp. CD Set

Substation Shops

MEP Process Equip

DD Set

SD Set

Install/Connect Process Equipment

NetRiskOPRA

As a check, an OPRA simulation with 1000 iterations was run, which showed Criticality indices within 2% points of those calculated by NetRisk.


TAKE-AWAYS1 Any project or contract schedule that is not risked through its

life cycle does not conform to scheduling best practices

2 Any schedule that does not expressly reserve reasonable schedule margin does not conform to best practices either

3 The CPM optimism bias impacts CPM risk analysis results in that ‘p dates’ are biased early/are optimistic

4 GPM planned dates are better suited to risk modeling than deterministic SNE constraint dates

5 Activity durations should be ranged using benchmarking

6 With schedule margin as critical path float, early completion schedules are the new normal

7 There is a new sheriff in town!

NetRisk Development Path 2013 - 2014


1 Visual Risk

2 High-priority User Requests

3 Automated Risk Removal in Risk Sensitivity Analysis

4 Full Stochastic Network Modeling

6 Full Interoperability with Cost Risk Software

7 Integrated Resource Leveling During Simulation

5 Weather Risks


REFERENCES

1) Kahneman, D. (2011). Thinking, Fast and Slow. New York: Farrar, Straus & Giroux.

2) Flyvbjerg, B. (2004). Procedures for dealing with optimism bias in transport planning and Flyvbjerg, B. (2008). Curbing optimism bias and strategic misrepresentation in planning: reference class forecasting in practice

3) NASA. (1962). PERT/COST Systems Design. DOD and NASA Guide

4) Van Slyke, R. (1963). Monte Carlo methods and the PERT problem.

5) Pritsker, A. (1966). GERT: Graphical evaluation and review technique.

6) Project Management Institute. (1996). Project management body of knowledge (1st ed.)

7) National Defense Industrial Association. (2011). Planning & scheduling excellence guide (PASEG)

8) United States Government Accountability Office. (2012). GAO schedule assessment guide

9) AACE International. (2012). CPM Schedule risk modeling and analysis: special considerations

10) AACE International. (2012). CPM Schedule risk modeling and analysis: special considerations (p. 7)

11) Kennedy, K. & Thrall, R. (1976). PLANET: A simulation approach to PERT (p. 324).

12) Ponce de Leon, G., Jentzen, G., Fredlund, D., Spittler, P. & Field, D. (2010). Guide to the forensic scheduling body of knowledge Part I

13) Williams, T. (1992). Criticality in stochastic networks

Ask Questions


Get Answers


A budget reserve is to contractors as red meat is to lions, and they will devour it!

Attributed to Bent Flyvbjerg by Kahneman in Thinking Fast and Slow

Photo source: http://www.vegansoapbox.com/we-are-not-lions/


THANK YOU!Gui Ponce de Leon PhD, PE, PMP, LEED AP

Inventor of GPM® & Developer of NetPoint®/NetRiskTM

Truth in Scheduling®

risk modeling and analysis (mitigating the planning fallacy)

Business

schedule risk analysis

pma technologies2

pma technologies5

pma technologies6

pma technologies10

pma technologies14

pma technologies15

risk management