Pragmatic Uses for Risk Management Practices

A work in progress !!

Presented by Douglas Brown, PMPPMO HeadSEC Office of Information Technology202-551-8176BrownDo@SEC.gov

PMI Silver Spring Chapter12 April 2006

Disclaimer

The Securities and Exchange Commission, as a matter of policy, disclaims responsibility for any private publication or statement by any of its employees. The views expressed herein are those of the author and do not necessarily reflect the views of the Commission or of the author’s colleagues upon the staff of the Commission.

Perspectives

SEC IT Approximately $95 million annually

Went from $40 M to over $100 M 2002-2004 Seismic culture shift, aftershocks still settling

103 internal IT employees, 300+ contractors supporting 4000 agency employees at HQ and 10 regional offices 1000-3000 contractors and other users Several thousand “regulated entities” Millions of individual investors (EDGAR)

40-60 new projects or new phases annually 60-100 ongoing projects at any one time

Risk Management – by the PMBoK

Risk IdentificationRisk AnalysisRisk Response PlanningRisk Monitoring and Control

Generally results in A Risk Management Plan That sits on the shelf But meets a compliance requirement

Risk Basics

A good word for OMB-300 E-300 specifies that project schedules and budgets must

be specifically risk-loaded This is the genesis of SEC’s approach

Differentiate between Risk and Issue Risk = EVENT or CONDITION that MAY occur and, if it did,

would lead to project failing to meet baseline ISSUE = something that is happening (or, usually, NOT

happening) now that will result in failing to meet baseline if not resolved by X date.

ACTION ITEM = something that someone needs to do to carry out their part of the plan. If they do not, it would be a problem but we do not have any reason to believe they will not (or if we do so believe, then we have an ISSUE).

SEC OIT Uses for Risk Management

Initial project concept ROM estimation – FY and TCO ID most obvious risks

Selection of SDLC/PM style

Pre-Acquisition Review Account for the 19 OMB risk elements Derive risk score Assign risk strategy Allocate cost and schedule buffers

Control

Evaluate

Ranges of Estimates

-40

-20

0

20

40

60

80

Co

nc

ep

t

Pla

nn

ing

Ex

ec

utio

n

Te

st &

Tra

in

De

plo

y

Ma

inta

in

PMBoK range estimates at various points in SDLCBudgeting process does not recognize rangesConflicting interests:

Pad to avoid failure Understate to avoid project

disapproval

Uncertainty = risk. Recognition is 90% of the battleBuffersProgram-level unallocated funds

Pre-Select: ROM Estimation

Need a number 24-48 months out

Estimating cost of 18 servers is easy – but why 18?

Software: SLOC meaningless

nowadays Function points imply design

work largely done Can’t estimate from user

requirements – or can we? What about re-use

SEC Directions Establishing repository to

permit development of parameters

Establishing EA maps to functional components to permit identification of re-use opportunities

Provide a ROM estimate tool for use in those “not a clue” situations

Will be refined over time based on actuals

Seeking to work with other agency estimation processes

ROM estimator

Investment: As of: 2/2/2005Prepared By:

Select One

YesNoNoNoNo

Storage Impact Yes Hardware Yes Software Yes Services YesAmount of Storage N/A Complexity Easy Complexity Easy Complexity Easy

Facility - Data Center Project Cost Breakdown (w/o Buffer & Steady State Costs)Size Required N/A Project Category Cost

Hardware $73,750Hardware ROM Cost Software $6,500

Hardware ROM $73,750 Services $200,040ROM + Complexity Factor $73,750 Storage Impact $0

Hardware Buffer 20% Facility $0 + Buffer $14,750 Security $8,025

Software ROM Total $88,500 Total $280,290

Software ROM Cost Buffer Total

Software ROM $6,500 $56,058ROM + Complexity Factor $6,500

Software Buffer 20% Lifecycle Breakdown + Buffer $1,300 Phase % Cost Buffer

Software ROM Total $7,800 Planning 0% $0 $0Analysis 25% $70,073 $10,002

Services ROM Cost Solution 45% $126,131 $31,165Services ROM $200,040 Test/Training 10% $28,029 $6,890

ROM + Complexity Factor $200,040 Deployment 20% $56,058 $8,002Services Buffer 20% ROM Cost Est. 100% $280,290 $56,058

+ Buffer $40,008Services ROM Total $240,048 Total Cost through Deployment by Lifecycle Phase (Cost + Buffer)

Phase Total CumulativeSecurity 10% Planning $0 $0

Analysis $80,075 $80,075Maintenance 28% Solution $157,295 $237,370

Test/Training $34,919 $272,288Steady State 25% Deployment $64,060 $336,348

ROM Breakdown Steady State Total (Security / Maintenance / Steady State)Category Cost $211,899

Hardware $88,500Software $7,800 ROM Cost EstimateServices $240,048 Project Category Buffer Steady State Total

Storage Impact $0 $280,290 $56,058 $211,899 $548,247Facility $0 N/A N/A

Security $33,635 Yes Easy SmallMaintenance $94,177 No Moderate MediumSteady State $84,087 Difficult Large

ROM Cost Estimate $548,247

Update the cells highlighted in green. All sub-totals will be automatically calculated in the grey cells. The Rough Order of Magnitude total will be calculated in the purple cell.

Test Project

Rough Order of Magnitude (ROM) Cost Estimate

J. Kluger

This project affects only one small Program This project affects a large Program Office, but only This project affects multiple Program This project affects Regional/District Offices as well as the SEC This project affects the SEC Enterprise (all users, all locations)

Decide how much you can accomplish

and/or afford in this FY

Cost Breakdown

Qualitative data entry in green cells- complexity- scope

Cost through deployment

Buffer

TCO ROM

Pre-Select: Concept Approval

ROM process identifies buffer (30-150% of base)

PM can request less (usually acquisition-only)

Concept request identifies “most likely reasons why project might fail” – bullet list

SDLC and PM Style as Risk Tools

3 SDLCs Structured (waterfall) Iterative (releases of functionality) Acquisition-only (straight purchases) - assigned at time of pre-

select decision

3 PM Styles PM-Lite Custom PM: as needed, based on risk and complexity; Level 1

or Level 2 PM assigned

PM Levels – conform to Acquisition Workforce Collateral duty Level 1 – system supporting single SEC office Level 2 – enterprise or complex functional system Level 3 – multi-agency (no such project yet)

Select: Pre-Acquisition Review

19 Risk Elements assigned in OMB Circular A-11 Some overlap but consistency has value Each area assigned High, Medium or Low for

probability and impact Positive outcomes also assessed, treated as HIGH

to protect them Identify one or more risk statements per area to

explain Explain AVOIDANCE plan for HIGH-HIGH, HIGH-

MED risks

Risk Assessment

Risk Category Risk Level Score Risk Category Risk Level Score

Schedule: Risk that some projects in the program will overrun current timeline

1 Overall Failure Risk: Risk that the some projects will fail to achieve scope, cost or schedule

1

Initial Cost: Risk that some projects in the program will overrun cost estimates for current request

1Organizational Change: Risk of program failure because affected divisions, offices, or external parties resist the required behavioral change

1

Life Cycle Cost: Risk that some projects in the program will overrun future-year costs

1Fluid Requirements: Risk that some projects in the program requirements will substantially change during implementation period

1

Obsolescence: Risk that technology being implemented will create future cost, integration, or support issues because of aging/obsolescence

1Data Stability: Risk that the nature of data used by the some projects will change during the course of the program (including operational phase)

1

Feasibility: Risk that some projects will run into implementation issues due to unknown technology OR business-side realities (right idea, wrong time)

1Technical Delivery: Risk that some projects in the program will run into implementation issues due to excessive complexity, or technical difficulty or failure

1

Reliability: Risk that delivered technology will experience too much downtime or other discontinuity of service to fulfill requirements

1

Strategic Impact: +ve: Project success = large benefits

–ve: Project failure = damage agency’s ability to do its mission.

1

Interoperability: Risk that some projects will not integrate well with rest of SEC technical environment

1 Security: Risk that some projects in the program could create or run into significant security issues

1

Asset Protection: Risk that some program products (including data generated) will be lost or stolen.

1 Data Privacy: Risk that some projects may result in exposure of protected personal data

1

Procurement Monopoly: Risk that approach will result in vendors being “locked in” and able to raise prices or degrade support over time with impunity

1Program Resources: Risk that most projects in the program will not receive the funding or staff participation required for success

1

SEC Oversight: Risk that the SEC will not effectively manage this program

1 Total Risk Score 19

H M L += 4 = 2 = 1 = 4

L

L

L

L

L

L

L

L

L

L

L

L

L

L

L

L

L

L

L

Assigning risk buffers

Pretty simple: risk score = expected buffer

Review board questions buffers that deviate from the risk score

Last year, tried 1-4 as scale (minimum 19, maximum 76)

For 2006, adjustments: Accommodating acquisition-only projects BUT experience that

most projects WERE under-estimated = revise buffer range to 5% to over 100%

Minimum score 4.75, maximum could be 161 – but highly unlikely to approve project with 4-6 high risk elements

Introduction of pre-acquisition review (more detail available)

Future directions in measurement

Specify risk management approaches HIGH = Avoid Moderate (<9) = Mitigate, Transfer Low (0.25) = Accept

Refine buffer calculations as data gained

Narrow the total size of buffer assigned by ROM

Reward PMs for declaring and returning buffer – without encouraging padding to get reward

Reward contractors for early delivery under budget with incentive-based contracts

Risk Management in Control Phase

Risk log for regular risks

Project dashboard Performance light based on schedule and cost buffer

consumption “Management attention” light for PM to declare need

for help (risk has become issue) Customer satisfaction light for customer to sound

alarm

Re-evaluation at SDLC phase gates

Buffer Consumption

% complete

0%

100%

0% 100%

SDLC Phases and Associated Reviews

Business case,Approval

Contract award

High-levelreqts

DetailedBusiness &TechnicalReqt’s

TechnicalDesign

Solution OITAcceptance

BusinessValue

Acquisition- only

Structured

Iterative

Output

Initiation Planning Analysis Design Solution Test Train/Deploy Opns Retirement

Release 1

Release 2

Release N

Diamonds are mandatory go-forward milestones.GOLD = Formal review; BLUE = sign-offs

Steady State

Evaluation Phase

Project Close-out Reports Review issues Compare to initial risk assessments Gather actual cost and schedule data

Conduct 90-120 Day Operational Assessments

Recap: Uses for Risk Management

Initial project concept ROM estimation – FY and TCO ID most obvious risks

Selection of SDLC/PM style

Pre-Acquisition Review Account for the 19 OMB risk elements Derive risk score Assign risk strategy Allocate cost and schedule buffers

Control

Evaluate

?

?

Conclusions

Knowing the enemy and yourself

Those who are ignorant of history are doomed to repeat it

Pride goeth before a fall

Pragmatic Uses forRisk Management Practices

Douglas M. Brown, Ph.D., PMPPMO HeadU.S. Securities and Exchange Commission Office of Information Technology

202-551-8176BrownDo@SEC.gov

PMI Silver Spring Chapter12 April 2006