1 2009 cdc project management summit eplc project reviews & specifics on the integrated baseline...

1

2009 CDC Project Management Summit EPLC Project Reviews

&Specifics on the Integrated Baseline Review

Auditorium B-210:30 – 11:30am

February 6, 2009

Gustavo A. CalderónCompass Systems Consulting, Inc.

2

EPLC Project Reviews

&

Specifics on the Integrated Baseline Review

1. EPLC Project Reviews – A Synopsis

2. Understanding the IBR

3. IBR Deliverables

3

Session Description

Project reviews are formal reviews by the project manager conducted at specific points in the life cycle to ensure that events have occurred and decisions have been made before continuing with the project. The EPLC contains 13 project reviews throughout the lifecycle. This session will highlight these and provide detailed information of the IBR.

EPLC Project Reviews – A Synopsis

4

HHS EPLC State Gate Review Requirements

1 2 3 4

5

6

7

8

9

10

11

13

12


5

Enterprise Performance Life Cycle – 13 Project Reviews

Architecture Review

Requirements Review

Detailed Design Review

Validation Readiness

Review

Independent Verification &

Validation Assessment

System Re-Certification

System Re- Accreditation

Disposition Review

Implementation Readiness

Review

System Certification

System Accreditation

PlanningRequirements

AnalysisDesign TestDevelopment Implementation

Operations & Maintenance

DispositionConceptInitiation

1 2 3 4

6

7

8

9

11 13

12

5Integrated Baseline Review

EPLC Project Reviews

“Project reviews are formal reviews by the PM conducted at specific points in the life cycle to ensure that events have occurred and decisions have been made before continuing with the project.” EPLC Framework, Page 19.

Post-Implementation

Review

10


6

Architecture Review

EPLC Project Reviews (#1-4)





1

The Architecture Review is performed to ensure that the Business Needs Statement is sound and is consistent with the Enterprise Architecture. Pg. 28.

Integrated Baseline Review

2

The Integrated Baseline Review (IBR) is an internal inspection led by the Integrated Project Team to verify that the project baseline is in place, together with a realistic budget to accomplish all planned work. The IBR includes an evaluation of the Performance Measurement Baseline for realism and inherent risks. When contractor resources are involved, the IBR provides a forum through which the government’s team gains a sense of ownership and understanding of the contractor’s management process and assurance that earned value management has been appropriately established for the project. Pg. 34.

2

2

3

Requirements Review

3

Detailed Design Review

4

The Requirements Review is conducted to verify that the requirements are complete, accurate, consistent and problem-free; to evaluate the responsiveness of the requirements to the business requirements; to ensure that the requirements are a suitable basis for subsequent design activities; to ensure traceability within the requirements and between the design documents; and to affirm final agreement regarding the content of the Requirements Document. Pg. 38

The Detailed Design Review (DDR) is conducted subsequent to a PDR to achieve confidence that the individual design components (units/modules) of an automated system/application, and how they interface with one another, have been completely defined and documented in sufficient detail such that the design of the automated system/application is complete, fully integrated, and ready to move to the Development Phase.Pg. 41.


7

Validation Readiness Review






5

2

2 Independent Verification &

ValidationAssessment

6

The VRR is conducted to provide assurance that the software that is about to enter validation (system) testing has completed thorough unit/module/software integration testing during the development of the automated system/application and is ready for turnover to the formal, controlled test environment where validation testing will be conducted. The scope of the VRR is to inspect the test products and test results obtained during development testing for completeness and accuracy, and to verify that test planning, test cases, scenarios, and scripts provide adequate coverage of documented system requirements. In addition, a review of the test environment, test setup, and test data is performed to ensure they are adequately prepared for validation testing. Pg. 44

An IV&V Assessment is conducted by an independent third party to identify potential improvements that may not be apparent to those working directly on a project, or identify problems before they occur and thus avoid loss and minimize the cost of any necessary corrective action. IV&V Assessment also provides management with an independent perspective on the full scope of project activities, from planning through implementation. Pg. 44.


8

Implementation Readiness Review






7

2

2 System Certification

8

IRR is conducted to ensure that the IT solution or automated system/application that has been developed is ready for implementation activities, such that the required system hardware, networking and telecommunications equipment; COTS, GOTS, and/or custom-developed software; and database(s) can be installed and configured in the production environment(s). Pg. 46

System Accreditation is the official management decision to authorize operation of an information system. To make an informed decision, the organization’s Chief Information Officer (CIO) / Designated Approval Authority (DAA) must have sufficient knowledge and understanding of the current status of the security programs and security controls in place to protect the system and information processed, stored, or transmitted by the system. This is a business-driven, risk-based decision founded upon current, credible, comprehensive documentation and test results provided in the System Certification package prepared as a result of predecessor System Certification activities. Pg. 49.

2

2 System Accreditation

9

System Certification is the comprehensive evaluation of the management, operational, and technical security controls implemented for an information system to ensure compliance with information security requirements. The certification evaluation includes review of the Information Security Risk Assessment (IS RA), System Security Plan (SSP), other system life cycle documentation, and any findings from past assessments, reviews and/or audits, as well as technical testing and analysis. The technical certification assessment, called the Security Test and Evaluation (ST&E) process, is the execution of test procedures and techniques by an independent third party designed to evaluate the effectiveness of information security controls in a particular environment, and to identify any vulnerabilities in the information system. Pg. 49.


9

Post-Implementation

Review






10

2

2System Re-certification

Post-Implementation Review. After a period of sustained operation (after at least one full processing and reporting cycle has been completed and all users have been trained and are comfortable with the operation), a PIR is conducted of the completed IT solution or automated system/application that was released into the production environment to determine if it is operating as expected. The purpose of the review is to ascertain the degree of success from the project (in particular, the extent to which it met its objectives, delivered planned levels of benefit, and addressed the specific requirements as originally defined), to examine the efficacy of all elements of the working business solution to see if further improvements can be made to optimize the benefit delivered, and to learn lessons from the project that can be used to improve future project work and solutions. Pg. 50.

System Re-Certification is the comprehensive re-evaluation of the management, operational, and technical security controls implemented for an information system that is performed during the Operations & Maintenance Phase to ensure that the system is continuing to operate at an acceptable risk level. Over the life of the system, many changes occur that may reduce the effectiveness of internal security controls. Security controls typically become outdated and less effective as threats and vulnerabilities evolve. The objective of the System Re-Certification is to ensure that system certification is an on-going process, and that information security is managed throughout the life of the system. Pg. 55.

2

2 System Re-Accreditation

11

12

System Re-Accreditation is the official management decision to authorize continued operation of an information system after acceptable System Re-Certification and any necessary adjustments have been completed. Pg. 56.


10

Disposition Review

EPLC Project Reviews (#13)





13

2

Disposition Review. A Disposition Plan should be developed and reviewed by the project team should the [yearly] Operational Analysis conclude that the investment should be terminated. Pg. 56.


11

What is an Integrated Baseline Review?

Is it a meeting?

Is it a seminar?

Is it a workshop?

Is it an event?

Is it a process?

How long does it take to do one?

Who participates?

What do we mean by the word “Integrated”?

What do we mean by the word “Baseline”?

What do we mean by the word “Review”?Understanding the IBR

12

What is the Baseline? The PMB!

Scop

e Cost

Time

The Performance Measurement Baseline (PMB) is the embodiment of the Triple Constraint!

WBS

Critical Path Schedule

Budget

Understanding the IBR

13

ANSI/EIA 748-B Criterion No. 8 Performance Measurement Baseline (PMB)

Establish and maintain a time-phased budget baseline, at the control account level, against which program performance can be measured. Initial budgets established for performance measurement will be based on either internal management goals or the external customer negotiated target cost including estimates for authorized but undefinitized work. Budget for far-term efforts may be held in higher level accounts until an appropriate time for allocation at the control account level. If an over-target baseline is used for performance measurement reporting purposes, prior notification must be provided to the customer.

PMBOK ®Performance Measurement Baseline (PMB)

An approved plan for the project work against which project execution is compared and deviations are measured for management control. The performance baseline typically integrates scope, schedule, and cost parameters of a project, but may also include technical and quality parameters.

What is a PMB?


14

Source: EVM Gold CardDefense Acquisition University

Management Reserve

Cost Variance

Schedule Variance

ACWP

BCWP

BCWS

$

EAC

TimeNow

Completion Date

PMB

TAB

BAC

time

What does it mean when the PMB starts at $0:

-for a program?

-for a project?

-for a contractor?

How should the baseline & subsequent completion dates be determined?

68

How is a PMB Constructed?


15

Are all PMBs & IBRs the Same?

Initiation

Planning

Closeout

Project Charter

Project Plan } PMB

ImplementationDevelopment“Execution

& Control”

Product Delivery

} Go Live

FINAL PRODUCT

EVM Reporting

EVM SetupHow does the PMI Planning Process Area Compare to the EPLC Planning Phase?

IBR


! !

16


1. Lays a solid foundation for mutual understanding of project risks;

2. Provides an invaluable opportunity to compare PMs’ expectations, and to address differences before problems arise;

3. Provides project management teams with a thorough understanding of the project plan and its risks, allowing early intervention and the application of resources to address project challenges;

4. Increases confidence in the project PMB, which provides a powerful, proactive, program management capability to obtain timely and reliable cost and schedule projections;

5. Enables the principles of management by exception and improves problem traceability rather than require continuous oversight of all tasks;

6. Early warning. Indicates potential problems early; and

7. Earned value management. Enables management to quantify the impact of known problems, to measure work accomplished, and to obtain realistic estimates at completion.

Source: Program Manager’s Guide to the IBR (April 2003)

Why conduct an IBR? To understand & manage risks …


17


Why conduct an IBR? To understand & manage risks …

An IBR’s purpose is to verify the technical content and realism of the interrelated performance budgets, resources, and schedules. It helps the auditor understand the inherent risks in offerors’ or contractors’ performance plans and the underlying management control systems, and it should contain a plan to handle these risks.

Source: GAO Cost Assessment Guide (July 2007)


18


Tim Lister’s Surprises:

1. The biggest risk an organization faces is lost opportunity, the failure to choose the right projects.

2. The real reason we need to do risk management is not so much to survive our risks, but to enable risk-taking.

3. The only reason to quantify cost (schedule and budget) is to have something to compare to your quantification of budget.

4. There is no sense making cost quantification more precise than value quantification.

5. An aggressive delivery date is often driven by a cost containment motive, not an urgent date motive.

Tim Lister

PMI RISK SIG 1-3 May 2005 Brief

Why conduct an IBR? To avoid surprises …


19


What are the risk events in the program?

Known Unknowns (RISK) can assess probability & consequences) – Contingency Planning

Unknown Unknowns (UNCERTAINTY) True Uncertainty – Management Reserve

Where are the risk events located in the program?

Technology, Design, T&E, M&S, Cost, Funding, Schedule

Program Environment/Structure – Functional or Product or both

Oversight and Politics – Changing Department Guidance or Priorities

US and World Markets – Economic cycles, Labor Markets, Elections.

Examine Sources / Areas of Risk

• Strategic and Tactical - Outside and Inside the PMO.

• Product or Process – Predicts the Product Problems.

Why conduct an IBR? To manage risk & uncertainty …


20


Why conduct an IBR? To manage risk & uncertainty …

Risk represents a situation in which the outcome is subject to an uncontrollable random event stemming from a known probability distribution. Risks have Known-unknowns. Plan for risk with the appropriate level of contingency reserves.

Uncertainty represents a situation in which the outcome is subject to an uncontrollable random event stemming from a unknown probability distribution. Uncertainty has unknown-unknowns. Plan for uncertainty with the appropriate level of management reserves.


21


New IT products & IT systems introduce change into organizations, and change is never uniform in its impact on different constituencies.

The basic rule is:

Every time an IT product is delivered, somebody gains power and somebody else loses power. Both the power gainers and the power losers are, by definition, stakeholders.

Tom DeMarco and Tim Lister

0 7 4 0 - 7 4 5 9 / 0 3 / $ 1 7 . 0 0 © 2 0 0 3 I E E E

Why conduct an IBR? To manage stakeholders expectations …


22


New IT products & IT systems introduce change into organizations, and change is never uniform in its impact on different constituencies.

The basic rule is:

Every time an IT product is delivered, somebody gains power and somebody else loses power. Both the power gainers and the power losers are, by definition, stakeholders.

Tom DeMarco and Tim Lister

0 7 4 0 - 7 4 5 9 / 0 3 / $ 1 7 . 0 0 © 2 0 0 3 I E E E



23

1. Work Breakdown Structure – with a WBS Dictionary Definition for each Work Package and Planning Package

2. Risk adjusted Cost Estimates – comprehensive documentation of the estimating methods, risks, and assumptions (refer to the GAO Cost Assessment Guide)

3. Project Schedules – Two critical path schedules are needed; one at the Work Package level used to create the PMB and the other at the detailed task level where resources are loaded and assignments made

The 6 essential deliverables for an IBR:

IBR Deliverables

IBR Deliverables

Scop

e Cost

Time

WBS

Critical Path ScheduleB

udget

#1 #2

#3

24

T I M E

C O

S T

IBR Deliverables

The 6 essential deliverables for an IBR:

PMB

#4 4. Performance Measurement Baseline (PMB) – multiple baselines may be used; Program Level, Vendor-specific, Customer-specific other.

5. Risk Log and Risk Management Plan – comprehensive identification and plan to handle risks

6. Earned Value Management System (EVMS) – detailed documentation of the EVM approach to be adopted vis-à-vis the 32 ANSI/EIA 748-B criteria.

IBR Deliverables

25

IBR Deliverables

Which other deliverables are important but not essential for an IBR?

7. Project Charter – the key deliverable of the Concept Phase.

8. Business Case (Exhibit 300) – a deliverable started in the Concept Phase.

9. Project Management Plan – a key deliverable of the Planning Phase.

Risk Management - Acquisition Plan - Change Management - Configuration Management - Project Categorization - Requirements Management - Communications Plan - Work Breakdown

Structure (WBS) /Project Schedule - IV&V Planning - Quality Assurance - Records Management - Staff Development Plan - Security Approach

Please note that Deliverables 7-9 plus the results of the IBR will be reviewed in detail at the Project Baseline Review (PBR) - Stage Gate Review

IBR Deliverables

26

New IT products & IT systems introduce change into organizations, and change is never uniform in its


IBR Deliverables

IBR Deliverables

27

Thank You!

Gustavo A. CalderónPresident

Compass Systems Consulting, Inc.1934 Old Gallows Road, Suite 350

Vienna, VA 22182-4050(main) 703-832-8181(fax) 703-832-1136

(mobile) [email protected]

www.compass-sc.com

1 2009 cdc project management summit eplc project reviews & specifics on the integrated baseline...

Documents