closeout presentation lbnf cd-3a evms implementation review · closeout presentation lbnf cd-3a...
TRANSCRIPT
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 2 of 16
This page intentionally left blank
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 3 of 16
Table of Contents
Table of Contents .............................................................................................................................................. 3
1.0 Executive Summary ............................................................................................................................... 5
2.0 Introduction ............................................................................................................................................ 6
3.0 EVMS Implementation ........................................................................................................................... 7
3.1 EVMS Processes Including Cost and Schedule ...................................................................................... 7
4.0 Appendices ........................................................................................................................................... 12
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 4 of 16
This page intentionally left blank
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 5 of 16
1.0 Executive Summary
An Earned Value Management (EVM) Implementation Review of the Long-Baseline Neutrino Facility
(LBNF) Project was held on January 11-12, 2016 at the Fermi National Accelerator Laboratory (Fermilab).
At the request of Marc Kaducak, Head of Office of Project Support Services, the purpose of the review is to
assess the progress and plans for execution of the EVM processes on the CD-3a scope for the LBNF Project,
in accordance with standard DOE and Laboratory requirements. LBNF is a pre CD-2 project and not subject
to the full DOE Order 413.3B project EVMS processes. Nonetheless, it is managed and is evaluated according
to standard project management practices and has been maintaining a practice preliminary baseline. The focus
of this review is to ensure the implementation of the Fermilab EVMS on the LBNF Far Site Conventional
Facilities (FSCF) practice preliminary baseline and determine if the project team is ready to effectively
monitor and manage cost, schedule, and technical performance of the CD-3a portion of the FSCF upon award
of the first construction package. Only this area of scope was assessed for this review.
Overall, the committee found that the project team is committed to implementing EVM practices and
processes. The project team presented their own lessons learned based on evaluation of their preliminary
practice baseline, which is encouraged to continue. The Committee determined that there are two main areas
where additional improvements will be needed: reviewing issues identified with Work Authorization
Documents (WADs) prior to implementing a formal baseline and application of already identified lessons
learned once the formal baseline is established. By addressing these recommendations, the LBNF FSCF
project scope should be well placed to successfully set a baseline and begin official reporting.
The Committee identified additional areas of improvement – in the form of comments – that relate primarily
to performing within an EVMS more effectively. These, while important, are not necessarily preventing the
project from proceeding to establishing a successful baseline for reporting. Practice related to the EVMS
should continue to allow for a successful baseline establishment and reporting of the entire LBNF project.
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 6 of 16
2.0 Introduction
The purpose of the review is to assess the progress and plans for execution of the EVM processes on the CD-
3a scope for the LBNF Project, in accordance with standard DOE and Laboratory requirements. This review
is to satisfy a recommendation from the DOE’s CD-3a Independent Project Review in December 2015 that
the project successfully complete an EVM implementation review prior to award of the first construction sub-
package.
This closeout report is broken down into two basic sections. The first section of this presentation has the
Committee’s answers to the review’s charge question. The second section provides the assessments of the
project’s EVM implementation and related data. Each area within this first section is organized by Findings,
Comments and Recommendations. Findings are statements of fact that summarize noteworthy information
presented during the review. The Comments are judgment statements about the facts presented during the
review and are based on reviewer's experience and expertise. The comments are to be evaluated by the project
team and actions taken as deemed appropriate. Recommendations are statements of actions that should be
addressed by the project team.
The LBNF Project is encouraged to develop a response to the assessment recommendations. The
recommendations should be tracked to closure and a documented status of the project’s resolution should be
available for future reviews.
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 7 of 16
3.0 EVMS Implementation
3.1 EVMS Processes Including Cost and Schedule
Committee Lead: Jennifer Fortner
Committee Members: Sally Anderson, Sherese Humphrey, Fran Leavell
Charge Questions:
● Is the project team knowledgeable of its EVM roles and responsibilities and have they been trained on
FRA’s EVM processes and procedures? Yes, the LBNF FSCF project team members appeared
knowledgeable and have been trained regarding EVM.
● Has the project established and maintained a preliminary baseline for the Far Site Conventional
Facilities satisfying the requirements of FRA EVMS System Description (SD) and procedures as they
pertain to ANSI Standard 748-B? Yes, while there are areas of improvement identified below, the
preliminary baseline appears to have functioned in a way to allow the project team to learn and practice
EVM prior to full implementation.
● Is the project following FRA’s EVM requirements and standards? Yes, however, there are areas of
improvement and compliance identified below.
● Are Corrective Actions from the FRA March 2016 annual surveillance review being addressed and
implemented by the LBNF project team? Yes, while not all corrective actions are fully implemented
they have been identified as issues or lessons learned and will be implemented going forward.
● Are monthly reports being used to inform project stakeholders and shape advanced planning? Yes, the
project is using monthly reports for information and resulting corrective actions to advance planning.
● Are corrective actions being identified and improvements implemented in forward planning? Yes,
while not all corrective actions are fully implemented they have been identified as issues or lessons
learned and will be implemented going forward.
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 8 of 16
● Is change control being properly administered in accordance with FRA’s requirements? Yes, however,
there are areas of improvement identified below.
● Is the project ready to implement a baseline on the CD-3a scope of work prior to award of the first
construction package? Yes, with implementation of the items identified below. It should be noted that
upon award of the construction package, changes to the baseline will occur to allow for the
incorporation of the contract awards. This is understood by the project team.
Findings
● An Organizational Breakdown Structure (OBS) was presented for the entire Long Baseline Neutrino
Facility/Deep Underground Neutrino Experiment (LBNF/DUNE) project including specifics on the
Far Site Conventional Facilities (FSCF) scope. This included internal and external organizations
responsible for accomplishing work.
● A Responsibility Assignment Matrix (RAM) was presented for the preliminary baseline Far Site
Conventional Facilities (FSCF) scope. This included eleven Control Accounts (CAs) and one
Control Account Manager (CAM).
● The project team presented details on lessons learned gathered over the course of implementing the
preliminary baseline.
● One CA was 100% complete at the time of the review (131.01.02.02.02 FSCF Preliminary Design)
and two CAs contain a mixture of LOE and discrete activities below the 15% threshold outlined
within the FRA EVM system description and procedures.
● There were eleven Work Authorization Documents (WADs) provided for the review with updates
through BCR135.
● One CAM was interviewed during the review, responsible for the FSCF preliminary baseline and
CD-3a project scope.
● A preliminary “practice” baseline was established March 2016 for the Far Site Conventional
Facilities (FSCF), WBS 131.01.02.02. The initial baseline was set at $ $317,140,998 in Project
131.01.02.02 (aka “B” project) and does not include completed work under four of the 11 FSCF
control accounts, of $3,644,860, which is in Project 131.01.02.02 A.
● The CD-3a scope is captured as planning packages. These planning packages are broken down into
discrete activities as the contracts are awarded.
● Multiple calendars are used within the project and are assigned at the activity level.
● Funding constraints are on the critical path making the project funding driven until FY19 when the
schedule becomes technically driven.
● The FSCF project has 77 relationships with other projects including links to project office, scope for
each of the caverns, filling the shaft and cryostat, staffing oversight activities and the non-DOE
funded cryostat for the first cavern.
● In the Far Site Conventional Facilities baseline schedule there are 1113 activities, 36 completed, 1 in
progress, 200 with constraints, 85 without predecessors, 29 without successors, 11 out of sequence.
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 9 of 16
● Project controls works with the CAM to collect, record, and produce monthly performance
information: this includes the Turn-Around Report and Validation Report; Cost and Schedule
Reports; Milestone Reports; Corrective Action Log; and various other miscellaneous reports.
● The project uses the thresholds defined in the Laboratory’s EVM Description for variance analysis.
● The project uses a Corrective Action Log to log and monitor through to completion corrective
actions identified while performing variance analysis.
● CAM reviews project information and assist the project manager in the reporting of project
performance.
● PARS-II Reporting will occur upon completion of the baseline for the CD-3A scope. The project
anticipates a June 2017 upload for the May 2017 reporting period.
● There were 17 changes approved against the FSCF baseline through end of November, with a total
cost impact of $10,788,647, resulting in a total approved budget of $327,929.645.
● Adding completed work (not included in the FSCF baseline) to the end of November FSCF Current
Approved Budget results in the BAC shown on the November 2016 stoplight report of $331,575K
for WBS 131.01.02.02.
● The FSCF CD-3a scope is a subset of WBS 131.01.02.02. The CD-3a scope is contained in 11
unique control accounts and the baseline will be maintained in a separate Cobra project.
● The FSCF CD-3a scope is not being tracked on the FSCF change log. It is, however, being tracked
on the total project log, which has columns to show the BCWS and Schedule Impact at the BCR
summary level for total project, FSCF post Baseline, and CD-3a Scope.
● The FSCF CD-3a scope had an initial baseline of $219,137,742. Changes approved through
November 2016 of $4,988,971, result in an approved budget of $224,126,715. The log shows 18
changes are associated with CD-3a scope
● The HEP Project Status Report – November 2016 for LBNF/DUNE Project includes changes
implemented in October 2016. The Current Approved Value contained in the summary change
status matches the change log for FSCF and FSCF CD-3a, as of end of October 2016.
● There are two change control logs being maintained. The LBNF-DUNE (total project) log and a
FSCF log. The logs are consistent with FRA’s requirements.
● Management Reserve (MR) is held at the LBNF project level and this is noted on the FSCF Change
Control Log. MR will not be partitioned to the CD-3a subproject.
● The FRA Change Control database being developed as the FRA tool is being used by the project and
is expected to finish testing next week.
● The change package for the BCRs contain the reports required by FRA-- BCR Implementation
History Unchanged Validation, BCR Milestone Tier Schedule, BCR Impacted Activities, BCR
Affect on Critical Path, BCR Change Control Account Log --as well as signature information and
Before-After details down to the WP level.
● CR-0125, Update FNAL Rates for 2017, reflected a change in the current period, which Richard
Marcum confirmed had been approved by OPSS.
● The CAM explained that CR-0124 was the movement of scope and budget to an earlier date within
the same fiscal year. The cost impact of $24 does not show up on the WAD BCR Note, because the
WAD Note is in $K.
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 10 of 16
● Changes are incorporated in a timely manner. The log shows all FSCF changes were implemented
in the same month they were approved. There have been no retroactive changes.
● All changes to the FSCF budget, which were available for review, followed the project’s change
control process and were documented via BCR.
● Current budgets are reconciled to prior budgets, as are start and finish dates, at the control account
level (see BCR Change Control Account Log).
Comments
● Communication between the team, despite being at different geographical locations, appears
effective and well-coordinated.
● The CAM interviewed demonstrated ownership of schedule, cost, and scope information. The CAM
understood responsibilities relating to change control and was able to articulate process steps. The
CAM is knowledgeable about the monthly status reporting requirements, comfortable reviewing
variance documentation, and drafting explanations associated with variances.
● Prior review corrective actions were addressed, if not actually implemented, within the preliminary
baseline. There is one corrective action that still needs to be demonstrated related to long duration
activities and objective information. The project team acknowledged that this needs to be addressed
going forward for the CD-3a scope.
● The project team should be commended for collecting lessons learned during the implementation of
the preliminary baseline. This practice should continue going forward and be shared with other
projects within the laboratory to further learning and growth. The project is encouraged to
incorporate the identified issues into the CD-3a scope.
● Five of the eleven WADs were compliant. The exception being related to one (131.01.02.02.01)
which had not been updated for all November BCRs due to an issue with rounding of delta reports
and five others that had issues with period of performance consistency between the CAP, EV engine,
and schedule. A consistent method of displaying dollar values should be used going forward and an
indicator of change below $K should be considered. Additionally, issues related to the period of
performance inconsistencies should be investigated and corrected.
● Planning Packages use LOE activity types as opposed to task dependent which may cause the project
to show the total float and critical path incorrectly. Additionally, LOE activities use a task dependent
activity type as opposed to LOE which could cause them to appear on the critical path incorrectly or
drive logic within the schedule.
● Discrete activities spanning more than two months must have defined, objective means of measuring
progress. This backup is missing from the schedule within the preliminary baseline and should be
reviewed and updated per the schedule guidelines. This was, also, identified by the project team as a
lesson learned.
● Schedule documentation regarding some lead/lags and constraints is missing within some areas of
the project. These should be reviewed and updated within the schedule.
● Since the completed portion of four control accounts under WBS 131.01.02.02 are budgeted in a
separate project file (131.01.02.02 A), there is inadequate traceability at a control account level
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 11 of 16
within the preliminary practice FSCF baseline project CA 131.01.02.02. The project needs to resolve
how best to trace approved FSCF budget against the BAC being reported at a WBS or control
account level in any LBNF monthly reports.
● CA 131.01.02.02.02 FSCF Preliminary Design Variance Analysis Report reflects a CPI of 4.44.
This was the result of the project miscalculating remaining/outstanding cost for baseline portion of
this control account. The cost performance index is more in-line with thresholds when the entire
control account is combined with the unbaselined portion of the control account. This can be
confusing to a reviewer when trying to investigate or reconcile performance since the is one CA split
between two baseline files. This also impacts the true variance-at-complete that is being reflected
and can mislead the project team.
● Since the preliminary “practice” baseline and the CD-3a scope changes are being tracked on the total
project change log, the project should see if there is any value added by maintaining a separate FSCF
change log. The items identified as impacting CD-3a should also have impacted the FSCF report.
This should be investigated and corrected as appropriate.
● The initial baseline for FSCF CD-3a scope is expected to be established prior to work starting May
2017. The project will create a separate Cobra file to maintain this baseline. Since there is no MR on
this subset of the project, the Cobra project file - which will be uploaded to PARS-II - should
consider not have the change log turned on. The DOE-FPD will be informed of any changes
processed via the project’s monthly report and, accordingly, will be able to reflect the changed
contingency. How PARS-II data will be uploaded and contingency maintained for CD-3a should be
discussed and resolved with DOE FPD.
● Since CD-3a baseline will only be increased/decreased via use of contingency, the project needs to
resolve with DOE FPD how this will affect BCR approval thresholds.
● On the BCR Implementation History Unchanged Validation report, it would be beneficial to have
the current period highlighted and a note added to reflect authorization of any current period change.
Recommendations
1. Review issues identified with the WADs and correct prior to implementing baseline.
2. Apply lessons learned identified during use of the preliminary baseline within the CD-3a scope.
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 12 of 16
4.0 Appendices A. Charge
B. Agenda
C. Review Committee Contact List and Writing Assignments
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 13 of 16
Appendix A
Charge LBNF CD-3a EVMS Implementation Review
January 11-12, 2017
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 15 of 16
Appendix B
Agenda LBNF CD-3a EVMS Implementation Review
January 11-12, 2017
Closeout Presentation
LBNF CD-3a EVMS Implementation Review January 11-12, 2017
Page 16 of 16
Appendix C
Review Committee Contact List and Writing Assignments LBNF CD-3a EVMS Implementation Review
January 11-12, 2017
EVMS Assignments
1. Organization – Jennifer Fortner
2. Planning, Scheduling, & Budgeting – Sally Anderson
3. Analysis & Management Reports – Sherese Humphrey
4. Revisions – Fran Leavell
*Accounting was not covered during this review
Charge Question Assignments
Organization – 1, 2, 3, 4, 5, 6, 8
Planning, Scheduling, & Budgeting – 1, 2, 3, 4, 5, 8
Analysis & Management Reports – 1, 2, 3, 4, 5, 6, 8
Revisions – 1, 2, 3, 4, 6, 7, 8