CRWG Project PlanJune 17, 2009

1

CRWG Mission Statement

• The Cost Risk Working Group’s mission is to address the issue of cost uncertainty and risk in Life Cycle Cost Estimates by:1. Developing a mechanism for sharing ideas/information,2. Doing research on relevant topics,3. Writing guidance as needed,4. Encouraging more cost risk training throughout the Army,5. Analyzing cost uncertainty and risk within real Life-Cycle Cost

Estimates at each participating Army command for the purpose of sharing lessons learned,

6. Facilitating discussion and action amongst key Army and DoD stakeholders, and

7. Becoming a core group of focal points on cost uncertainty and risk analysis within the Army.

2

Outline1. Army Cost Uncertainty & Risk Epicenter2. Research Topics3. Appendix K4. Training5. Real Estimates/Lessons Learned6. Facilitating Discussion amongst Stakeholders7. Army Cost Risk Focal Points

Army Cost Risk Working Group Missions

Research Appendix K Training Estimates Discussions ACURE Focal Points• All CRWG Members

• All CRWG Members

• All CRWG Members

• All CRWG Members

• Completed • Arnie Horowitz• Joe Golebieski• Bill Boston

• Trevor VanAtta• Jasmine Hubbel 3

1. Army Cost Uncertainty & Risk Epicenter (ACURE)• Description:

– ACURE will be a web-based knowledge sharing system open to the entire Army cost community for storing, retrieving, and sharing any files, examples, ideas, studies, or guidance related to cost uncertainty and risk assessment.

• Basic Criteria:– AKO-based and accessible by CAC– Archive files, studies, discussion forums, etc.– Customized and easy to navigate.– Open to all Army cost personnel with upload/admin

rights limited to selected individuals.

Research Appendix K Training Estimates Discussions ACURE Focal Points4

1. Army Cost Uncertainty & Risk Epicenter (ACURE)

• Who:– Trevor VanAtta– Jasmine Hubbel

• Timeline:– July 30th: Have ACURE site ready for initial CRWG review– August 28th: Open ACURE on limited basis to train

individuals at each command to be ACURE POCs.– September 30th: Full release of ACURE, welcoming entire

Army cost community as members.

Research Appendix K Training Estimates Discussions ACURE Focal Points5

2. Research Topics

• Descriptions:– Deciding a Level of Confidence: Research on this topic will

examine the positives and negatives of selecting various levels of confidence for an estimate. What are the reasons for choosing 50%, 60%, or 80%? What do the experts say and what are the other services doing? Researchers will also need to study the possibilities of funding to different confidence levels for different appropriations. Are there reasons to fund to 80% for RDT&E, then only 60% for procurement?

– Addressing Correlation: Research on this topic will examine the process of addressing correlation in an estimate, first in a general sense, then in an ACEIT-based LCCE. How does an analyst run a correlation report, recognize functional correlation, and address non-functional correlation?

Research Appendix K Training Estimates Discussions ACURE Focal Points6

2. Research Topics

• Descriptions:– Pareto/Tornado/Variance Charts in ACEIT: Research

on this topic will require finding guidance to walk analysts through the process of developing graphs that are useful in presenting cost risk to decision-makers. What information does each graph portray? How are they developed in ACEIT? What options are available in ACEIT when making each graph?

– @Risk and Crystal Ball: Research on this topic requires the procurement and testing of @Risk and Crystal Ball Software for the purpose of eventually passing along whatever information and materials are necessary to help Army Cost Analysts do Cost Uncertainty and Risk Analysis when doing their estimates in Excel. To adequately use these programs in their estimates, what materials and information do analysts need?

Research Appendix K Training Estimates Discussions ACURE Focal Points7

2. Research Topics

• Basic Criteria:– Research for each topic will be an attempt to find

direct and concise guidance that adequately answers the relevant questions. (See Research Topics Descriptions of previous slides.)

– If research on any given topic doesn’t produce clear and concise material that could be easily understood by the average cost analyst, then CRWG can decide that a paper or briefing be done on the topic.

– The SAG Chair will coordinate with SAG members to oversee the development process and ensure that all relevant questions are addressed (see topic descriptions on previous slides).

Research Appendix K Training Estimates Discussions ACURE Focal Points8

2. Research Topics

• Who:– Deciding a Level of Confidence

• CRWG: Joe Golebieski and Arnie Horowitz• SAG Member: Melissa Potter

– Addressing Correlation• CRWG: Ray Kleinberg and Craig Penny• SAG Member: Dave Holm

– Pareto/Tornado/Variance Charts in ACEIT• CRWG: Angela Lemke and Jared Clonts• SAG Member: Claudia Rhen

– @Risk and Crystal Ball• CRWG: Troy Wilke and Keonna Andrews• SAG Member: John Conolly

Research Appendix K Training Estimates Discussions ACURE Focal Points9

2. Research Topics

• Timeline:– June 30th: CRWG will decide what further work

(papers, briefings, etc.) is needed on any topics– July 19th: Researchers submit plans/outlines of

further work to be done on their specific topic to the CRWG Chair and SAG Members assigned to their Research Topic

– Final materials and information on each topic to be completed and assembled by September 25th, and uploaded onto ACURE the following week.

Research Appendix K Training Estimates Discussions ACURE Focal Points10

3. Appendix K

• Final version of Appendix K has been completed and approved by DASA-CE. It will eventually be posted as Section 12 of the 2009 Army Cost Analysis Manual

Research Appendix K Training Estimates Discussions ACURE Focal Points11

4. Training

• Description:– CRWG Members at each command should encourage the

expansion of Cost Risk training to their respective Cost Chief by providing course descriptions and assessing the level of Cost Risk training at their command. Who has been trained, and in what courses? What courses should your Cost Chief consider to add the most value?

• Relevant Courses:– BCF 206 (DAU), Risk 102 (Tecolote), & Risk 201 (Tecolote)

Research Appendix K Training Estimates Discussions ACURE Focal Points12

5. Real Estimates & Lessons Learned

• Description:– Analysts in the CRWG at each command should find Life-Cycle Cost

Estimates to begin analyzing risk and uncertainty. These estimates will act as early trials done by the CRWG to create reports and/or briefings on lessons learned that can be uploaded in ACURE. Also, this effort will require gathering up example files already in ACEIT that were done by Tecolote.

• Basic Criteria:– Estimates must be done on real systems.– Documents loaded into ACURE will primarily be briefings or reports on

lessons learned.

Research Appendix K Training Estimates Discussions ACURE Focal Points13

• Who:– CECOM – C4ISR Systems:• Joe Golebieski and Arnie Horowitz

– TACOM – Wheeled and Tracked Combat Vehicles:• Jasmine Hubbel, Trevor VanAtta, Ray Kleinberg, Craig

Penny– AMCOM – Helicopters, Missiles, & UAVs:• Jared Clonts and Angela Lemke

– Example Files already in ACEIT:• Dave Balfour and Bill Boston

Research Appendix K Training Estimates Discussions ACURE Focal Points14

5. Real Estimates & Lessons Learned

• Timeline:– June 30th: Teams will report to CRWG on their

progress via monthly teleconference (and at any subsequent monthly teleconferences)

– September 18th: All relevant files will be submitted to CRWG Chair for upload into ACURE. Any files generated after September 18th can be uploaded at time of completion.

Research Appendix K Training Estimates Discussions ACURE Focal Points15

5. Real Estimates & Lessons Learned

6. Facilitating Discussion amongst Army and DoD Stakeholders• Description:– Participating CRWG members need to determine what

topics need to be discussed amongst key stakeholders, who those stakeholders are, when, where and how to facilitate discussions, and finally how best to record or capture the key points of those discussions.

• Who:– Joe Golebieski, Arnie Horowitz, and Bill Boston

• Timeline:– Immediate and Ongoing

Research Appendix K Training Estimates Discussions ACURE Focal Points16

7. Army Cost Risk Focal Points• CRWG Members at each command will become “Focal Points”

on Cost Uncertainty and Risk Analysis. As Focal Points, members will be expected to perform the following roles and responsibilities:

1. Field any questions about Cost Uncertainty and Risk Analysis within their command, relaying back more difficult questions to the CRWG.

2. Find small Life-Cycle Cost Estimates being developed at their command to perform early trials of Cost Uncertainty and Risk Analysis.

3. Encourage analysts within their command to perform Cost Uncertainty & Risk Analysis within their Life-Cycle Cost Estimates by advocating its benefits over standard point estimates.

4. Give briefings, advice, guidance, and other support as necessary to analysts performing Cost Uncertainty and Risk Analysis within their command.

5. Upload useful files and information into ACURE (see slides 4-5) to share with other commands.

Research Appendix K Training Estimates Discussions ACURE Focal Points17