2010-08-pmo in a box conops(2010.12.08 )

PMO-in- Box

A Proposed Formula for Program Office Staffing During the Lifecycle of an IT Project

LEONARD H. MOORE

1

Table of ContentsINTRODUCTION.................................................................................................................................................5

PROGRAM MANAGEMENT ISSUES.......................................................................................................................7

PROGRAM LIFECYCLE.........................................................................................................................................9

PROJECT PHASES............................................................................................................................................10

CONCEPT DEVELOPMENT DECISION.................................................................................................................10

INITIAL CONCEPT STUDIES................................................................................................................................12

Organization Chart During Initial Concept Studies....................................................................................14

Program Management Office Roles Staffed During Initial Concept Studies..............................................15

Program Manager......................................................................................................................................................15

Financial Manager.....................................................................................................................................................17

Earned Value Management (EVM) Monitor.............................................................................................................18

Expenditures Monitor................................................................................................................................................18

Cost Analyst..............................................................................................................................................................19

Administrative Director.............................................................................................................................................20

Contracting Officer Technical Representative (COTR)............................................................................................20

Oversight / Planning Officer.....................................................................................................................................21

Legal Officer.............................................................................................................................................................22

Systems Engineer......................................................................................................................................................23

Systems Architect......................................................................................................................................................24

Requirements Engineer.............................................................................................................................................25

Quality Assurance Engineer......................................................................................................................................26

2

Security Engineer......................................................................................................................................................30

Scheduling Engineer..................................................................................................................................................31

Test Engineer.............................................................................................................................................................32

Logistics Engineer.....................................................................................................................................................33

Program Management Office Cost During Initial Concept Studies............................................................33

Deliverables for Milestone A – entry point for Phase A: Concept Refinement –Technology Maturity Demonstration.............................................................................................................................................35

PHASE A: CONCEPT REFINEMENT AND TECHNOLOGY MATURITY DEMONSTRATION......................................38

Organization Chart During Phase A - Concept Refinement and Technology Maturity Demonstration.....39

Program Management Office Roles Staffed During Phase A - Concept Refinement and Technology Maturity Demonstration..............................................................................................................................40

Administrative Director.............................................................................................................................................40

Communications Manager........................................................................................................................................40

Risk Manager............................................................................................................................................................41

Configuration Management Specialist......................................................................................................................42

Software Engineer.....................................................................................................................................................43

Program Management Office Cost During Phase A: Concept Refinement and Technology Demonstration.....................................................................................................................................................................43

Deliverables for Milestone B-entry point for Phase B: Development, Integration, and Demonstration....44

PHASE B: DEVELOPMENT, INTEGRATION, AND DEMONSTRATION....................................................................49

Organization Chart During Phase B: Development, Integration, and Demonstration...............................50

Program Management Office Roles Staffed During Phase B: Development, Integration, and Demonstration.............................................................................................................................................51

Change Management Manager..................................................................................................................................51

Training Curriculum Developers and Trainers:........................................................................................................51

3

Scrum Team Lead.....................................................................................................................................................52

Interface Development Lead.....................................................................................................................................56

Production Database Development Lead..................................................................................................................56

Assistants to the Various Engineers and Other Leads:..............................................................................................57

Program Management Office Cost During Phase B- Development, Integration, and Demonstration.......59

DELIVERABLES FOR MILESTONE C – ENTRY POINT FOR PHASE C: PRODUCTION / SUSTAINMENT.................59

PHASE C: PRODUCTION/SUSTAINMENT.............................................................................................................64

Organization Chart During C: Production/Sustainment.............................................................................64

Program Management Office Roles Staffed During Production/Sustainment............................................65

Existing Rollout Discrepancy /Request for Change Lead.........................................................................................65

Site Installation Engineer..........................................................................................................................................65

Program Management Office Cost During Phase C- Production/Sustainment..........................................65

FUTURE DEVELOPMENTS............................................................................................................................72

BIBLIOGRAPHY...............................................................................................................................................74

4

Introduction

Over the past several years, many requests have come from both the government and Federally Funded Research Centers (FFRDCs) for a “recipe” on how to build a project management office (PMO). Most responses to such requests from management experts have consisted of a few lines of text followed by the words “it depends.” Unfortunately, the requests have increased in frequency and stridency.

A response requires a number of possible permutations based on the projects complexity, scope, and size. Building ships, satellites, jet fighters, office buildings, and software solutions will always entail different timelines, job roles, and budgets.

Variations also depend on how the commercial, Department of Defense (DoD), or Intelligence Community (IC) element (e.g. NRO, NGA, CIA, and NSA.) envisions PMOs to work within their organizational structure. For example, some organizations prefer to have a huge and centralized PMO structure with program managers and other staff familiar with standard program management processes. These program management “ninjas” can be assigned as needed to the organization’s projects. These PMOs also tend to provide centralized training, program management tools, and advice to the remainder of the organization.

Other organizations use PMOs as a knowledge management entity, providing standard templates and methodologies, while capturing lessons learned. However, these entities do not participate in the normal day-to-day management of the program office.

Another permutation is the use a standard structure where a PMO is stood up, reporting to a major functional department or a CIO, and the PMO is equipped with jobs and roles designed to perform detailed fulfillment of the panoply of program management activities. In these agencies, there is rarely a centralized PMO that can be used to supply significant resources or significantly more advanced expertise.

The purpose of this paper is to outline a recommended PMO structure based on the hypothetical scenario an information technology (IT) program that involves the analysis of intelligence data, gathering information from multiple databases in multiple agencies. It is specifically patterned for an IC element. However, many of the IC phases and control gates can be translated to DoD or commercial methodologies. This hypothetical project may also gather personal information that may result in privacy concerns. It is estimated that the solution will reside on the primary intelligence community classified network and the various internal networks of the Intelligence Community (IC) elements. It is envisioned that the solution will have approximately 20,000 users worldwide. The

5

study links program office roles to expected deliverables and required activities for this type of IT project. The program is expected to take six years to develop and deploy and go into the operations and maintenance phase (project conclusion) the seventh year. It is estimated to cost a total of $130 million to deliver in 2010 dollars. This scenario uses the Intelligence Community Acquisition Model (ICAM). The proposed structure assumes that the exploratory efforts before Phase A and Phase B in the ICAM framework will take two years. A two year period prior to Milestone B may seem long, but the rush to Milestone B, which many programs attempt, often results in cost, schedule, and performance breaches. An assumption is that many programs do not spend enough time doing upfront systems engineering and mitigating technology risks.1It assumes that Phase B (Development, Integration, and Demonstration), will take one year. Phase C, prior to slipping into O&M, is assumed to take place over three years.

This study addresses program office overhead as well as design, development, and deployment costs. A proposed approach for integrating agile development methodologies is also presented. The analysis includes what positions should be staffed at each phase of the process, and the deliverables that staff members are responsible for completing. The activities and deliverables are structured around Intelligence Community Directive (ICD) 801 and Intelligence Community Policy Guidance (ICPG 801.1). The current ICD 801 and ICPG 801.1 were formerly known as ICD 105and ICPG 105.1. These two documents describe the Intelligence Community Acquisition Model (ICAM).

The approach designed herein supports a model where the government serves as the lead systems integrator and where implementation staffs are acquired through various contracting instruments. However, it could easily support a model in which one lead systems integrator wins a contract to build the system. In the second case, the PMO consists of government staff providing oversight. Many of the jobs and roles discussed in this study could be transitioned from government staff to contractors.

The model provides a flexible template around which a new project can remove or add positions deemed more or less important. Best practice staffing practices are used. The study also discusses “bad practices” that can often develop. For example, many programs make the error of staffing up the implementation team before they have done up front systems engineering. The author has been on projects where implementation teams are developing software systems without any overall architecture or requirements definition. The software designed in these projects had to be thrown out because it conflicted with the architecture and requirements, once they became defined. 2

1 2010 Program Management Plan Submission to Congress, Office of the Director of National Intelligence (ODNI)/Senior Acquisition Executive, Appendix D.

6

Program Management Issues

The term “PM” means many different things to different people. In the Department of Defense (DoD), program management is used to describe the entire spectrum of activities necessary to deliver a product or functionality, including what many would call systems engineering. In the commercial world, it has tended to have a more narrow definition, which does not include the pure technical and engineering aspects incorporated in the DoD definition.

The Program Management Institute (PMI) is one of the organizations using a slightly different definition. It defines a Project as a temporary endeavor undertaken to create a unique product, service, or result. The temporary nature of projects indicates a definite beginning and end. The end is reached when the project’s objectives have been achieved or when the project is terminated, because its objectives will not or cannot be met, or when the need for the project no longer exists. Temporary does not mean short in duration. Temporary does not apply to the product, service, or result created by the project; most projects are undertaken to create a lasting outcome.3

The institute defines a Program as a group of related projects managed in a coordinated way to obtain benefits and control not available from managing them individually. Programs may include elements of related work (e.g. ongoing operations) outside the scope of the discrete projects in a program.4

Program Management is defined by PMI as the centralized coordinated management of a program to achieve the program’s strategic objectives and benefits. Projects within a

2 In this example, an IC program expended $250 million over a four year period using an implementation team to develop a solution, only to have the bulk of their efforts prove useless as an architecture was never developed, or even a preliminary design review of the system held.

3 Project Management Body of Knowledge (PMBOK), 4th edition, Project Management Institute (PMI), Newtown Square, PA, 2008, p. 5.


7

program are related through the common outcome or collective capability. If the relationship between projects is only that of a shared client, seller, technology, or resource, the effort should be managed as a portfolio of projects, rather than as a program. Program management focuses on the project interdependencies and helps to determine the optimal approach for managing them. Actions related to these interdependencies may include:

resolving resource constraints and/or conflicts that affect multiple projects within the program.

aligning organizational/strategic direction that affects project and program goals and objectives; and

resolving issues and change management within a shared governance structure.

A Program Management Office can be defined as an organized body or entity assigned various responsibilities related to the centralized and coordinated management of those projects under its domain. 5An example of a program would be a new communications satellite system with projects for design of the satellite “bus”, design and production of communications and multiple sensor packages that go on the bus, design and construction of ground stations, integration of the system, and launch of the satellite.

Many program management professionals have asked why there is no “practice standard” on how to stand up a project or program office. PMI and the Defense Acquisition University (DAU) generally avoid establishing “practice standards” if there is no agreement on what the standard should be. Because of the great variety found among PMOs in various organizations, discussions on this topic tend to be characterized by diversity of opinion and confusion. There is no “standard” on PMO design, because there is no agreement among qualified practitioners. 6

Many of the benchmarks created for project offices and project management processes in general are based on commercial paradigms. Government projects have unique requirements relative to the private sector. Legal constraints are more stringent and burdensome. In the private sector, project managers are accountable to the immediate client and a limited number of stakeholders, such as shareholders and employees. In government, the project manager is accountable to a much larger broader group, including legislators, government executives, and the general public. Government

5 Hobbs, Brian, Aubrey, Monique, The Project Management Office: A Quest for Understanding, Project Management Institute (PMI), Newtown Square, PA, 2010, p. 1.


8

projects normally use money that is appropriated. The process for obtaining this money is complex. Using the money can involve complex processes of commitment, obligation, and expense. Oversight can be a burdensome process that is not present in the private sector. To proceed beyond these constraints, more approvals are required from more organizations. 7

Jointly managed program offices, where programs transcend multiple government agencies, involve even more complexity. Joint program offices have more customers and more complicated requirements and must plan on having positions vacant for 9-12 months or more when assigned personnel rotate out or retire. There is no pool or established pipeline of trained or experienced acquisition personnel readily available, such as there is at a product center. They must be individually sourced from the agencies, which are always reluctant to part with their people, especially their talented ones.8

Program Lifecycle

The majority of programs in government and larger commercial companies are executed using a program lifecycle. The lifecycle model employed must be tailored to accommodate the unique characteristics of the program. With such planning, the work can be broken down into a series of efforts and an orderly sequence of execution steps can evolve. Without this planning, execution is likely to be haphazard and disorderly. In the case of this study, the Intelligence Community Acquisition Model (ICAM) model will be used to describe the program lifecycle. An outline of the model follows.

7 Government Extension to the PMBOK Guide, 3rd edition, Project Management Institute (PMI), Newtown Square, PA, 2008, p. 5.

8 Quicklook Handbook, OSD-C3I, p. 1.

9

Project Phases

Concept Development Decision

The first step in the ICAM model determines whether the need or idea that has been formulated merits investigation to better flesh out the project. This decision to further investigate and research options does not commit to a formal project. However, it determines whether sufficient information exists regarding requirements, and possible solutions.

11

Intelligence Community Acquisition Model

The following tables show the types of deliverables required for pre-Milestone A and the Concept Development Decision, including the individuals responsible for delivering them:

Legend P= principal drafterC= contributor

Blue denotes gov't staffPurple denotes direct report to PM

Initial WBS Structure

Compelling Intelligence Need

(CIN) Initial LCC Estimates Agency Cost PositionProgram Manager C C C CFinancial Manager C P P

EVM Monitor C C CExpenditures Monitor C CCost Analyst C C C

Admin Director C C CCOTR C COversight/Planning Officer C C CLegal Officer

Systems Engineer CSystems Architect C C CRequirements Engineer C P C CSecurity Engineer C C CScheduling Engineer P C CTest Engineer C C CLogistics Engineer C C C

Contributor



Intelligence Capability Baseline Descriptions (ICBD)

Initial CBJB Submission/

Reprogramming Document SOC-A

Technology Development

StrategyProgram Manager C C CFinancial Manager P P

EVM Monitor C CExpenditures Monitor C CCost Analyst C C

Admin Director C C C CCOTR C COversight/Planning Officer C C C CLegal Officer C C

Systems Engineer C C PSystems Architect C C CRequirements Engineer P C CSecurity Engineer C C CScheduling Engineer C C C CTest Engineer C C CLogistics Engineer C C C

Contributor

12



CONOPS System ArchitectureInitial Program

Management PlanSecurity Classification

GuideProgram Manager P C P CFinancial Manager C C


Admin Director C C C CCOTR C COversight/Planning Officer C C C CLegal Officer C C

Systems Engineer C C C CSystems Architect C P CRequirements Engineer C C CSecurity Engineer C C C PScheduling Engineer C CTest Engineer C CLogistics Engineer C C C

Contributor

At the earliest stage of the program, the PMO staff members remain full time members of line organizations or other programs9. After approval, the program enters the next stage, Initial Concept Studies, and the PMO begins to acquire a permanent staff.

Initial Concept Studies

The purpose of the Initial Concept Studies period is to determine whether, from a strategic perspective, a full scale acquisition development program is required to address a capability gap/shortfall. Efforts during this phase should focus on identifying a range of approaches to address the identified need, including high-risk solutions and the effects of maintaining the status quo. Staffed skilled in specific software products should not be brought onboard, as the solution may not use their products.

If the program is a Major Systems Acquisition (MSA), and will be entirely National Intelligence Program (NIP) funded, the National Intelligence Acquisition Board (NIAB) will serve as the Milestone Decision Authority, permitting the project to move through the control gates of each phase. When funding is shared between DoD and the IC, by dint of the Memorandum of Agreement (MOA) signed in 2008 between the Director of National Intelligence (DNI) and the Secretary of Defense, ICPG 801.1 will apply, and a

9 Hill Gerald, The Complete Project Management Office Handbook, CRC Press, 2007, pp. 210-213.

13

Joint Intelligence Acquisition Board (JIAB) will govern the milestone decisions of the project.

A key to the healthy growth of a PMO are the relationships within that organization. Al Grasso, the President of MITRE, stated in the previously mentioned Senate testimony that, “successful programs are characterized by a strong government PM office—or PMO—capable of a peer relationship with the contractor(s) on systems engineering and PM issues.” The metaphor Grasso employs is “an Architect’s relationships with the user and the builder of a building.” He says that “the architect works to understand the user’s operational needs and translate them into technical requirements enabling builders to develop the needed capability. The Architect evaluates development feasibility and performs an independent conceptual design and cost estimate. These Architect functions enable the user to make informed cost and capability tradeoffs and prioritize requirements. The Architect is accountable to the user to ensure that the delivered capability meets the user’s highest-priority needs within the constraints imposed by available technology, funding and time.” The PMO starts performing this role during the Initial Concept Studies phase. 10 During the Initial Concept Studies phase, 70-75% of the costs decisions are made. 11

10 Grasso, Al, U.S. Committee on Homeland Security, Subcommittee on Federal Financial Management, Government Information, Federal Services, and International Security, 2009.

11 Kelley, Mike, Kepner, Robert, Achieving Effective Results in Acquisition, The MITRE Corporation, January 2009, p. 12.

14

Organization Chart During Initial Concept Studies

Program Manager

Financial Manager

Oversight/Plans Officer

COTR

Legal Officer

Systems Engineer

Systems Architect

Requirements Engineer

SecurityEngineer

EVM Monitor

Expenditures Monitor

SchedulingEngineer

Test Engineer

Logistics Engineer

Cost Analyst

Pre-Phase A =PM deputies=PM

=government staff/ FFRDC

15

Program Management Office Roles Staffed During Initial Concept Studies

Program Manager

The first two positions that should be staffed are the Program or Project Manager and the System Engineer. For the purpose of our study, the terms “program manager” and “project manager” will be assumed to be synonymous conforming to the usage of the government. The program manager is responsible, on behalf of the executive level sponsor, for successful delivery of a new capability or desired outcome. The role requires the effective coordination of the program and its projects and their interdependencies, and any risks and other issues that may arise. As the program is implemented, changes to policy, strategy, or infrastructure may have an impact across or outside of the program. The program manager is responsible for the overall integrity and coherence of the program, and develops and maintains the program environment to support each individual project within it. The Program Manager (PM) has the responsibility and authority to accomplish program objectives for development, production, and sustainment to meet the user's operational needs. The PM shall be accountable for credible cost, schedule, and performance reporting to the Milestone Decision Authority (MDA).

The effective program manager should have the "big picture" perspective of the program, including in-depth knowledge of the interrelationships among its elements. An effective program manager:

is a leader and a manager, not primarily a task "doer";

understands the requirements, environmental factors, organizations, activities, constraints, risks, and motivations impacting the program;

knows and is capable of working within the established framework, managerial systems, and processes that provide funding and other decisions for the program to proceed;

comprehends and puts to use the basic skills of management-planning, organizing, staffing, leading, and controlling that assure people and systems harmonize to produce the desired results;

coordinates the work of defense industry contractors, consultants, in-house engineers and logisticians, contracting officers, and others, whether assigned

16

directly to the program office or supporting it through some form of integrated product team or matrix support arrangement;

builds support for the program and monitors reactions and perceptions that help or impede progress; and

serves both the needs of the user in the field and the priority and funding constraints imposed by managers in the Office of the Director of National Intelligence (ODNI) and the parent agency. 12

As an expert, the Program Manager will have knowledge and understanding of:

effective leadership, interpersonal and communication skills; techniques for planning, monitoring and controlling programs; project management approaches; budgeting and resource allocation procedures; problem solving techniques; change management techniques; stakeholder management and communications; risk management and issue resolution; program planning and control; program management tools; business case management; quality management.

As an expert, the PM will be able to:

manage the program’s budget , monitoring expenditures and costs against benefits;

facilitate the appointment of individuals to the program and project teams; manage third-party contributors; manage stakeholder communications; manage the dependencies and interfaces between projects; manage risks to the program’s successful outcome; report progress of the program at regular intervals to leadership and oversight; critically evaluate and select the most appropriate approach and techniques for

each specific program;

12 Defense Acquisition University, Acquipedia, Definition of a Program Manager.

17

identify suitable monitoring techniques for a program and explain how to implement them;

identify a suitable information flow and reporting process for a specific program; create and complete progress and variance reports for a specific program; specify and manage suitable transition management activities for a specific

program; specify and deliver program benefits; develop the Acquisition Strategy, which is far more than a contracting strategy.

During the initial phases of the program, the PM is focused on start-up activities that identify the program’s mission, purpose, organization, and advocacy on which the program will be established. The PM owns the program outcome, not the contractor. The PMO staff must be at least peer equivalents to the contractor’s team leads.

If a program charter does not exist, then the PM should draft one indicating the PM’s chain of command and authorities. It should include governance, responsibilities, and reporting requirements. 13It establishes a partnership between the performing organization and the requesting organization. 14

PM longevity is often a problem. PM’s who lead the design phases, and then leave, often leave their successors with a litany of problems that the initial PM chose to avoid. The PM must be kept as long as possible, at least through Milestone C. 15

Financial Manager

The Financial Manager is responsible for the formulation, submission, and expenditure tracking of the program office budget. The Financial manager submits budget



15 DoDI 5000.02, December 2008, Enclosure 10.

18

justifications to the Intelligence Planning, Programming, Budgeting, and Evaluation System (IPPBE). The Financial Manager will work with the Requirements Engineer to ensure that budget submissions, including Congressional Budget Justification Book (CBJB) inputs, are in concert with the Intelligence Community Capability Requirements (ICCR) process.

The Financial Manager’s duties include:

working with the Scheduling Engineer to ensure that every element of the WBS is supported by resources and that every resource dollar is associated with a WBS element;

developing requests for funding in accordance with the ODNI integrated report request process;

developing and maintaining cost and budget documents that monitor the execution of current year funds;

providing inputs to portfolio management processes; formulating the program cost position as a counterpoint to the independent cost

estimate (ICE), which is normally performed by an FFRDC or the DNI Cost Analysis Improvement Group (CAIG);

providing financial reports, presentations, and metrics as required.

Because no program, even at the earliest stages, can function without budgetary authority, the Financial Manager is a must fill billet at one Full Time Equivalent (FTE) during pre-Milestone A activities.

Earned Value Management (EVM) Monitor

The EVM monitor reports to the Financial Manager, but works closely with the Scheduling Engineer. This individual is responsible for EVM metrics. During pre-Milestone A activities, because the Work Breakdown Structure (WBS) work packages being executed are not numerous, only .5 FTE will be required for this role.


The Expenditures Monitor reports to the Financial Manager, but works closely with the Scheduling Engineer. This individual is responsible for issuing funding documents as required to cover the cost of program resources. The Expenditures Monitor also

19

documents commitment, obligation, and expenditure rates for budget line items supporting the programs. The Expenditures Monitor maintains and reconciles Budget Line Item (BLI) documents. The Expenditures Monitor supports the EVM reporting efforts of the Scheduling Engineer and the EVM monitor.

During pre-Milestone A activities, the expenditures that are being executed are probably not numerous, and so only .5 FTE will be required for this role. The roles of the EVM Monitor and Expenditures Monitor could be shared by one individual. However, these roles will require one full time FTE during subsequent phases.

Cost Analyst

The Cost Analyst is responsible for reviewing the program’s budget and scope to ensure it is adequately resourced. This person may also review proposals and estimates for contracts. Cost estimating and analysis is a profession that combines the understanding of math and statistics principles with engineering, program management, procurement, budgeting, and accounting disciplines. The mastery of cost estimating and analysis requires a solid foundation of cost knowledge woven together with a variety of cost experiences. The Cost Analyst’s duties include:

creating life cycle cost estimates; conducting cost/benefit analyses; preparing business case analyses and analyses of alternatives; creating budget estimates; preparing or reviewing complex cost reports and cost allocation plan; reviewing financial documents prepared by others for accuracy, form, and content; interpreting and resolving audit findings; reviewing, advising, and conducting cost, schedule, and contract performance. performing independent assessment and validation of existing, mission-generated

cost estimates for spacecraft missions. assisting with the development of cost controls, procedures, systems, and forecasting

techniques to evaluate contract or program status, ensuring compliance with government and customer requirements.

providing variance and risk analysis and other economic studies. assisting with the development of quantitative cost models that provide effective and

insightful analysis. advising internal fiscal units on cost accounting procedures and practices. evaluating work process, organizational systems, policies, procedures, and computer

technology in order to establish cost accounting data. developing and validating the program cost estimate and compares it with that of the

DNI CAIG.

20

The Cost Analyst should be staffed at the initiation of the program and should report to the Financial Manager. The Cost Analyst should remain on the project until the PMO has been dissolved. Toward the later part of Phase C, the role can be diminished and merged with that of the Expenditures Monitor.

Administrative Director

The Administrative Director supervises the legal, contracting, oversight, communications, and risk processes. During the pre-Milestone A activities, these roles can initially be shared by one or two people, such as the Oversight/ Planning Officer.

Contracting Officer Technical Representative (COTR)

The COTR, who works with a contracting officer (CO), is the program’s link with the contract creation and execution process. During the pre-Milestone A activities, the only contracting that may be required would include hiring a small number of people or leasing office space. As a result, a full-time COTR would not be required. However, the need for a full-time COTR will grow in subsequent phases of the contract.

Duties of the COTR include:

documenting project purchasing decisions, specifying the acquisition approach, and identifying potential contractors;

drafting the request for proposals; obtaining contractor responses, selecting a contractor, and awarding a contract; managing procurement relationships, monitoring contract performance; and

making changes and corrections as needed. 16

Oversight / Planning Officer

The Oversight/ Planning Officer is responsible for the required reporting to oversight authorities. This position also functions as an acquisition chief, providing direction and


21

guidance to the PM for purchasing products and/or services. This includes conducting market research and analysis for best pricing, adhering to budgetary guidelines, and reporting research and pricing to management to assist in making acquisition decisions. This person may also prepare source selection documents and participate in source selection. This individual requires a solid understanding of program governance, engineering, program management, and program requirements, as well as contracting, budgeting and financing disciplines. This individual’s duties include:

presenting quarterly program reviews that will be required for the Office of the Director of National Intelligence (ODNI)/ Senior Acquisition Executive (SAE), if the program is designated as a Major Systems Acquisition (MSA);

preparing for mid-year reviews by the ODNI/SAE; reviewing the Intelligence Community Program Management Report (PMP),

which is sent to Congress; collecting and submitting program metrics to support the ODNI/SAE snapshot

process; assembling Exhibit 300 documentation if required; developing program reviews and presentations for higher authority. advising the PM on acquisition matters; preparing the Program Manager's Charter; preparing the Program Management Plan; preparing the Single Acquisition Management Plan; preparing proposal and source selection documents; participating in source selections; coordinating on acquisition documents prepared by others for accuracy, form and

content; shepherding acquisition documents through the review and approval process; reviewing and advising on cost, schedule, and system performance data; advising internal units on acquisition procedures, practices and requirements; participating in milestone decision reviews.

This position captures much of the unique government oversight reporting that is not present in commercial industry. It must be staffed at a full FTE at the beginning of pre-Milestone A activities. This individual acts as the Administrative Director and Communications Director, until the beginning of Phase A.

22

Although some might contend that detailing specific individuals to deal with oversight activities is excessive, what usually happens on government projects is that the engineering staff gets pulled away from engineering functions and is compelled to assemble oversight documents. As a result, systems engineering tasks are sacrificed to comply with government oversight requirements. The staffing of this position helps to mitigate against that problem.

Legal Officer

Any IC system that seeks to capture personal information through accessing the databases of multiple agencies can run afoul of a nexus of privacy issues as well as the desire of the agencies to protect their data. The Markle Foundation studies that were published after the attacks of 9/11 focused on the need for privacy concerns in designing better approaches to combat similar attacks. They also focused on the legal interests of the various agencies that are a part of the Intelligence Community, and how these rules and equities can interfere with information sharing. The Legal Officer will provide advice on:

privacy concerns with regard to what data can be handled and how; 17

legal authorities of agencies to resist common access to their data; 18

contracting processes in order to avoid protests; key deliverables with legal implications.

Although not requiring a full FTE during pre-phase A activities, this individual becomes more critical during Phase A and B. Privacy concerns have derailed several projects in the Department of Homeland Security, such as CAPPS II19 and its successor Secure

17 Markle Foundation, “Protecting America’s Freedom in the Information Age”, October 2002, p. 31.

18 Markle Foundation, “Creating a Trusted Network for Homeland Security”, December 2003, p. 30.

19 “The Five Problems with CAPPS II”, American Civil Liberties Union, August 25, 2003.

23

Flight. 20 The Carnegie Foundation estimated that the federal government has spent over $500 million as of 2006 on projects that had to later be cancelled because of privacy concerns. 21

Systems Engineer

After the program manager, the second most important position, after the program manager, is the Systems Engineer (SE). This position is sometimes known as a Chief Engineer (CE). Upfront systems engineering is absolutely critical to the success of any program.22 The SE is the principal design architect of the project. Without a system engineering strategy and design, the PM is blind about what is being developed. Systems Engineering can do more than build a system correctly; it can provide the processes and mechanisms to ensure that the right system is built. Systems engineering is far more than requirements management. 23

As an expert, the Systems Engineer will:

manage enterprise architecture and alignment; control the requirements process; develop a technological maturity management approach; maintain a risk management and quality management approach; maintain an enterprise level program schedule; manage program technical and data standards; develop project security architectures; map out interfaces with other systems; provide configuration management;

20 Testimony by Kip Hawley, Assistant Secretary of the Transportation Security Administration, before the United States Senate Committee on Commerce, Science, and Transportation, February 9, 2006.

21 Spadunata, Laura, “The Privacy Problem: Homeland Security Failures”, News 21: A Journalism Initiative of the Carnegie and Knight Foundations, September 1, 2006.

22 “Pre Milestone A and Early-Phase Systems Engineering: A Retrospective View and Benefits for Future Air Force Acquisition”, National Resource Council, 2008.

23 Kelley, Mike, Kepner, Robert, Achieving Effective Results in Acquisition, the MITRE Corporation, January 2009, p. 27.

24

perform systems modeling and simulation; oversee engineering processes; develop new software; use acquisition strategies, such as competitive prototyping, before determining a

final solution; 24

execute a discrepancy correction and request for change process.

The SE is focused on engineering activities required to develop, analyze, and mature the concept in the lead up to Milestone A as well as during Phase A. During Phase B, the systems engineer’s focus shifts to the production of knowledge and infrastructure necessary to conduct the end-item acquisition, development, and fielding efforts. During Phase C, the SE is focused on transitioning legacy systems into new capabilities. As with the PM, the SE should be on the PMO staff as long as possible.

Systems Architect

Simultaneous with the staffing of the Systems Engineer, a Systems Architect should be employed full time from the inception of the project. The systems architect, who reports to the Systems Engineer, establishes the basic structure of the capability being implemented by the project, defining the essential core design features and elements that provide the framework for all that follows; these design features are the hardest to change later. The systems architect provides the engineering view of the users' vision for what the capability needs to be and do, and the paths along which it must be able to evolve, and strives to maintain the integrity of that vision as it evolves during detailed design and implementation.

As an expert, the Systems Architect will:

interface with the users, sponsors and all other stakeholders in order to determine their evolving needs;

generate the highest level of system requirements, based on the user's needs and other constraints such as cost and schedule;

ensure that this set of high level requirements is consistent, complete, correct, and defined by performance metrics;

24 Directive Type Memorandum (DTM) 09-021. Implementation of the Weapons Systems Acquisition Reform Act of 2009, Attachment 1, p. 5.

25

perform cost-benefit analyses to determine whether requirements are best met by manual, software, or hardware functions; making maximum use of commercial off-the-shelf or already developed components and processes;

partition large systems into subsystems and components, each of which can be handled by a single engineer or team of engineers or subordinate architect;

work with the design and implementation engineers, or subordinate architects, so that any problems arising during design or implementation can be resolved in accordance with the fundamental architectural concepts, and user needs and constraints;

create a set of acceptance test requirements, together with the designers, test engineers, and the user, which determine that all of the high level requirements have been met, especially for the computer-human-interface.

generate products such as sketches, models, an early user guide, and prototypes to keep the user and the engineers constantly up to date and in agreement on the system to be provided as it is evolving;

ensure that all architectural products and products with architectural input are maintained in the most current state and never allowed to become obsolete.25


The Requirements Engineer determines the needs or conditions to be met for a new or altered product, taking account of the possibly conflicting requirements of the various stakeholders, such as beneficiaries or users. These requirements must be documented, actionable, measurable, testable, related to identified business needs or opportunities, and defined to a level of detail sufficient for system design. Requirements can be functional (defining specific results of the system) and non-functional (defining characteristics of the system, such as cost). This position is a key one, and will generally involve one full time equivalent of support at the beginning stages of defining the project.

The Requirements Engineer manages the following processes:

25 Wikipedia, “Systems Architect” article.

26

eliciting requirements: the task of communicating with customers and users to determine what their requirements are, which is sometimes also called requirements gathering;

analyzing requirements: determining whether the stated requirements are unclear, incomplete, ambiguous, or contradictory, and then resolving these issues;

recording requirements: requirements might be documented in various forms, such as natural-language documents, use cases, user stories, or process specifications.

Requirements analysis and definition can be a long and arduous process during which many delicate psychological skills are involved. New systems change the environment and relationships between people, so it is important to identify all the stakeholders, take into account all their needs and ensure they understand the implications of the new systems. The Requirements Engineer can employ several techniques to elicit the requirements from the customer. Historically, this has included such things as holding interviews, or holding focus groups (more aptly named in this context as requirements workshops) and creating requirements lists. More modern techniques include prototyping, and use cases. Where necessary, the Requirements Engineer will employ a combination of these methods to establish the exact requirements of the stakeholders, so that a system meeting the business need is produced.26

Requirements definitions are a part of good up front systems engineering. For example, many projects are defined by requirements that are too broad or unrealistic. Requirements engineers must define those requirements into more detailed ones. Inadequately defined requirements lead to misunderstandings of the capabilities desired by the project as well as insufficient testing of the software. 27

Quality Assurance Engineer

The Quality Assurance (QA) Engineer is responsible for the systematic monitoring and evaluation of the various aspects of a project, service, or facility to ensure that standards of quality are being met. These standards of quality are determined by the program sponsor. QA includes regulation of the quality of raw materials, assemblies, products and components; services related to production; and management, production and inspection

26 Wikipedia, “Requirements Engineering” article.

27 “Haas, Robert”, iFlorida Model Deployment Final Evaluation Report, U.S. Department of Transportation, Federal Highway Administration, January 2009, p. 23.

27

processes. The Quality Assurance Engineer is responsible for overall Capability Maturity Model (CMM) compliance of the project. The Quality Assurance Engineer works closely with the Requirements Engineer and the Configuration Management Specialist.

It is important to realize that quality is determined by the intended users, clients or customers, not by society in general. It is not synonymous with 'expensive' or 'high quality'. Even goods with low prices can be considered quality items if they meet a market need and meet defined performance standards. QA is more than just testing the quality of aspects of a product, service or facility; it analyzes the quality to make sure it conforms to specific requirements and complies with established plans.

The quality assurance engineer manages the following processes:

evaluating and monitoring software quality assurance activities that assure the software development and control processes described in the project's program management plan are correctly carried out and that the project's procedures and standards are followed;

monitoring configuration management (CM) activities to determine that they are being performed in accordance with CM and program management plan processes and procedures;

verifying and validating technical reviews, inspections, and walkthroughs; auditing the quality requirements and the results from quality control

measurements to ensure that appropriate quality standards and operational definitions are used;

monitoring and recording results of executing the quality assurance activities to assess performance and recommend necessary changes:

monitoring software testing, including reviewing test documentation for completeness related to the test plans, specifications, and reports. 28

During the Initial Concept Studies Phase, a full time Quality Assurance Engineer is not required. However, initial constructs need to be formulated to determine how quality assurance will take place during the developing the Program Management Plan. During later phases of the project, a full time QA Engineer will be needed.


28

In the 1980s, several US military projects involving software subcontractors ran over-budget and were completed much later than planned, if they were completed at all. In an effort to determine why this was occurring, the United States Air Force funded a study at the Software Engineering Institute (SEI) at Carnegie-Mellon University. The result of the Air Force study was a model for the military to use as an objective evaluation of software subcontractors' process capability maturity. It became known as the Capability Maturity Model (CMM). The Quality Assurance Engineer is responsible for managing the CMM process.

The Capability Maturity Model involves the following aspects:

• Maturity Levels: a 5-Level process maturity continuum, where the uppermost (5th) level is a notional ideal state where processes would be systematically managed by a combination of process optimization and continuous process improvement.

• Key Process Areas: a Key Process Area (KPA) identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important.

• Goals: the goals of a key process area summarize the states that must exist for that key process area to have been implemented in an effective and lasting way. The extent to which the goals have been accomplished is an indicator of how much capability the organization has established at that maturity level. The goals signify the scope, boundaries, and intent of each key process area.

• Common Features: common features include practices that implement and institutionalize a key process area. There are five types of common features: Commitment to Perform, Ability to Perform, Activities Performed, Measurement and Analysis, and Verifying Implementation.

• Key Practices: The key practices describe the elements of infrastructure and practice that contribute most effectively to the implementation and institutionalization of the KPAs.

There are five levels defined along the continuum of the CMM and, according to the SEI: "Predictability, effectiveness, and control of an organization's software processes are believed to improve as the organization moves up these five levels. While not rigorous, the empirical evidence to date supports this belief."

Level 1 - Initial (Chaotic)

29

It is characteristic of processes at this level that they are (typically) undocumented and in a state of dynamic change, tending to be driven in an ad hoc, uncontrolled and reactive manner by users or events. This provides a chaotic or unstable environment for the processes.

Level 2 - Repeatable

It is characteristic of processes at this level that some processes are repeatable, possibly with consistent results. Process discipline is unlikely to be rigorous, but where it exists it may help to ensure that existing processes are maintained during times of stress. This level is sometimes known as “program management basic.”

Level 3 - Defined

It is characteristic of processes at this level that there are sets of defined and documented standard processes established and subject to some degree of improvement over time. These standard processes are in place (i.e., they are the AS-IS processes) and used to establish consistency of process performance across the organization.

Level 4 - Managed

It is characteristic of processes at this level that, using process metrics, management can effectively control the AS-IS process (e.g., for software development). In particular, management can identify ways to adjust and adapt the process to particular projects without measurable losses of quality or deviations from specifications. Process Capability is established from this level.

Level 5 - Optimizing

It is a characteristic of processes at this level that the focus is on continually improving process performance through both incremental and innovative technological changes/improvements.

At maturity level 5, processes are concerned with addressing statistical common causes of process variation and changing the process (for example, to shift the mean of the process performance) to improve process performance. This would be done at the same time as maintaining the likelihood of achieving the established quantitative process-improvement objectives.

30

An program is usually not certified in CMM; instead, an organization or program is appraised. Depending on the type of appraisal, the organization can be awarded a maturity level rating (1-5) or a capability level achievement profile.

Moving a project’s CMM Level to Level 3 or above, usually requires additional resources and expense. Special training teams are usually set up to ensure the organization is compliant with the standards associated with these levels. They are most commonly referred to as Engineering Process Groups (EPGs) and Process Action Teams (PATs). This approach requires that members of the EPG and PATs be trained in CMM, that an informal CMM appraisal be performed, and that process areas be prioritized for improvement. The staffing profile envisioned in this scenario should be able to achieve Level 3, but the EPGs and PATs will need to be established to achieve higher CMM compliance.

Security Engineer

The Security Engineer develops the detailed engineering plans and designs for security features, controls and systems. These controls and systems provide confidentiality, integrity, and availability. Security engineering is similar to other systems engineering activities in that its primary motivation is to support the delivery of engineering solutions that satisfy pre-defined functional and user requirements, but with the added dimension of preventing misuse and malicious behavior.

The Security Engineer’s activities include:

securing network(s) and allied infrastructure; securing applications and databases; security testing; information systems auditing; business continuity planning; resolving competing security protocols that endanger project success; managing the information assurance (IA) and certification and accreditation

processes;29

conducting digital forensics science.


31

A full time security engineer is not required for the Initial Concept Studies phase. On a part-time basis, security engineers should be brought in from the outset for any IT system that crosses agency domains. This engineer will become more important as the solution is better defined. However, during early phases of the project, the security engineer will be able to advise the program office on which concepts have dramatic security shortcomings.

Scheduling Engineer

The Scheduling Engineer develops the work breakdown structure (WBS) and work packages for 100% of the work of the project. The WBS is expected to evolve and become more detailed as the project approaches Milestone B. The WBS articulates the project scope. WBS descriptions tell what capabilities will be produced, not the process to produce it. The Scheduling Engineer is responsible for the project schedule and GANT charts. The Scheduling Engineer works with the Earned Value Monitor (described earlier) to develop the project’s EVM metrics.

The Scheduling Engineer’s duties include:

defining all of the program’s efforts in terms of work packages as early as possible, beginning with pre-Milestone A activities;

documenting the inputs (both personnel and infrastructure) required to complete work packages, the time required to complete the work package, and the outcomes expected from the work package;

revising and developing more detailed work packages as the project moves through subsequent stages. 30

A full time scheduling engineer is not required during the Concept Studies phase, but a part-time resource is needed to help build the initial WBS structure for the program. Later phases will require this individual to be staffed at one FTE to better define work packages and program schedules.

30 Practice Standard for Work Breakdown Structures, Project Management Institute (PMI), Newtown Square, PA, Second Edition, pp. 1-8.

32

Test Engineer

The Test Engineer, while not needed for a full FTE of support during pre- Milestone A activities, should be brought in as a part-time FTE. It is critical that test planning not wait until after Milestone B, as is often the case. Understanding the scope and duration of testing that will be required for this project must be defined at the earliest stages of the project.

The Test Engineer’s duties include:

assisting in defining test requirements based on the systems requirements, and verifying traceability of requirements from the lowest to highest level of integration;

developing system test and evaluation scenarios and acceptance criteria for operational states, cases, missions, and environmental factors;

assisting in developing an approach to verify and validate requirements by inspection, demonstration, analysis, and testing (static, dynamic, usability, and stress tests);

assisting with developing and defining test and evaluation plans and procedures; creating test and evaluation strategies to field an effective and interoperable

system through developmental and operational testing; observing and communicating developmental and operational test results; influencing sponsor/customer input on mitigation strategy and system acceptance,

based on the results of the developmental and operational tests.

If engaged early in the program’s life cycle, the Test Engineer can assist analyzing alternative solutions. For example, one solution might entail more testing than another. Perhaps the most flagrant program disaster that resulted from the lack of robust upfront test engineering were the failures associated the MK 14 Torpedo’s failed Mk VI magnetic exploder during WWII. The “testing” of the system ended up happening in battle. Several submarines were lost as a result of trying to counter the torpedo’s deficiencies and much Japanese tonnage was not sunk as a result. 31

31 Shireman, Douglas, “U.S. Torpedo Troubles During WWII”, WWII Magazine, February 1998.

33

Logistics Engineer

The Logistics Engineer may be part time, but someone needs to be focused on the O&M requirements (such as bandwidth and help desk support) that may be required for the program. These issues may influence the decision on which concepts to pursue.

The Logistics Engineer will:

design processes that incorporate reliability, maintainability, and availability into the system;

consider reliability of all subcomponents or subsystems and modify system design accordingly;

work with complex mathematical models that consider elements such as mean time between failures (MTBF), mean time to failure (MTTF), mean time to repair (MTTR), failure made and effects analysis (FMEA), statistical distributions, queuing theory, and a host of other considerations.

The total program office size should be limited to fewer than 50 people. Smaller program offices are more efficient in making decisions. 32 Based on a survey of 503 program managers and 12 program offices over a four year period, the average time to stand up a program office to a point where its processes were repeatable and not chaotic is one to two years. 33

Program Management Office Cost During Initial Concept Studies

In summary, the positions and their costs during pre-milestone A activities can be summed up as follows:



34

There will also be some cost for office space, office expense, and travel.34 The office expense is computed at a rate of $30 per sq. foot per year35, with an average of 250 square feet per staff member36. The cost of copiers, computer access, printers, and telephones is priced at $2,000 per staff member per year. Travel is computed based on airfare, transportation, and hotel fare costing $2, 500 per trip per PMO staff member.

34 Abdi, Kaman, “Washington DC Commercial Real Estate on Track for Robust Recovery”, www.awsomedc.com.

35 Lewis, Christiana, “Rents Signal Rise of D.C., Fall of NY”, Wall Street Journal, 8 January, 2010.

36 “How Much Office Space for This? How Much Office Space for That?”, www.officefinder.com.

35

Hourly Rate FTE NeededAnnual Rate (1900

hours) Legend

Program Manager $230 1.0 $437,000Blue shading denotes government staff or FFRDC

Financial Manager $180 1.0 $342,000Purple Shading denotes lead staff beneath PM

EVM Monitor $170 0.5 $161,500 Italics denotes contractor

Expenditures Monitor $170 0.5 $161,500

Pink color denotes implementation team vice PMO

Cost Analyst $180 1.0 $342,000Admin Director $180 0.0 $0

COTR $180 0.1 $34,200Oversight/Planning Officer $180 1.0 $342,000

Legal Officer $180 0.1 $34,200Systems Engineer $230 1.0 $437,000

Systems Architect $220 1.0 $418,000Requirements Engineer $180 1.0 $342,000Quality Assurance Engineer $180 0.3 $85,500Security Engineer $220 0.5 $209,000Scheduling Engineer $180 0.5 $171,000Test Engineer $220 0.5 $209,000Logistics Engineer $180 0.5 $171,000Infrastructure (Building Costs) $78,375Office Expense (connectivity, IT, copiers) $20,900Travel $26,125

Total PMO 10.5 3,996,175.0

Percent of Cost Attributable to PMO 100%

COST (Initial Studies)POSITION

Deliverables for Milestone A – entry point for Phase A: Concept Refinement –Technology Maturity Demonstration.

Milestone A determines whether the program is ready to enter into Phase A: Concept Refinement- Technology Maturity Demonstration. The Milestone Decision Authority (MDA) reviews the documentation and determines whether there is a potential solution or affordable increment of useful capability. It also ensures that resources and capability needs are matched. These milestone reviews can be “paper reviews” or they may be staffed with appropriate representatives. The MDA may be delegated to the IC element, or it may be retained by the DNI at using a National Intelligence Acquisition Board (NIAB) or by ODNI and DoD in the case of a Joint Intelligence Acquisition Board (JIAB).

In preparation for Milestone A, the following document responsibility matrix is recommended:

C= contributorBlue denotes gov't staff

Purple denotes direct report to PM

Initial WBS Structure

Analysis of Alternatives Initial LCC Estimates

Independent Cost Estimate

Program Manager C C C CFinancial Manager C P P

EVM Monitor C C CExpenditures Monitor C CCost Analyst C C C C

Admin Director C CCOTR C COversight/Planning Officer C C C CLegal Officer C C

Systems Engineer P C CSystems Architect C C C CRequirements Engineer C C C CSecurity Engineer C C C CScheduling Engineer P C C CTest Engineer C C C CLogistics Engineer C C C C

Contributor

36


Blue denotes gov't staff


Intelligence Capability Baseline Descriptions (ICBD)

Initial CBJB Submission/

Reprogramming Document

Independent Program Assessment (IPA)

Exhibit 300 (if designated)

Program Manager C CFinancial Manager P P C

EVM Monitor C C CExpenditures Monitor C C CCost Analyst C C

Admin Director C C P PCOTR COversight/Planning Officer C C C CLegal Officer C C

Systems Engineer C C C CSystems Architect C C CRequirements Engineer P C CSecurity Engineer C C CScheduling Engineer C C C CTest Engineer C C CLogistics Engineer C

Contributor




Clinger-Cohen Certification


StrategyMisson Utilities Studies (MUS) CONOPS

Program Manager C C PFinancial Manager

EVM MonitorExpenditures MonitorCost Analyst

Admin Director C C CCOTROversight/Planning Officer C C C CLegal Officer C

Systems Engineer P P C CSystems Architect C C C CRequirements Engineer C P CSecurity Engineer C C CScheduling Engineer C CTest Engineer C CLogistics Engineer C

Contributor

37




System ArchitectureProgram Management

Plan

Risk Management, Identification, and

Mitigation PlanSecurity Classification

GuideProgram Manager C P C CFinancial Manager C C

EVM MonitorExpenditures MonitorCost Analyst

Admin Director C C C CCOTROversight/Planning Officer C C CLegal Officer P C

Systems Engineer C C C CSystems Architect P CRequirements Engineer C CSecurity Engineer C C PScheduling Engineer CTest Engineer C CLogistics Engineer C C

Contributor




Statement of Capabilities -A

Systems Engineering Master Plan (SEMP)

Technology Maturity Plan/ Assessment

Test and Evaluation Master Plan (TEMP)

Program Manager C C CFinancial Manager P C

EVM Monitor CExpenditures Monitor CCost Analyst C C

Admin Director C C CCOTROversight/Planning Officer C C C CLegal Officer

Systems Engineer C P P PSystems Architect C C C CRequirements Engineer C C C CSecurity Engineer C C C CScheduling Engineer C CTest Engineer C CLogistics Engineer C C C

Contributor

38

Senior acquisition executives are frequently frustrated by the lack of upfront and detailed systems engineering and in-depth acquisition strategies. 37 They want program managers to take more time and invest more resources in these initial efforts. Appendix D of the DNI Annual Program Management Plan (PMP) Report (2010) to Congress contains a similar recommendation. 38

Phase A: Concept Refinement and Technology Maturity Demonstration

The purpose of this phase is to identify, document, analyze, and refine operational requirements, explore alternative solutions for meeting mission needs, reduce technology risks, identify appropriate technologies, and develop a functional specification and design. The Analysis of Alternatives (AOA) continues to be refined, including the evaluation of counterintelligence risks. Technology risk is mitigated and technology maturity is evaluated. The AOA outlines the most probable solution to use in estimating budget requirements. During this phase, decisions impacting 5-15% of the life cycle cost are made, which is a significant drop from the pre-phase A impact of 75%. 39

A program architecture and concept of operations are developed in order to assess IC Information Technology Architecture compliance. A JIAB validated and approved requirements document (known as a “Capability Development Document”) for Phase B identifying key performance indicators and schedule goal is developed. An initial Program Management Plan (PMP) is created which includes cost, schedule, and performance goals. An Intelligence Community Baseline Description (ICBD) is established to facilitate program estimates. A DNI CAIG developed Independent Cost

37 Castelli, Christopher, “Lack of Detail in Acquisition Strategies Irks Pentagon Procurement Shop”, Inside the Navy, August 20, 2010.

38 2010 Program Management Plan (PMP) Submission to Congress, ODNI/Senior Acquisition Executive, Appendix D.


39

Estimate (ICE) is crafted and an Independent Program Assessment (IPA) is conducted. One of the purposes of the IPA is to assess all potential risk areas. Details of these and other documents are contained in Intelligence Community Policy Guidance (ICPG) 801.1.

Organization Chart During Phase A - Concept Refinement and Technology Maturity Demonstration

Program Manager

Financial Manager

Admin Director

COTR

Legal Officer


Systems Engineer

Systems Architect


SecurityEngineer

EVM Monitor


SchedulingEngineer

Test Engineer

Logistics Engineer

Cost Analyst

Phase A

Communications

Manager

Risk Manager


*Configuration Management

Specialist

*Software Engineer

=contractors=PM deputies=PM

* = implementation team


40

Program Management Office Roles Staffed During Phase A - Concept Refinement and Technology Maturity Demonstration

In addition to the roles staffed during pre-Milestone A Concept Studies, the following roles are staffed during this phase.

Administrative Director

The Administrative Director may be part time at the beginning of the project. However, as the project grows, this will become a full time position. Supporting this billet are the previously discussed positions of COTR, Legal Officer, and Oversight/Plans Officer.

Communications Manager

This individual is responsible for the processes for ensuring timely generation, collection, distribution, storage, retrieval, and disposition of program information. Effective communication creates a bridge between diverse stakeholders involved in a project, connecting various cultural and organizational backgrounds, different levels of expertise, and various perspectives and interests in the project outcome. 40 The communications manager will:

identify all people or organizations impacted by the project and document relevant information regarding their interests, involvement, and impact on project success;

design a plan to address stakeholder communication needs and address an approach to provide them with appropriate communications at the appropriate time;

work with the oversight/ planning manager to collect and distribute performance information, including status reports, progress measurements, and forecasts;

ensure that stakeholders are informed about the completion of key milestones and deliverables.

During pre-Milestone A activities, these functions can be consolidated with that of the Oversight/ Planning Manager. However, after Phase A, the functions performed by the


41

Communications Manager become a full time position. The Communications Manager reports to the Admin Director. 41

Risk Manager

This individual is responsible for conducting risk management planning, identification, analysis, response planning, and monitoring and control on a project. The Risk Manager promotes processes that increase the probability and impact of positive events, while decreasing the probability and impact of negative events in the project. The Risk Manager will:

define how to conduct risk management activities; identify risks that affect the program and document their characteristics; prioritize risks for further analysis or action by assessing and combining their

probabilities of occurrence and impact; numerically analyze the effect of risks on overall project objectives; develop options and actions to enhance opportunities and reduce threats; monitor and control the tracking of identified risks, identifying new risks, and

evaluating risk process effectiveness throughout the project. 42

The risk process must begin after Milestone A and should be a full time FTE. The Risk Manager reports to the Administrative Director.

The risk management process should not just consist of a Risk Manager who meets with the PM once a month to go over risks and then inserts the risk, with its probability and impact into a risk management tool, and then adds mitigation and action strategies that have not been vetted with the senior PMO team, including the various engineers, Financial Manager, and Admin Officer. A good process involves the integration of all of the senior PMO staff into the risk management process.



42

Configuration Management Specialist

The Configuration Management Specialist manages process and software baselines, as well as configuration identification, control, status accounting, and authentication. The CM Specialist uses development libraries that provide for the controlled handling of software code, documentation, media, and related data in their various versions until final approval and implementation. This individual does not have to be government staff, and is generally viewed as being a part of the implementation team as opposed to formal PMO staff overhead.

The Configuration Management Specialist manages the following processes:

assuring that the software configuration used in critical phases of testing, acceptance, and delivery is compatible with the associated documentation;

status accounting, including the recording and reporting of data reflecting the software's configuration identification, proposed changes to the configuration identification, and the implementation status of approved changes;

conducting configuration audits; managing configuration change control processes to change a configuration

baseline. 43

Software Engineer

The Software Engineer is responsible for the design, development, and maintenance of the information systems of the project. Software engineers apply the principles and techniques of computer science, engineering, and mathematical analysis to the design, development, testing, and evaluation of the software and systems that enable computers


43

to perform their many applications. This individual does not have to be government staff, and is generally viewed as being a part of the implementation team as opposed to formal PMO staff overhead. The Software Engineer’s activities include:

determining the development methodology (waterfall, prototype, spiral, incremental, iterative/agile);

supervising the software implementation team; determining the release schedule for software; working with other members of the PMO and implementation team to implement the

software solution.

The Software Engineer is not a necessary position during pre-Milestone A activities as the program “solution” is relatively ill-defined. This individual is often brought on at a later point because of his/her discrete skills in the implementation areas that are pertinent to the IT solution that is finally decided upon. The engineering staff until Phase A consists of the Systems Engineer, the Systems Architect, Requirements Engineer, Security Engineer, Scheduling Engineer, and Test Engineer. Their focus in the early stages of the project should be to design an architecture and define capabilities.

Program Management Office Cost During Phase A: Concept Refinement and Technology Demonstration

In summary, the positions and their costs during pre-milestone A activities can be summed up as follows:

44


hours) PMOAnnual Rate (1900 hours)

Implementation Team Legend

Program Manager $230 1.00 $437,000

Blue shading denotes government staff or FFRDC

Admin Assistant $130 1.00 $247,000

Purple Shading denotes lead staff beneath PM

Financial Manager $180 1.00 $342,000Italics denotes contractor

EVM Monitor $170 1.00 $323,000


Expenditures Monitor $170 1.00 $323,000Cost Analyst $180 1.0 $342,000

Admin Director $180 1.00 $342,000COTR $180 0.25 $85,500Oversight/Planning Officer $180 1.00 $342,000Legal Officer $180 1.00 $342,000Communications Manager $150 1.00 $285,000Risk Manager $200 1.00 $380,000

Systems Engineer $230 1.00 $437,000Systems Architect $220 1.00 $418,000

Asst Systems Architect $150 1.00 $285,000Requirements Engineer $180 1.00 $342,000

Asst Requirements Engineer $150 1.00 $285,000Quality Assurance Engineer $180 1.00 $342,000

Configuration Management Specialist $150 1.00 $285,000Software Engineer $220 1.00 $418,000Security Engineer $220 1.00 $418,000Scheduling Engineer $180 1.00 $342,000Test Engineer $220 1.00 $418,000Admin Assistant $130 1.00 $247,000

Logistics Engineer $180 1.00 $342,000Infrastructure (Building Costs) $166,875Office Expense (connectivity, IT, copiers) $44,500Software Licenses $4,850Travel $55,625

Total PMO 22.25 7,666,500.00Total Implementation 2.00 $703,000Percent of Cost Attributable to PMO 91.60%

POSITION

The orange shading denotes billets that are normally considered to be part of the implementation team, and not the program office. Italics denote that they are usually contractors. There for office space, office expense, and travel will increase as the project advances. Several billets that were part-time during pre-Milestone A, are now full time.

Deliverables for Milestone B-entry point for Phase B: Development, Integration, and Demonstration

Similar to Milestone A, the Milestone B decision determines whether the program is ready to enter into Phase B: Development, Integration, and Demonstration. The completion of all required exit criteria and requirements of Phase A are evaluated by the MDA. The MDA makes an affordability assessment, reviews and considers the program key assumptions, and establishes the PMP baseline, including program specific exit criteria.

45

In preparation for Milestone B, the following document responsibility matrix is recommended:


Blue denotes gov't staff Purple denotes direct report to PM

CBJB / IPBS Submission

Financial Reports Exhibit 300

Bandwidth Capability

AssessmentProgram Manager C C C C

Admin AssistantFinancial Manager P P C

EVM Monitor CExpenditures Monitor C CCost Analyst C C C

Admin Director PCOTR COversight/Planning Officer C C CLegal OfficerCommunications Manager C C C CRisk Manager

Systems Engineer C C PSystems Architect C C C

Asst Systems ArchitectRequirements Engineer C C C

Asst Requirements EngineerQuality Assurance Engineer C

Configuration Management SpecialistSoftware EngineerSecurity Engineer CScheduling Engineer C C CTest Engineer C CAdmin Assistant C C C CLogistics Engineer C

Contributor

46



Acquisition

Strategy

Nunn-McCurdy Reporting

(if applicable) CONOPS

Program Manage

ment Plan

(PMP)

Systems Engineering Master Plan

(SEMP)Program Manager C C P P C

Admin Assistant CFinancial Manager C P C


Admin Director P C C CCOTR COversight/Planning Officer C C C C CLegal Officer C CCommunications Manager C C C CRisk Manager C C

Systems Engineer C C C C PSystems Architect C C C


Asst Requirements EngineerQuality Assurance Engineer C C C

Configuration Management Specialist C CSoftware Engineer C CSecurity Engineer C C CScheduling Engineer C C CTest Engineer C C CAdmin Assistant C C CLogistics Engineer C C C

Contributor

47



Test and Evaluation

Master Plan (TEMP)

Risk Identification

and Mitigation Plan

Intelligence Capability Baseline

Descriptions (ICBD)

Program Office

Estimate (POE)


(ICE)Affordability Assessment

Program Manager C C C C C CAdmin Assistant C C C C C C

Financial Manager C C P P PEVM Monitor C C C CExpenditures Monitor C C C CCost Analyst C C C C C

Admin Director C C C C C CCOTR COversight/Planning Officer C C C C CLegal Officer CCommunications Manager C C C C C CRisk Manager C P C C C

Systems Engineer C C C C C CSystems Architect C C C C

Asst Systems ArchitectRequirements Engineer C P C C

Asst Requirements EngineerQuality Assurance Engineer C C

Configuration Management SpecialistSoftware Engineer C C C C CSecurity Engineer C C C CScheduling Engineer C C C CTest Engineer P C C C CAdmin Assistant C C C C CLogistics Engineer C C C C C

Contributor

48



Integrated Program

Assessment (IPA)

Clinger-Cohen

Certification

IC Enterprise Architecture Compliance

Information Support

Plan


Strategy (TDS) and

Technlogy Maturity

AssessmentProgram Manager C C C C

Admin Assistant C C C C CFinancial Manager C

EVM Monitor CExpenditures Monitor CCost Analyst

Admin Director C C C C CCOTR COversight/Planning Officer P C C CLegal Officer CCommunications Manager C C C C CRisk Manager C

Systems Engineer C P C C PSystems Architect C C P C C


Asst Requirements EngineerQuality Assurance Engineer C

Configuration Management Specialist CSoftware Engineer C CSecurity Engineer C C CScheduling Engineer C CTest Engineer C P CAdmin Assistant C C C C CLogistics Engineer C C C

Contributor

49



Security and

Counter Intellige

nce Assessm

ent

Analysis of

Alternatives

(AOA)

Security Classification

GuideVulnerability Assessment

Statement of

Capabilities

Document (B)(SOC-B)

for NIAB/JIAB

Program Manager C CAdmin Assistant C C C C

Financial ManagerEVM MonitorExpenditures MonitorCost Analyst C

Admin Director C C CCOTR C C COversight/Planning Officer C C C C CLegal Officer C C CCommunications Manager C C C C CRisk Manager C C

Systems Engineer C P C P CSystems Architect C C C

Asst Systems ArchitectRequirements Engineer C C P

Asst Requirements EngineerQuality Assurance Engineer

Configuration Management SpecialistSoftware EngineerSecurity Engineer P C P CScheduling Engineer C CTest Engineer C C CAdmin Assistant C C C C CLogistics Engineer C C

Contributor

Phase B: Development, Integration, and Demonstration.

The focus of this phase is to develop a system, capability, or increment using a robust systems engineering approach. During the phase, the program should integrate subsystems in a build-test-build approach to solve compatibility issues as they are encountered thereby ensuring compliance with the IC Enterprise Architecture. It should complete Phase B exit criteria, and demonstrate that the program will perform as advertised in stressful environments. Discrete increments of capabilities should be

50

deployed, including prototype capabilities, to ensure user comfort with the functionality developed. Mission endangering security and counterintelligence risks should be reduced and mitigated. Updates are made to the ICBD, ICE, IPA, and PMP.

Organization Chart During Phase B: Development, Integration, and Demonstration

Program Manager

Financial Manager

Admin Director

COTR

Legal Officer


Systems Engineer

Systems Architect


SecurityEngineer

EVM Monitor


SchedulingEngineer

Test Engineer

Logistics Engineer

Cost Analyst

Phase B


Risk Manager


*Configuration Management

Specialist

*Software Engineer

*Change Management

Manager

*Training Curriculum Developers

*Scrum Team Lead

*Interface Development Lead

*Production Database

Development Lead

=contractors=PM deputies=PM



51

Program Management Office Roles Staffed During Phase B: Development, Integration, and Demonstration

In addition to the roles discussed for Initial Concept Studies and Phase A, the following additional people are added to the program during Phase B.

Change Management Manager

The Change Management Manager is responsible for the process of identifying, documenting, analyzing, prioritizing, and agreeing on changes from the project and their impact on the project stakeholders. This individual seeks to transition users, teams, and organizations from a current state to a desired future state. The change management manager will:

engage in creative marketing to enable communication between change audiences, but also understand social understanding about leadership’s styles and group dynamics;

align groups’ expectations, communicates, integrates teams and manage people training;

make use of metrics, such as leader’s commitment, communication effectiveness, and the perceived need for change to design accurate strategies, in order to avoid change failures or solve troubled change projects.

Training Curriculum Developers and Trainers:

As the solution is defined in greater detail during Phase B, training curricula and trainers need to be defined to support the implementation of the program’s solution.

52

Scrum Team Lead

In order to understand the role of the Scrum Team Lead, it is necessary to understand Agile methodologies. The proponents of Agile techniques have created a common approach among their various methodologies, which includes:44

• early and continuous software delivery, which cuts short onerous requirements development cycles;

• incremental rapid delivery of small capabilities;

• customers and developers work daily together;

• frequent testing;

• motivated teams, where the project manager and Systems Engineer are “Black Belts” in their fields;

• the use of working software primary measure of success to measure earned value;

• acceptance of changing requirements late in delivery cycle;

• continuous customer feedback.

Agile acquisition is generally appropriate for two types of systems development: 1) enterprise systems with a high degree of uncertainty, even if the purpose and intent of the project’s solution are well-defined; and 2) small, tactical systems that may have a short shelf life, but an immediate and pressing need. 45

Common “methods” associated with Agile include: • Rapid Application Development (RAD);

• Extreme programming;

• Dynamic System Development Model (DSDM);

44 Agile Manifesto, Snowbird, Utah, 2001.

45 Dobbins, Jim, and Schuh, D., “Agile Acquisition Strategy”, Systems Engineering Guide, The MITRE Corporation, 2009.

53

• FDD (Feature Driven Development);

• EVO (Evolutionary Development);

• RUP (Rational Unified Process);

• SCRUM;

• Extreme Programming (XP);

• Crystal.

Agile proponents favor common team behaviors, which include:

• keeping meetings short, but occurring frequently.

• during these meetings:

• report status since previous day, and what the team will be working on in the near term;

• describe lessons learned at the end of each iteration;

• validate that the team has fixed amount of time to complete deliverables;

• assigning the team a prioritized list of use cases to work on during each iteration;

• ensuring the appointment of team members are experts in Program Management and Systems Engineering.

In this scenario, it is assumed that the project will be implemented using the SCRUM Agile framework. SCRUM is not an acronym, but comes from the sport of rugby, where it is a method of restarting the game, usually after a foul or when the ball has gone out of play. During each “sprint” or iteration, typically a four to eight week period (with the length being decided by the team), the team creates a potentially shippable product increment (for example, working and tested software). The set of features are often captured in “use cases” or “story boards” that describe the requirement that is to be

54

addressed. These use cases are part of a “backlog” or collection of use cases that capture all of the requirements of the program. These use cases in the back log are prioritized at a high level.

Agile planning is done at two levels. At a higher level, requirements fitting a certain “theme” (such as database development or a particular user function), which incorporate prioritized “features” are designed for each “release”, iteration, sprint, or “SCRUM.” At a more defined level, the “release” or “iteration” is designed around “use cases” or “story cards” that spell out in detail what is to be developed. At the conclusion of the sprint, the team assesses, adapts, and adjusts in preparation for the next iteration.

For each iteration or sprint, the team should have as a goal the implementation of at least one demonstrable function. The function should be defined in advance with the requirements, security, and quality engineers. They need to define the tasks and efforts in implementing these features or use cases as well as what dependencies might impact the prioritization. 46

Working with other members of the program office, the SCRUM Team Lead, Software Engineer, Systems Engineer, and Requirements Engineer should define over time the highest impact use cases/ story boards, and be constantly adjusting or better defining them in order to implement those with highest impact during each iteration. Their assigned steps should include:

users writing story boards/use cases;

programmers estimating cost of stories; users prioritizing stories/use cases based on business value and estimated costs,

which results in “story points” for each use case. recognizing that customers may need to change, split, merge stories to fit each

iteration;

During the sprint planning meeting, the team determines which backlog items are included. During this meeting, the SCRUM Team Manager informs the team of the items in the product backlog that he or she wants completed. In making this decision, the SCRUM Team Manager works with the Requirements, Security, and Scheduling Engineers. The team then determines how much of these use cases, story boards, or

46 Griffiths, Mike, “Quadrus: Managing Agile Projects”, PMI Seminar, Anaheim, CA, 2006.

55

features they can commit to complete during the next sprint. During a sprint or iteration, no one is allowed to change the sprint backlog, which means that the requirements are frozen for that sprint. After a sprint is completed, the team demonstrates how to use the software. They then begin planning for the next sprint.

Agile iterations can support earned value metrics. Prior to Milestone B, every Use Case Feature will be assigned a difficulty factor number from 1 (easiest) to 10 (most difficult)

56

B C

Production, Deployment &Sustainment

Phase CConcept Refinement

Technology Maturity DemonstrationPhase A

Development, Integration &DemonstrationPhase B

MSA A

PDR

MS BSRR

CDRTRRMS CIOCFOC

SDR


Phase CDevelopment, Integration &Demonstration

Phase BIncrement 1CB







TRRMS CIOCFOC

PDRCDRTRRMS CIOCFOC

Evolutionary Approach:Increased emphasis on Phase A tech maturation Shorter, well-defined increments support both pre- and unplanned tech insertion opportunitiesPhase C emphasis on sustainment and recap (O&M); may not be designated as MSA

Technology A TRL 6

Technology BTechnology C TRL 6

Technology X(unplanned)

PDRCDR

TRL 6

TechnologyMaturity Plan

TRL 6

Well defined

increments could be separate Phase B

MSAs

IC Elements plan and execute

technology

roadmap

and a level of effort (LOE) estimate in hours. This will be done using the following estimation technique:

P*J*I= cost of work performed in points for the use case, where,

P= points assigned, based on level of difficulty (1 to 10).

J= time for developers to develop the use case in days

I= FTE cost of development team.

In estimating P and J, the Scrum Team Lead should include the preparatory meetings and preliminary work for that iteration. Budgeted cost of work scheduled (BCWS) for a particular run should be based on the formula, but should not exceed 5*(total days in iteration)*(total developer FTE on iteration), where “5” denotes an “average” level of difficulty. BCWS can then be compared with Actual Cost of Work Performed at the end of the SCRUM.

Interface Development Lead

The Interface Development Lead is responsible for capturing the touch points between the system being created by the project and any other existing systems. This role is particularly critical when different software systems must be integrated, in which the owners of each of those systems have equities to protect.

The Interface Development Lead’s activities include:

determining how data can be transferred back and forth with other systems; identifying the software or application programming interfaces (APIs) that

facilitate these exchanges; developing policy and protocol solutions where needed.

57

Production Database Development Lead

The Production Database Development Lead is responsible for setting up databases to support development of the software product. The Production Database Development Lead’s activities include managing the creation, maintenance, and use of the database storage structures of social organizations and of their users.

Assistants to the Various Engineers and Other Leads:

As the program now begins detailed design and implementation of the solution, additional individuals are needed. Administrative Assistants will be needed for the Program Manager and Systems engineer to:

prepare and maintain calendars; arrange office space for meetings; perform printing and copying in support of other managers; manage office support systems (phone lines, printer maintenance); assist in preparing briefings and certificates.

Scrum Team Developers are needed to assist the Scrum Team Lead in designing the solution. Interface developers will assist the Interface Development lead. Database developers and designers will also be needed to assist with the need for various development and production database solutions. Test engineering support will be needed to support the Test Engineer in designing test cases and database support will be needed for test database support instances.

During Phase B, the full implementation team (in orange) is staffed out. At this point, the PMO cost drops to 21 percent of total costs.

58



Implementation Legend






EVM Monitor $170 1.00 $323,000



Admin Director $180 1.00 $342,000COTR $180 1.00 $342,000Oversight/Planning Officer $180 1.00 $342,000Legal Officer $180 1.00 $342,000Communications Manager $150 1.00 $285,000Change Management Manager $180 1.00 $342,000Training Curriculum Developers and Trainers $150 8.00 $2,280,000Risk Manager $200 1.00 $380,000


Asst Systems Architect $150 1.00 $285,000Requirements Engineer $180 1.00 $342,000

Asst Requirements Engineer $150 1.00 $285,000Security Engineer $220 1.00 $418,000Quality Assurance Engineer $180 1.00 $342,000

Configuration Management Specialist $150 1.00 $285,000Software Engineer $220 1.00 $418,000

Scrum Team Lead $190 1.00 $361,000SCRUM Team Developer $180 24.00 $8,208,000

Interface Development Lead $190 1.00 $361,000Interface Developer $181 10.00 $3,439,000

Production Database Development Lead $190 1.00 $361,000Production Database Developer $185 10.00 $3,515,000

Scheduling Engineer $180 1.00 $342,000Test Engineer $220 1.00 $418,000

Asst Test Engineer $180 10.00 $3,420,000Testing Database Support Engineer $180 6.00 $2,052,000

Admin Assistant $130 1.00 $247,000Logistics Engineer $180 1.00 $342,000Infrastructure (Building Costs) $165,000 $562,500Office Expense (connectivity, IT, copiers) $44,000 $150,000Hardware (servers, routers) $1,000,000Bandwidth leases $500,000Software Licenses $200,000Travel $55,000 $500,000

Total PMO 22.00 7,581,000.00Total Implementation 75.00 $28,296,500

Percent of Cost Attributable to PMO 21.13%

POSITION

COST ( Phase B)

59

Program Management Office Cost During Phase B- Development, Integration, and Demonstration

This model assumes, from an agile perspective, six iterations by the Software Team, each lasting two months during phase B. Hardware costs assume some investment needed in hardware, bandwidth and routers. Travel is based on $2500 per trip with 25 trips times the number of trainers. It is assumed that these journeys are to serve remote users.

Deliverables for Milestone C – Entry Point for Phase C: Production / Sustainment

The decision that results from Milestone B authorizes entry into the Production, Deployment, and Sustainment Phase of the ICAM. The MDA ensures that resources and capability needs are matched, which includes an assessment that the program is properly staffed and the schedule executable within the existing budget. The MDA also makes an affordability assessment and approves the latest PMP baseline.

Suggested deliverables produced for Milestone C include the items on the following charts:

60



CBJB / IPBS Submission

Financial Reports Exhibit 300

Acquisition

Strategy

Analysis of

Alternatives

Nunn-McCurdy

Reporting (if applicable)

Program Manager C C C C C CAdmin Assistant C C C C C C

Financial Manager P P C PEVM Monitor C CExpenditures Monitor C C CCost Analyst C C C C C C

Admin Director C C P P C CCOTR C C COversight/Planning Officer C C C P CLegal Officer C CCommunications Manager C C C C C CChange Management ManagerTraining Curriculum Developers and TrainersRisk Manager

Systems Engineer C C C C CSystems Architect C C C



Configuration Management SpecialistSoftware Engineer

Scrum Team LeadSCRUM Team Developer

Interface Development LeadInterface Developer

Production Database Development LeadProduction Database Developer

Security Engineer CScheduling Engineer C C CTest Engineer C C

Asst Test Engineer Testing Database Support Engineer

Admin Assistant C C C CLogistics Engineer C C

Contributor

61



Interoperability

Assessment CONOPS

Program Manage

ment Plan

Systems Engineering Master Plan

(SEMP)

Test and Evaluation

Master Plan (TEMP)

Program Manager C p P C CAdmin Assistant C C C C

Financial Manager CEVM MonitorExpenditures MonitorCost Analyst

Admin Director C C C CCOTROversight/Planning Officer C C C C CLegal Officer C C CCommunications Manager C C C CChange Management ManagerTraining Curriculum Developers and TrainersRisk Manager C C C

Systems Engineer P C C P CSystems Architect C C C C C

Asst Systems ArchitectRequirements Engineer C C C C C


Configuration Management SpecialistSoftware Engineer C




Security Engineer C C C C CScheduling Engineer C C C C CTest Engineer C C C C P


Admin Assistant C C C CLogistics Engineer C C C C

Contributor

62



Risk Identification

and Mitigation Plan

Intelligence Capability Baseline

Descriptions (ICBD)


(ICE)Affordability Assessment

Integrated Program

Assessment (IPA)

Clinger-Cohen

CertificationProgram Manager C C C C C C

Admin Assistant C C C C C CFinancial Manager C P P C

EVM Monitor C C CExpenditures Monitor C C CCost Analyst C C C C

Admin Director C C C C C CCOTROversight/Planning Officer C C C P CLegal Officer C C C CCommunications Manager C C C C C CChange Management ManagerTraining Curriculum Developers and TrainersRisk Manager P

Systems Engineer C C C C C PSystems Architect C C C C

Asst Systems ArchitectRequirements Engineer P C C


Configuration Management SpecialistSoftware Engineer




Security Engineer C C CScheduling Engineer C C CTest Engineer C C C C


Admin Assistant C C C CLogistics Engineer C

Contributor

63



Information Support

Plan


Strategy (TDS) and

Technlogy Maturity

Assessment

Security and

Counter Intellige

nce Assessm

entVulnerability Assessment

Statement of Capabilities_C

(SOC-C)

Security Classification

GuideProgram Manager C C C C C C

Admin Assistant C C C C C CFinancial Manager

EVM MonitorExpenditures MonitorCost Analyst C

Admin Director C C C C C CCOTROversight/Planning Officer C C C C C CLegal Officer C C CCommunications Manager C C C C C CChange Management ManagerTraining Curriculum Developers and TrainersRisk Manager C C C

Systems Engineer P P C C C CSystems Architect C C C C

Asst Systems ArchitectRequirements Engineer C C P


Configuration Management SpecialistSoftware Engineer C




Security Engineer C C P P PScheduling Engineer C CTest Engineer C C C C


Admin Assistant C C C C C CLogistics Engineer C C

Contributor

64

Phase C: Production/Sustainment

The purpose of this phase is to produce and deploy the system. As appropriate, removal and disposal of systems being replaced are included in this phase. Full operational capability (FOC) is achieved.

Organization Chart During C: Production/Sustainment

Program Manager

Financial Manager

Admin Director

COTR

Legal Officer


Systems Engineer

Systems Architect


SecurityEngineer

EVM Monitor


SchedulingEngineer

Test Engineer

Logistics Engineer

Cost Analyst

Phase C-Beginning


Risk Manager


*Configuration Management Specialist

*Software Engineer*Change Management

Manager

*Training Curriculum Developers

*Scrum Team Lead

*Interface Development Lead

*Production Database

Development Lead

*RFC/DR Lead

=government staff/ FFRDC=contractors=PM deputies=PM


*Site Installation Engineer

65

Program Management Office Roles Staffed During Production/Sustainment

Existing Rollout Discrepancy /Request for Change Lead

The Existing Rollout Discrepancy /Request for Change Lead is responsible for setting up processes to handle requests for change (RFCs) after the system is deployed, as well as those needed for handling discrepancy reports (DRs). Normally, this Lead should have his own development team to design, correct, and test these changes, as the SCRUM Team Lead is still rolling out the primary solution software.

Site Installation Engineer

The Site Installation Engineer is responsible for installing the software and infrastructure at various user sites upon project rollout.

Program Management Office Cost During Phase C- Production/Sustainment

As the Production/Sustainment (Phase C) continues, fewer program office staff are required. For example, the Legal Officer has performed most of the design and development work that is needed, and can be reduced to a part-time role at the beginning of Phase C. Requirements should be stable and so the assistant for the Requirements Engineer can be eliminated. Systems Architecture work can also be reduced. While the program office totals begin dropping from $7.6 million to $6.5 million at the beginning of phase C, the implementation team increases its cost from $28 million to $32 million. Travel cost now includes $2,500 per trip to 25 destinations for the site installation engineers.

66









EVM Monitor $170 1.00 $323,000



Admin Director $180 1.00 $342,000COTR $180 1.00 $342,000Oversight/Planning Officer $180 1.00 $342,000Legal Officer $180 0.25 $85,500Communications Manager $150 1.00 $285,000Change Management Manager $180 1.00 $342,000Training Curriculum Developers and Trainers $150 8.00 $2,280,000Risk Manager $200 1.00 $380,000


Asst Systems Architect $150 0.00 $0Requirements Engineer $180 1.00 $342,000

Asst Requirements Engineer $150 0.00 $0Security Engineer $220 1.00 $418,000Quality Assurance Engineer $180 1.00 $342,000



Interface Development Lead $190 1.00 $361,000Interface Developer $181 10.00 $3,439,000

Production Database Development Lead $190 1.00 $361,000Production Database Developer $185 10.00 $3,515,000

Existing Roleout Discrepancy /Request for Change Lead $200 1.00 $380,000

Request for Change/Discrepancy Report Developer $180 10.00 $3,420,000Site Installation Engineer $180 5.00 $1,710,000

Scheduling Engineer $195 1.00 $370,500Test Engineer $195 1.00 $370,500

Request for Change/Discrepancy Report Tester $180 4.00 $1,368,000Testing Database Support Engineer $180 5.00 $1,710,000

Admin Assistant $130 1.00 $247,000Logistics Engineer $180 1.00 $342,000Infrastructure (Building Costs) $140,625 $637,500Office Expense (connectivity, IT, copiers) $37,500 $170,000Hardware (servers, routers) $1,000,000Bandwidth leases $500,000Software Licences $2,000,000Travel $46,875 $812,500

Total PMO 18.75 6,526,500.00Total Project 85.00 $33,620,000

Percent of Cost Attributable to PMO 16.26%

POSITION

As the program reaches the middle of phase C, the Scrum Team starts reducing in numbers as most of the development finishes. Increasingly the software team becomes focused on implementing requests for change (RFC) to the existing system and discrepancy reports (DRs) where the product delivered is not meeting performance specifications.

67









EVM Monitor $170 1.00 $323,000



Admin Director $180 0.50 $171,000COTR $180 0.25 $85,500Oversight/Planning Officer $180 1.00 $342,000Legal Officer $180 0.00 $0Communications Manager $150 0.50 $142,500Training Curriculum Developers and Trainers $150 8.00 $2,280,000Risk Manager $200 0.50 $190,000


Asst Systems Architect $150 0.00 $0Requirements Engineer $180 0.50 $171,000

Asst Requirements Engineer $150 0.00 $0Security Engineer $220 0.00 $0Quality Assurance Engineer $180 1.00 $342,000



Interface Development Lead $190 1.00 $361,000Interface Developer $181 2.00 $687,800

Production Database Development Lead $190 1.00 $361,000Production Database Developer $185 2.00 $703,000


Request for Change/Discrepancy Report Developer $180 10.00 $3,420,000

Site Installation Engineer $180 5.00 $1,710,000Scheduling Engineer $195 1.00 $370,500Test Engineer $195 1.00 $370,500


Admin Assistant $130 1.00 $247,000Logistics Engineer $180 1.00 $342,000Infrastructure (Building Costs) $105,000 $397,500Office Expense (connectivity, IT, copiers) $28,000 $106,000Hardware (servers, routers) $1,000,000Bandwidth leases $500,000Software Licences $2,000,000Travel $35,000 $812,500

Total 14.00 4,816,500.00Total 53.00 $22,622,800Percent of Cost Attributable to PMO 17.55%

POSITION

COST ( Mid Phase C)

As late phase C approaches, the program is preparing to shut down and fold into a functional organization. Most of the developers (except for the RFC/DR team) are gone, as well as the Oversight/Planning Officer, Communication Manager, and training team, since most of the users have been trained. Most of the required project deliverables are completed. However, financial reports, budgets, and earned value metrics must be maintained until the end.

68









EVM Monitor $170 1.00 $323,000



Admin Director $180 0.00 $0COTR $180 0.25 $85,500Oversight/Planning Officer $180 0.00 $0Legal Officer $180 0.00 $0Communications Manager $150 0.00 $0Training Curriculum Developers and Trainers $150 0.00 $0Risk Manager $200 0.50 $190,000

Systems Engineer $230 1.00 $437,000Systems Architect $220 0.00 $0

Asst Systems Architect $150 0.00 $0Requirements Engineer $180 0.00 $0


Configuration Management Specialist $150 1.00 $285,000Software Engineer $220 0.00 $0

Scrum Team Lead $190 0.00 $0SCRUM Team Developer $180 0.00 $0

Interface Development Lead $190 0.00 $0Interface Developer $181 0.00 $0

Production Database Development Lead $190 0.00 $0Production Database Developer $185 0.00 $0



Site Installation Engineer $180 0.00 $0Scheduling Engineer $195 0.00 $0Test Engineer $195 1.00 $370,500


Admin Assistant $130 1.00 $247,000Logistics Engineer $180 1.00 $342,000Infrastructure (Building Costs) $73,125 $165,000Office Expense (connectivity, IT, copiers) $19,500 $44,000Hardware (servers, routers) $250,000Bandwidth leases $500,000Software Licences $2,000,000Travel $24,375 $0

Total 9.75 3,348,750.00Total 22.00 $10,464,000Percent of Cost Attributable to PMO 24.24%

POSITION

COST (late Phase C)

As the program transitions into O&M, the Program Manager and Financial Manager pass the reins to the functional office managing the resulting IT system. The RFC/DR team remains as an O&M necessity. In this example, it is assumed that Phase C lasts for three years before transitioning into O&M.

70




Program Manager $230 0.00 $0


Admin Assistant $130 0.00 $0


Financial Manager $180 0.00 $0Italics denotes contractor

EVM Monitor $170 0.00 $0


Expenditures Monitor $170 0.00 $0Admin Director $180 0.00 $0

COTR $180 0.00 $0Oversight/Planning Officer $180 0.00 $0Legal Officer $180 0.00 $0Communications Manager $150 0.00 $0Training Curriculum Developers and Trainers $150 0.00 $0Risk Manager $200 0.00 $0

Systems Engineer $230 0.00 $0Systems Architect $220 0.00 $0

Asst Systems Architect $150 0.00 $0Requirements Engineer $180 0.00 $0


Configuration Management Specialist $150 1.00 $285,000Software Engineer $220 0.00 $0

Scrum Team Lead $190 0.00 $0SCRUM Team Developer $180 0.00 $0

Interface Development Lead $190 0.00 $0Interface Developer $181 0.00 $0

Production Database Development Lead $190 0.00 $0Production Database Developer $185 0.00 $0



Site Installation Engineer $180 0.00 $0Scheduling Engineer $195 0.00 $0Test Engineer $195 0.00 $0

Request for Change/Discrepancy Report Tester $180 2.00 $684,000Testing Database Support Engineer $180 1.00 $342,000

Admin Assistant $130 0.00 $0Logistics Engineer $180 0.00 $0Infrastructure (Building Costs) $0 $60,000Office Expense (connectivity, IT, copiers) $0 $16,000Hardware (servers, routers) $0Bandwidth leases $500,000Travel $0 $0

Total 0.00 $0 $0Total 8.00 $4,319,000Percent of Cost Attributable to PMO 0.00%

COST (O&M)POSITION

In the graphical display of program cost, the display of PMO vs. implementation cost is a typical one. In this example, two years of O&M cost are also displayed. The ratio of PMO costs to implementation costs is a typical band of 15 to 20 percent during Phase B. It is slightly higher than the commercial benchmark (15 %) due to the increased oversight costs required by government programs for security, fraud, and privacy concerns.

71

A normal rule of thumb for any project is that 15 percent of the program’s cost needs to be associated with PMO activities.47 However, in government programs, the demands of oversight processes, security guarantees, and privacy concerns require additional program support. Moreover, the benchmark of 15 percent is based on PMO cost averaged throughout the duration of the non-government project. Smaller PMO staffs are also associated with lower levels of Program Management maturity. 48

In establishing benchmarks for the project management office cost as a percentage of total costs, expert estimates vary widely. One study speculated that while capital intensive projects, such as building the Denver Airport, have PMO costs in the range of 7% of total cost, web and IT based processes will run 12%, primarily because IT projects tend to use large amounts of expert labor time and relatively little capital as a part of total cost. 49 Other experienced IT program managers have estimated the total to be as high as 35%. 50

This scenario describes a project with a PMO cost that is roughly 25% of total program cost. However, many “benchmarks” consider the PMO staff members under the Systems Engineer to be part of the implementation team, and not part of the PMO overhead. If that were the case, the overall cost of the PMO overhead would drop to 15%.

47 Hollows, Jolyon, The Program Management Office Toolkit, American Management Association, 2003, pp.43-44.


49 Brown, Douglas, “PMO Cost as a % of Total Services”, Ganthead: the Online Community for IT Project Managers, June, 2006.

50 Fichter, Cornelius, “PMO Overhead Cost”, www.cornelius-fichtner.com.

72

$0

$5,000,000

$10,000,000

$15,000,000

$20,000,000

$25,000,000

$30,000,000

$35,000,000

$40,000,000

PMO Cost

Implementation Cost

Future Developments

The first step as a follow-on to this paper would be to find programs that are considered to be “successful” and use them as a benchmark to either counter this analysis or make it more “bullet proof.” Conversely, program failures contain lessons learned that may refine this study’s arguments.

73

Another step would be to continue to develop the paper to capture a program with joint deliverables (DoD/IC). A program with more deliverables would probably increase the need for additional support to the oversight/planning officer.

Finally, different types of programs could be included increasing the complexity of the staffing and cost models captured in the paper. For example, projects that involve the development of a major weapon system or vehicle (airplane, ship, or satellite) would involve different timelines and different jobs to be filled at different times.

74

Bibliography2010 Program Management Plan Submission to Congress, ODNI/Senior Acquisition Executive, Appendix D.

Abdi, Kaman, “Washington DC Commercial Real Estate on Track for Robust Recovery”, www.awsomedc.com.

Agile Manifesto, Snowbird, Utah, 2001.

Brown, Douglas, “PMO Cost as a % of Total Services”, Ganthead: the Online Community for IT Project Managers, June, 2006.

Castelli, Christopher, “Lack of Detail in Acquisition Strategies Irks Pentagon Procurement Shop”, Inside the Navy, August 20, 2010.

Defense Acquisition University, Acquipedia, Definition of a Program Manager.

Directive Type Memorandum (DTM) 09-021. Implementation of the Weapons Systems Acquisition Reform Act of 2009, Attachment 1.

DoDI 5000.02, December 2008, Enclosure 10.

Dobbins, Jim, and Schuh, D., “Agile Acquisition Strategy”, Systems Engineering Guide, The MITRE Corporation, 2009.

Fichter, Cornelius, “PMO Overhead Cost”, www.cornelius-fichtner.com.

“The Five Problems with CAPPS II”, American Civil Liberties Union, August 25, 2003.

Griffiths, Mike, “Quadrus: Managing Agile Projects”, PMI Seminar, Anaheim, CA, 2006.

Grasso, Al, U.S. Committee on Homeland Security, Subcommittee on Federal Financial Management, Government Information, Federal Services, and International Security, 2009.

“Haas, Robert”, iFlorida Model Deployment Final Evaluation Report, U.S. Department of Transportation, Federal Highway Administration, January 2009.

75

Hill Gerald, The Complete Project Management Office Handbook, CRC Press, 2007.

Hobbs, Brian, Aubrey, Monique, The Project Management Office: A Quest for Understanding, Project Management Institute (PMI), Newtown Square, PA, 2010.

Hollows, Jolyon, The Program Management Office Toolkit, American Management Association, 2003.

“How Much Office Space for This? How Much Office Space for That?”, www.officefinder.com.

Kelley, Mike, Kepner, Robert, Achieving Effective Results in Acquisition, The MITRE Corporation, January 2009.

Lewis, Christiana, “Rents Signal Rise of D.C., Fall of NY”, Wall Street Journal, 8 January, 2010.

Markle Foundation, “Protecting America’s Freedom in the Information Age”, October 2002.

Markle Foundation, “Creating a Trusted Network for Homeland Security”, December 2003, p. 30.

Practice Standard for Work Breakdown Structures, Project Management Institute (PMI), Newtown Square, PA, Second Edition.

“Pre Milestone A and Early-Phase Systems Engineering: A Retrospective View and Benefits for Future Air Force Acquisition”, National Resource Council, 2008.

Project Management Body of Knowledge (PMBOK), 4th edition, Project Management Institute (PMI), Newtown Square, PA, 2008.

Quicklook Handbook, OSD-C3I.

Shireman, Douglas, “U.S. Torpedo Troubles During WWII”, WWII Magazine, February 1998.

Spadunata, Laura, “The Privacy Problem: Homeland Security Failures”, News 21: A Journalism Initiative of the Carnegie and Knight Foundations, September 1, 2006.

76

Testimony by Kip Hawley, Assistant Secretary of the Transportation Security Administration, before the United States Senate Committee on Commerce, Science, and Transportation, February 9, 2006.

Wikipedia, “Requirements Engineering” and “Systems Architect” articles.

77

2010-08-pmo in a box conops(2010.12.08 )

Documents