gao manufacturing presentation

Readiness of Manufacturing Technologies in Transition for a Responsive and

Sustainable Industrial Base

U.S. Government Accountability OfficeAcquisition and Sourcing Management

2010 Defense Manufacturing ConferenceDecember 1, 2010

Briefing Components

• Major defense acquisition programs have historically experienced cost, schedule, and performance problems

• Pre-conditions to manufacturing is grounded in a knowledge-based approach for successful outcomes

• GAO study of Manufacturing Readiness Levels (MRLs) shows the new criteria offers DOD and its defense industrial base potential to achieve major savings and efficiencies

Big Picture: DOD Investment Levels Remain High, But Are Not Sustainable

Fiscal year 2011dollars in billions

Research, Development, Test and Evaluation and Procurement Funding

Source: GAO analysis of National Defense Budget Estimates for the Fiscal Year 2011 Budget.

CONDITION

0

25

50

75

100

125

150

175

200

225

250

275

FY 81 FY 84 FY 87 FY 90 FY 93 FY 96 FY 99 FY 02 FY 05 FY 08 FY 11 FY 14

PROCUREMENT

RDT&E

Committed and Planned Spending on FY2009 Portfolio of 96 Programs

$0

$200

$400

$600

$800

$1,000

$1,200

$1,400

$1,600

1977

1980

1983

1986

1989

1992

1995

1998

2001

2004

2007

2010

2013

2016

2019

2022

2025

2028

2031

2034

2037

2040

2043

Spent $

Billions of FY 2009 dollars

Commitments outstanding-$786 Billion

CONDITION

Snapshot: Cost and Schedule Growth for the 2008 Portfolio of 96 Programs

Performance of DOD’s Major Defense Acquisition Program Portfolio

Source: GAO analysis of DOD data.

CONDITION

Acquisition Programs Often Start Without a Sound Business Case

• An executable business case requires a balance between user needs and available resources

• Key elements include:• Clear, well-defined, and realistic requirements• Mature technologies• Accurate cost and schedule estimates• Management and workforce capacity• Full funding

• Moving forward with too many “unknowns” usually translates into cost, schedule, and performance problems

CONDITION

A Knowledge-Based Approach is Key to Successful Program Outcomes

In the context of DOD, level of knowledge attained at key junctures

Knowledge Point 1: At milestone B, a match is achieved between the user’s needs and the developer’s resources. Technology maturity is demonstrated and preliminary design is achieved.

Knowledge Point 2: At critical design review, the product design demonstrates its ability to meet user needs and is stable.

Knowledge Point 3: At milestone C, demonstrate that the product can be produced within cost, schedule, and quality targets. Full-up, integrated product tested in relevant environment .

MANUFACTURING PRE-CONDITIONS

Knowledge-Based Acquisition Model Focuses on Retiring Risks by Key Decision Points

KnowledgeBased Model

A

Technology development Production

Knowledge Point 1Technologies, time, funding and

other resources match customer needs.

Decision to invest in product development.

Development Start

Product developmentIntegration Demonstration

PDR CDR

B B’ C

Material DevelopmentDecision

Production Start

• Model provides framework for incremental, time certain (development constrained to 5 to 6 years or less), and knowledge-based approach to weapon system acquisitions.

• Success requires structured, disciplined application and adherence to model.

• Knowledge points align with key investment inflection points.

• Controls are in place for decisions makers to measure progress against specific criteria and ensure managers capture key knowledge before moving to next phase.

Knowledge Point 2Design is stable and performs

as expected.

Decision to start building and testing production representative prototypes.

Knowledge Point 3Production meets cost, schedule, and quality targets.

Decision to produce first units for customer.

5 to 6 years or less

Source: GAO.

MANUFACTURING PRE-CONDITIONS

Areas of Agreement Between DOD and GAO Concerning Problem Sources

• Programs initiated with poor foundations and inadequate knowledge for developing realistic cost estimates

• Programs move forward with artificially low cost estimates, optimistic schedules and assumptions, immature technologies and designs, and fluid requirements

• Imbalance between needs and the resources available to meet them contributes to budget and program instability

• Changing or excessive requirements cause cost growth• Essential to find better ways of doing business to ensure

systems are manufactured on time and cost effectively.

PROBLEM SOURCES

Objectives

• What are the manufacturing problems experienced by selected DOD programs?

• How Manufacturing Readiness Levels (MRLs) can address DOD’s manufacturing problems?

• How MRL’s compare to the manufacturing best practices of leading commercial companies?

• What are the challenges and barriers to implementing MRLs at DOD?

GAO MANUFACTURING STUDY

• Office of the Secretary of Defense, Military Services and Missile Defense Agency; NCAT; NDIA; DAU; and Manufacturing Experts

• MRL pilot programs (Army, Air Force, Navy)

• DOD case study programs• Army (Light Utility Helicopter)• Air Force (JASSM, and MQ-9 Reaper)• Navy (H-1 Helicopter Upgrade, and CVN-78 Aircraft Carrier)• MDA (EKV, and Standard Missile 3)

• Commercial Companies• GE Aviation (turbine blade technology) Cincinnati, OH• GE Healthcare (CT scan—scintillator) Milwaukee, WI• Honeywell (aerospace technology) Phoenix, AZ• Siemens Mobility (commuter rail systems) Sacramento, CA• Toyota (design and manufacturing) Ann Arbor, MI /Georgetown, KY

Scope of Work


What we found• Manufacturing cost growth are attributed to several factors: Nearly two-thirds of programs

experienced significant cost growth after they entered production—manufacturing process should be in control. Four DOD weapon systems analyzed reveal an inattention to manufacturing during planning and design, poor planning for supplier management, and lack of a knowledgeable workforce.

• MRL pilot programs show positive benefits: Army and Air Force programs have pilot-tested MRLs on science and technology and some major acquisition programs. Officials reported several benefits and noted they are a good way to manage, mitigate, and communicate readiness and risks early and throughout the acquisition process to avoid major consequences from manufacturing problems.

• MRLs embody many best practices of leading commercial firms: Firms use manufacturing readiness criteria, similar to DOD’s MRLs, to assess manufacturing early and at every stage of development. Firms work closely with suppliers who must meet high quality standards. One difference is that commercial best practice have processes in control prior to the production. They realize they are unable to make predictions about costs and production performance until the process is stable and defects are predictable.

• MRLs acceptance is growing but policy approval has been difficult: While acceptance of MRLs is growing within DOD and the defense industry, the Army’s Navy’s, Air Force’s leadership appears to be resistant and adoption has been slow. In addition, DOD manufacturing workforce trends show an overall decline that may impede future implementation of MRLs.

Summary


• GAO’s annual review of major acquisition programs shows systems still have problems with manufacturing, resulting in cost and schedule inefficiencies and avoidable rework

• GAO’s prior work and recent case study reviews show that numerous programs did not have manufacturing processes under control when they moved into production

• Four DOD programs—the H-1 Helicopter upgrade program; Exoatmoshperic Kill Vehicle; Joint Air-to-Surface Sub-Munition; and Electro-mechanical Air Launch System for the CVN-78 ship program—were analyzed for manufacturing problems

• Sources of manufacturing problems include: inattention to manufacturing during planning and design; poor supplier management; and lack of workforce knowledge and skills

What are the manufacturing problems?


What is DOD doing to address the manufacturing problems?

• OSD Manufacturing Technology Program, in collaboration with the Joint Service/Industry Group proposed MRLs to be the common standard to identify, communicate, and manage manufacturing maturity and risk; support revisions made in 5000.02 (Dec 08)

• DOD developed MRL Deskbook (July 2010), set of MRL definitions, web-based tools (http://dodmrl.org/), and conducted pilot tests

• MRLs were developed from a common body of defense, industry, and academic manufacturing knowledge, including:• Willoughby templates; Military specifications and guidance; ISO

9000 quality requirements; Automotive standards; Lean standards; and MIT’s supplier model



• Army and Air Force MRL pilot programs show significant savings, reduced unit cost, improved production schedules, and better performance on selected S&T efforts and some major defense acquisition programs

• Army claims significant savings in cost avoidance:• Helmet shells, headgear, body armor—savings from

lower cost, and reduced labor • Micro-Electro-Mechanical Systems demonstrated cost,

schedule, performance benefits• Embedded Sensors derived savings in cost/benefit,

decreased weight, increased performance (strength)



• Air Force reports savings in cost avoidance on F-35 aircraft, F-135 propulsion engine, AMRAAM, MQ-9 Reaper, X-band radar, and sensors for tactical systems• F-135 engine thrust accelerated transition by four years• AMRAAM increased production from less than 12 to more

than 30 units a month• MQ-9 identified five risk areas to focus improvements to

meet production workload goals• MRL adoption has been slow since it was introduced in 2005:

• Two Air Force Centers issued policy on MRLs (ASC/AAC)• Several Army Commands require MRLs on S&T programs • MRLs offered in various training courses—DAU and AFIT


How does MRLs compare to practices used by commercial firms?

MRLs incorporate many best practices of leading firms• Plan and gain knowledge about manufacturing early• Emphasizes knowledge about producibility of the product at

every stage of development• Demonstrate manufacturing processes in a production

relevant environment• Emphasize relationship with critical suppliers


How does MRLs compare to practices used by commercial firms?

• Assessments of manufacturing technology and risk, supply chain, production facilities, materials and workforce

• Requirements for deliverables at each gate (GE Aviation)• Requirements for realistic cost/schedule estimates for their products • Mature processes by the manufacturing start (GE Healthcare)• Proving out tooling, equipment and processes before beginning

manufacturing (GE Aviation, Healthcare)• Working closely with suppliers, requiring high quality standards for

parts and supplies

• Requirements for multidisciplinary product development teams• MRLs are largely consistent with commercial best practices, but strong

criteria for process capability and control are required too late in the process to achieve the desired effect—after the production decision


What are the challenges and barriers that DOD faces in implementing MRLs?

• Acceptance of MRLs is growing in DOD and defense industrial base• Several Army commands use them on all S&T efforts with

manufacturing elements; new programs required to use them • Two of three Air Force product centers issued local policy that

mandate their use on all manufacturing-intensive programs• A number of defense contractors have implemented them and

have drawn benefits from them• Policy to standardize MRLs has not yet been addressed

• DOD is working on how to implement MRLs, but support for policy remain unclear

• Manufacturing workforce knowledge gaps may impede MRL adoption.• Each of the military services and DCMA are working on this


GAO Recommendations

• Require the assessment of manufacturing readiness across DOD programs using consistent MRL criteria as a basis for measuring, assessing, reporting, and communicating manufacturing readiness and risk on science and technology transition projects and acquisition programs [Partial concur]

• Direct DDR&E to examine strengthening the MRL criteria related to the process capability and control of critical components and/or interfaces prior to the Milestone C low rate initial production decision [Concur]

• Direct the DDR&E to assess the need for analytical models and tools to support MRL assessments [Concur]

• Assess the adequacy of the manufacturing workforce knowledge and skills base across the military services and defense agencies and develop a plan to address current and future workforce gaps [Concur]

• The report GAO-10-439, BEST PRACTICES: DOD Can Achieve Better Outcomes by Standardizing the Way Manufacturing Risks Are Managed is available for download at no cost on the GAO Web site at: http://www.gao.gov/Products/GAO-10-439

• Contact information: Mike Sullivan, [email protected] or 202.512.4841 John Ortiz, [email protected] or 256.922.7519

gao manufacturing presentation

Documents

key knowledge

knowledge points

inadequate knowledge

level of knowledge

knowledgebased approach

gao manufacturing study

key junctures knowledge

selected dod programs