Process Improvement in the Aerospace Industry

CMMI and Lean Six Sigma

Warren Scheinin

USC CS510

2

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches– Capability Maturity Model Integrated– Lean Six Sigma

• Northrop Grumman Approach

3

NDIA Top 5 Systems Engineering Issues (2006)

• Key systems engineering practices known to be effective are not consistently applied across all phases of the program life cycle.

• Insufficient systems engineering is applied early in the program life cycle, compromising the foundation for initial requirements and architecture development.

• Requirements are not always well-managed, including the effective translation from capabilities statements into executable requirements to achieve successful acquisition programs.

• The quantity and quality of systems engineering expertise is insufficient to meet the demands of the government and the defense industry.

• Collaborative environments, including SE tools, are inadequate to effectively execute SE at the joint capability, system of systems (SoS), and system levels.

Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by Mr. Robert Skalamera

4

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches– Capability Maturity Model Integrated– Lean Six Sigma– Agile

• Northrop Grumman Approach

5

Heritage of Standards for Systems Engineering

EIA / IS 632

ISO/IEC 15288

Mil-Std-499BMil-Std-

499A

1994

1994

1994

19982002

1974

(Not Released)Mil-Std-499

1969

(Trial Use)IEEE 1220

1998

(Full Std)

LegendSupersedesSource for

EIA632

1998

EIA/IS 731

SE CM

IEEE 1220

(Full Std)(Interim Standard)

2002

CMMI-

SE/SW/IPPD

(Interim Standard)

(FDIS)

ISO/IEC 19760

2002

(PDTR)

2002

ISO/IEC 15504(FDIS)

Standards for Systems Engineering, Jerry Lake, 2002

6

The Frameworks QuagmireSarah A. Sheard, Software Productivity Consortium

http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp

7

Two Complimentary Approaches to Process Improvement

Data-Driven (e.g., Lean Six Sigma)

• Clarify what your customer wants (Voice of Customer)

– Critical to Quality (CTQs)

• Determine what your processes can do (Voice of Process)

– Statistical Process Control

• Identify and prioritize improvement opportunities

– Causal analysis of data

• Anticipate your customers/ competitors (Voice of Business)

– Design for Six Sigma

Model-Driven (e.g., CMMI)

• Determine the industry best practice– Benchmarking, models

• Compare your current practices to the model

– Appraisal, education

• Identify and prioritize improvement opportunities

– Implementation– Institutionalization

• Look for ways to optimize the processes

8

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches– Capability Maturity Model Integrated– Lean Six Sigma

• Northrop Grumman Approach

9

What is the Capability Maturity Model Integrated?

• The CMMI is a collection of industry best-practices for engineering, services, acquisition, project management, support, and process management

– Developed under the sponsorship of DoD– Consistent with DoD and commercial standards

Three Constellations sharing common components and structure

• CMMI for Development - used by engineering organizations

• CMMI for Acquisition - used by buyers (e.g., govt. agencies)

• CMMI for Services - used by service providers (e.g., help desk)

10

Basic Building Blocks – 22 Process Areas

Project Management• Project Planning• Project

Monitoring and Control

• Supplier Agreement Management

• Integrated Project Management)

• Risk Management

• Quantitative Project Management

Engineering• Requirements

Development• Requirements

Management• Technical

Solution• Product

Integration• Verification• Validation

Support• Configuration

Management• Process and

Product Quality Assurance

• Measurement and Analysis

• Decision Analysis and Resolution

• Causal Analysis and Resolution

Process Management

• Organizational Process Focus

• Organizational Process Definition

• Organizational Training

• Organizational Process Performance

• Organizational Performance Management

Implemented byeach project

Implemented bythe organization

11

Expected Practices Provide Guidancefor Implementation & Institutionalization

SG 1 Establish Estimates SP 1.1 Estimate the Scope of the Project SP 1.2 Establish Estimates of Work Product

and Task Attributes SP 1.3 Define Project Lifecycle Phases SP 1.4 Estimate Effort and Cost

SG 2 Develop a Project Plan SP 2.1 Establish the Budget and Schedule SP 2.2 Identify Project Risks SP 2.3 Plan Data Management SP 2.4 Plan the Project’s Resources SP 2.5 Plan Needed Knowledge and Skills SP 2.6 Plan Stakeholder Involvement SP 2.7 Establish the Project Plan

SG 3 Obtain Commitment to the Plan SP 3.1 Review Plans That Affect the Project SP 3.2 Reconcile Work and Resource Levels SP 3.3 Obtain Plan Commitment

GG 2 Institutionalize a Managed Process GP 2.1 Establish an Organizational PolicyGP 2.2 Plan the ProcessGP 2.3 Provide ResourcesGP 2.4 Assign ResponsibilityGP 2.5 Train PeopleGP 2.6 Manage ConfigurationsGP 2.7 Identify and Involve Relevant

StakeholdersGP 2.8 Monitor and Control the ProcessGP 2.9 Objectively Evaluate AdherenceGP 2.10 Review Status with Higher Level

ManagementGG 3 Institutionalize a Defined Process

GP 3.1 Establish a Defined ProcessGP 3.2 Collect Improvement Information

Project Planning – Implementation Project Planning - Institutionalization

12

Practice Ratings for the Organization/Projects

13

How is the CMMI Used for Process Improvement?

www.sei.cmu.edu/ideal/

IDEAL Model

14

Typical CMMI Benefits Cited in Literature

• Reduced costs– 33% decrease in the average

cost to fix a defect (Boeing)– 20% reduction in unit

software costs (Lockheed Martin)

• Faster Schedules– 50% reduction in release

turnaround time (Boeing)– 60% reduction in re-work

following test (Boeing)

• Greater Productivity– 25-30% increase in productivity

within 3 years (Lockheed Martin, Harris, Siemens)

• Higher Quality– 50% reduction of software

defects (Lockheed Martin)

• Customer Satisfaction– 55% increase in award fees

(Lockheed Martin)

15

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches– Capability Maturity Model Integrated– Lean Six Sigma

• Northrop Grumman Approach

16

What is Lean Six Sigma (LSS)?

• Lean Six Sigma is a powerful approach to improving the work we do

• LSS improvement projects are performed by teams

• Teams use a set of tools and techniques to understand problems and find solutions

• Lean Six Sigma integrates tools and techniques from two proven process improvement methods

+

17

Six Sigma

• A management philosophy based on meeting business objectives by reducing variation

– A disciplined, data-driven methodology for decision making and process improvement

• To increase process performance, you have to decrease variation

Defects Defects

Too early Too late

Delivery Time

Reduce variation

Delivery Time

Too early Too late

Spread of variation too wide compared to

specifications

Spread of variation narrow compared to

specifications

• Greater predictability in the process

• Less waste and rework, which lowers costs

• Products and services that perform better and last longer

• Happier customers

18

DMAIC Roadmap

Define ControlAnalyze ImproveMeasure

Define project scope

Establish formal project

Identify needed data

Obtain data set

Evaluate data quality

Summarize& baseline data

Explore data

Characterize process & problem

Identify possible solutions

Implement (pilot as needed)

Define control method

Implement

Update improvement project scope & scale

Document

Select solution

Evaluate

[Hallowell-Siviy 05]

19

DMAIC Toolkit

Benchmark Contract/CharterKano ModelVoice of the CustomerVoice of the BusinessQuality Function Deployment

GQIM and Indicator TemplatesData Collection MethodsMeasurement System Evaluation

Statistical Controls:

Control Charts Time Series methods

Non-Statistical Controls: Procedural adherence Performance Mgmt Preventive measures

ControlImproveMeasureDefine AnalyzeDesign of Experiments

ModelingANOVATolerancingRobust DesignSystems ThinkingDecision & Risk AnalysisPSM Perform Analysis Model

Cause & Effect Diagrams/ Matrix

Failure Modes & Effects AnalysisStatistical InferenceReliability AnalysisRoot Cause Analysis, including 5 WhysHypothesis Test

20

Design for Six Sigma (e.g., DMADV)

Define VerifyAnalyze DesignMeasure

Define project scope

Establish formal project

Identify customers

Research VOC

Benchmark

Quantify CTQs

Explore data

Design solution

Develop detailed design

Develop pilot

Evaluate pilot

Scale-up design

Predict performance

Document

Refine predicted performance

21

Lean

• Series of tools and techniques refined by Toyota and called the “Toyota Production System”

– Called “Lean” by Womack, Jones and Roos in The Machine That Changed the World

• Focused on increasing efficiency by eliminating non-value added process steps and wasteful practices

• Being adopted world-wide by both manufacturing and transactional based organizations

• Utilizes tools like “Value Stream Mapping,” “Just in Time” and “Kaizen”

LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME

22

Wastes in Production

•

Typesof

Waste

CORRECTION

WAITING

PROCESSING

MOTION

INVENTORYCONVEYANCE

OVERPRODUCTION

Repair orRework Any wasted motion

to pick up parts or stack parts. Also wasted walking

Wasted effort to transportmaterials, parts, or finished goods into or out of storage, or between processes.

Producing morethan is needed before it is needed

Maintaining excessinventory of raw mat’ls,parts in process, orfinished goods.

Doing more work thanis necessary

Any non-work timewaiting for tools, supplies, parts, etc..

23

Organizational Adoption:Roles & Responsibilities

• Champions – Facilitate the leadership, implementation, and deployment

• Sponsors – Provide resources

• Process Owners – Responsible for the processes being improved

• Master Black Belts – Serve as mentors for Black Belts

• Black Belts – Lead major Six Sigma projects– Typically requires 4 weeks of training

• Green Belts – Lead minor Six Sigma teams, or serve on improvement teams under a Black Belt

– Typically requires 2 weeks of training

24

A Typical Lean Six Sigma Project in Aerospace

The organization notes that systems integration has been problematic on past projects (budget/schedule overruns)

A Six Sigma team is formed to scope the problem, collect data from past projects, and determine the root cause(s)

The team’s analysis of the historical data indicates that ineffective peer reviews are leaving significant errors to be found in test

Procedures and criteria for better peer reviews are written, using best practices from past projects

A pilot project uses the new peer review procedures and criteria, and collects data to verify they solve the problem

The organization’s standard process and training is modified to incorporate the procedures and criteria, to prevent similar problems on future projects

25

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches– Capability Maturity Model Integrated– Lean Six Sigma

• Northrop Grumman Approach

26

Northrop Grumman Approach:Mission Success Requires Multiple Approaches

Process Effectiveness

Program Effectiveness

MissionAssurance & Enterprise Excellence Operations

Effectiveness

Dashboards for Enterprise-Wide Measurement

Communications & Best-Practice Sharing

Robust Governance Model (Policies,

Processes, Procedures)

Risk ManagementSystems EngineeringIndependent Reviews

& Cost EstimatesTraining, Tools, &

Templates

CMMI Level 5 for Software, Systems, and

ServicesISO 9001 and AS-9100

CertificationSix Sigma

27

Organizational Infrastructure Required for CMMI Level 3

Policies, Processes,Templates & Tools

Best-Practice Libraries

Process Group

Audits & AppraisalsMeasurement RepositoriesPredictive Modeling

Developing and maintaining mature processes requires significant time and investment in infrastructure

Process ImprovementTraining Program

Communications

0

5

10

15

20

25

1 11 21 31 41 51 61 71

UCL

_X

Defe

cts

per c

ompo

nent

Component #

0

5

10

15

20

25

1 11 21 31 41 51 61 71

UCL

_X

Defe

cts

per c

ompo

nent

Component #

28

Northrop Grumman Approach:Institutionalizing Our Improvements

InternalBest

Practices

ISO/AS9100 Findings

CMMI Appraisal Findings

Policy

Procedures

Process

eToolkit PAL

WorkbenchStartIt! My MS Portal

Tools

Information

Checklists and Guides

Templates and Examples

Disposition

Independent Audits

• Systems/ Software Engineering Process Group

• QMS Working Group

• Program Management Advisory BoardLessons

Learned & Metrics

Analysis

Customer Comments

Configuration Control Board

ExternalBest

Practices

Industry Standards

Six Sigma Projects

We systematically analyze quality and process data and trends to determine how to improve our

processes

We improve our process assets based on internal and external

best practices Deploye

d to program

s

Increasing program efficiency

msCAS

PCDB

29

Northrop Grumman Approach:Lessons Learned

• Multiple improvement initiatives helps encourage a change in behavior as opposed to “achieving a level”

– Reinforces that change (improvement) is a way of life

• Benefits results from institutionalizing local improvements across the wider organization

– CMMI establishes the needed mechanisms

• CMMI and Lean Six Sigma compliment each other– CMMI can yield behaviors without benefits– Lean Six Sigma improvements based solely on data may miss innovative improvements

(assumes a local optimum)

• Training over half the staff as Lean Six Sigma Green Belts has resulted in a change of language and culture

– Voice of Customer, data-driven decisions, causal analysis, etc.– Better to understand/use tools in everyday work than to adopt the “religion”