metrics and performance measurement

8/20/2019 Metrics and Performance Measurement

1/47

Metrics and Performance MeasurementSystem for the Lean Enterprise

Professor Deborah Nightingale

October 24,2005


2/47

Overview

• Metrics and Performance measurement

•

Why measure? What is performance measure? What are goodmetrics?

• Performance measurement and Lean Transformation

• Current practices and Performance measurement

frameworks

•

Performance measurement system for the lean

enterprise

ESD.61J / 16.852J: Integrating the Lean Enterprise © Deborah Nightingale, 2005 Massachusetts Institute of Technology Page 2


3/47

Why Measure?

• “Performance control systems can

serve two purposes, to measure

and to motivate.”

- H. Mintzberg,

The Structure of Organizations, 1979

•

“The firm becomes what itmeasures”

- Hauser and Katz,

You are What You Measure, 2002

•

Metrics serve multiple purposes!

Strategy

Feed Decision

Back Making

Measures Outcomes Actions

Robert Nixon (1990)



4/47

What is performance

measurement?• Performance measurement is the process of measuring

efficiency, effectiveness and capability, of an action or a

process or a system, against given norm or target.

• Effectiveness is a measure of doing the right job - the extent to

which stakeholder requirements are met.

•

Efficiency is a measure of doing the job right - howeconomically the resources are utilized when providing a given

level of stakeholder satisfaction.

• Capability is a measure of ability required to do both the job

right and right job, in the short term as well as the long term.This can be tangible, such as, resources, technology, or

intangible, such as a corporate culture.



5/47

Characteristics of good metrics

• Metrics are meaningful, quantified measures

•

Metric must present data or information that allowsus to take action

•

Helps to identify what should be done

• Helps to identify who should do it

• Metrics should be tied to strategy and to “core”

processes - indicate how well organizational

objectives and goals are being met

• Metrics should foster process understanding and

motivate individual, group, or team action and

continual improvement.



6/47

Broad Criteria

A “Good” Metric Satisfies Three

1. Strategic• Enable strategic planning and then drive deployment of the actions

required to achieve strategic objectives• Ensure alignment of behavior and initiatives with strategic objectives

• Focus the organization on its priorities

2. Quantitative

•

Provide a clear understanding of progress toward strategic objectives• Provide current status, rate of improvement, and probability of

achievement

• Identify performance gaps and improvement opportunities

3. Qualitative• Be perceived as valuable by your organization and the people involved

with the metric



7/47

Metric Elements

Metric Elements

Title

Objective/purpose

Scope

Target

Formula

Units of measure

Frequency

Data source

Owner

Comments

Explanation

Use exact names to avoid ambiguity

The relation of the metric with the organizational objectives must be clear

States the areas of business or parts of the organization that are included

Benchmarks must be determined in order to monitor progress

The exact calculation of the metric must be known

What is/are the unit(s) used

The frequency of recording and reporting of the metric

The exact data sources involved in calculating a metric value

The responsible person for performance of that part of the organization,

collecting data and reporting the metric

Outstanding issues regarding the metric

Adapted from Neely, A., et al. (1995a) Performance measurement system design. 15, 80.



8/47

Role of Performance Measurement

• Monitoring • Measuring and recording actual performance

•

Control• Identifying and attempt to close the gap between planned target

and actual performance

• Improvement•

Identify critical improvement opportunities

• Coordination• Information for decision making – Leading Indicators

•

Internal communication across processes•

External communication with stakeholders

• Motivation•

Align Behavior and encourage transformation

Source: Vikram Mahidhar



9/47

© Deborah Nightingale, 2005 Massachusetts Institute of Technology ESD.61J / 16.852J: Integrating the Lean Enterprise Page 9

Transformation to the Lean Enterprise

Functional Enterprise

6

5

4

3

2

1

7

1

2 2

3 3

6

5

4

3

2

1

7

1

2 2

3 3

6

5

4

3

2

1

7

1 1

2 2

3 3

S a l e

s

E n g i n

e e r i n g

Manufacturing

P u r c h a s i n g

R & D

Value Delivery

Stakeholder Objectives

Customer

EmployeesSuppliers

Shareholders Community

Enterprise Leadership &Governance Processes

Enabling InfrastructureProcesses

Lifecycle Processes

Processes

S a l e s

C

U

S

T

O

M

E

R

Centers of

Excellence

CEO

Executive board

E n g i n e e r i n g

M a n u f a c t

u r i n g

R & D

Process Enterprise Lean Enterprise



10/47

Metrics Challenge Today

• Hierarchical organizational architectures givingway to networked enterprise architectures

•

The evolving structure and dynamics of networked enterprises display immensecomplexity

• Metrics response to such complexity has been adisappointment:

• Hierarchical metrics mindset still continues

•

Response to greater complexity has been a metrics explosion

• Challenge: How best to design metrics systems

for networked enterprises?ESD.61J / 16.852J: Integrating the Lean Enterprise © Deborah Nightingale, 2005 Massachusetts Institute of Technology Page 10


11/47


ROIC = Return on Invested CapitalORPIC* = Operational Readiness per Invested Capital

Performance measurement system for

Lean Transformation

T

r an sf or m

a t i on ov

er T i m e

T

r an sf or m

a t i on ov

er T i m e

Knowledge and Behavior Metrics and Assessments

New approaches (training andintroduction of new methods)

Engagement in “LAI-venue” with like-

minded people

Enterprise simulation, Lean Now and

LAI knowledge area teams

Local efforts and new capabilities

Industry Government

Skills, training hours, certification,lean deployment, joint assessments

and efforts

New local behavior Shift in thinking and behavior

New routines and ways of doing

business

Organization and group culture change

Cycle time, quality, WIP, on-timedelivery, customer satisfaction,employee turnover and attitude,

organizational climate and LESATmaturity

Local results and visible indicators

Industry Government

New enterprise capability

Integration of processes/methods &

tools supporting transformation across

the value stream enabling new

enterprise capabilities

ROIC

Enterprise impact and results

Industry GovernmentORPIC

Adapted from : Noel Nightingale, 2004


12/47

Generic Enterprise Management Process

Value delivery

Management Assessment j

Shareholder Value

Customer Satisfaction

Employee Satisfaction

Other stakeholders

Strategy Formulation

- Competitive Intelligence

- Internal Assessment

Strategic and Operational Planning

Resource Allocation Plan

Strategy ExecutionCascaded Ob ectives

VSM and Project Prioritization

Performance Mgmt Process

Communication

Operations

Balanced Score Cards

Operations Management

Financial ManagementHuman Resource

Information Systems

External

Environment



13/47

Strategic Metrics

• ROIC (Return on

Invested Capital)• Economic Value Add

(EVA)

• Net Operating Profit

• Inventory Turnover

• Revenue

• Cash flow

• Market Position

• Wall Street Expectations

Operations

Management Assessment

Value delivery

Strategy Execution






Efficiency metrics and Lagging indicators



14/47

Tactical Metrics

• Financial Turnover

•

Budget/Cost andExpenses

• Cost of quality

• Productivity

• Supply Chain

Excellence

Operations

j

Value Delivery





Communication

• Regulatory and social

compliance

Accuracy and timeliness of reporting and controlESD.61J / 16.852J: Integrating the Lean Enterprise © Deborah Nightingale, 2005 Massachusetts Institute of Technology Page 14


15/47

Operational Metrics

• Safety

•

Quality

• Environment

• Cost/Manufacturing

Efficiency• Delivery

• Time to market

•

Education anddevelopment

• Time to Hire

Value Delivery

Strategy Formulation Strategy Execution

Operations



Financial Management

Human ResourceInformation Systems



16/47

Value Delivery Metrics

•

Stock Price

• Revenue

• On time delivery

• Customer satisfaction

and loyalty

• Employee Satisfaction

• New product

Introduction

OperationsValue Delivery

Strategy Formulation Strategy Execution

Shareholder Value


Competition

Macro Economic Trends



17/47

Management Process

Value delivery

Management Assessment j

Time lags - Performance

Shareholder Value


Employee Satisfaction

Other stakeholders









Communication

Operations



Financial Management

Human Resource

Information Systems

External

Environment


Ti l P f


18/47

Management Process

Strategy

Formulation

Strategy

Execution

Operations Reporting

Yearly

Month

Day

Quarter

Time lags - Performance



19/47

Performance Measurement &

Management Frameworks

• Balanced Scorecard – More than 50%

companies have implemented in the US• Performance Prism

•

European Foundation QualityFramework

•

X-Matrix (EVSMA)



20/47


Balanced Scorecard/Strategy Map

Long-Term

Shareholder Value

Human capital

Information capital

Organization capitalCulture Leadership

Learning

& Growth

Operations

-Supply

-Production

-Quality

-Logistics

Customer Relationship

-Selection

-Acquisition

-Retention

-Growth

Innovation Process

-Opportunity Ident.

- R&D Portfolio

- Design/Development

- Time to market

Regulatory and Social

- Environment

- Safety and Health

- Employee development

- Community

Internal

Processes

Price Quality Availability Selection Functionality Services PartnershipCustomer

Financial Improve CostStructure

Increased Asset

Utilization

Expand Revenue

Opportunities

Enhance

Shareholder value

Growth StrategyProductivity Strategy

Adapted from : Kaplan and Norton, (2000) “Having Trouble with your strategy? Then Map it!” Harvard Business Review Sept_Oct 2000

P i P f th


21/47

Primary Purposes of the

Balanced Scorecard

• Align a balanced set of performance metrics

with business strategy and vision

• Provide management and work teams with

the information necessary and sufficient to

meet their objectives and goals

• Create “line-of-sight” at lower levels of the

organization• Foster and support process continuous

improvement initiatives


D i d Ch t i ti f P f


22/47

Desired Characteristics of Performance

Measurement Systems

• Performance Measures should support the

strategic intentions of the organizations

• Managers at all levels should understand

both drivers and results of their activities.

• Explicating Cause-Effect relationships

between drivers and results


Lean Enterprise Metric


23/47


S t a k e h ol d e r

V a l u e

Lean Enterprise Metric

Interdependencies

L i f e C y c l e

P r o c e s s e s

E

n a b l i n g

I

n f r a s t r u c t u r e

P

r o c e s s e s

E n t e r p r i s e

L e a

d e r s h i p

P r o

c e s s e s

Production

Quality

Control

Logistics

Training

Change

Management

Strategic

Planning

Process

Yield/CostQuality

Financial

Management

Financial

HealthDelivery

Customer

Value


Process Control View


24/47

Process Control View –

Performance Measurement System

Process

ComparisonFunction

ing

Output Input

Action

Targets

Performance

measures

Strategy

Making

Behavior

Strategic

Operational

Strategic

Tactical

F

e e d

b a c k

Control

Evaluat Plan

Stakeholders

DecisionObjectives



U d t di P f M t


25/47

Understanding Performance Measurement

Metric Cluster

Metric Set

Individual Metric

Illustrative

Structures – Causal Models

Example




26/47

“…although there may be a potentially long list of non-financial

indicators, individual firms have to be selective by linking

explicitly their choice of indicators to their corporate strategy.”

No One “Right” Set of Metrics

• The balanced scorecard has to be tailored to

each specific company

• The resulting scorecard of indicators should

be driven by the firm’s strategy if it is not to

consist merely of a listing of indicators:

indicators, individual firms have to be selective by linking

“…although there may be a potentially long list of non-financial

explicitly their choice of indicators to their corporate strategy.”

R.S. Kaplan and D.P. Norton, Harvard Business Review , January-February, 75-85 (1996)


Assessing a Performance


27/47

Assessing a Performance

Measurement System

• Does it clearly define what constitutes business excellence?

• Does it provide the information required to set aggressive yet

achievable strategic objectives and stretch goals?• Does it accurately portray our progress and probability of

achieving both long-term strategic objectives and near-termmilestones?

•

Does it identify the root causes of barriers?

• Does it focus the organization on the priority improvement needs?

• Does it drive the behavior and actions required to achieve the

objectives?• Does it align work with value?

• Is it easy to use?

•

Does it involve everyone?


Metrics Will Change Over an Item’s


28/47

Raytheon Systems, 1998

Metrics Will Change Over an Item s

Life CycleEntity

Emerging

Growth

Mature

Declining

•

•

•

•

Product Concept

Development

Phase

•

•

•

•

•

•

•

•

•

•

•

•

Recognition

Learn

Practi

Expert

•

•

•

•

• Use

• Apply

•

• Deployment

• Teach

•

• Combine and evaluate

Phases Attributes

Business Cash flowCompetitive advantage

Market share

Critical Mass

In market

-out

Creative backlogPotential product revenue

Cost per feature

Time to market

Performance requirements

Predicted product quality

Design tocost

ProfitabilityMarket expansion rate

Volume impact on cost

Inventory

Customersupport

Core

Competency

ce

Inventory of skills and capabilities

Competitive advantage

Acquire knowledgeCycles of learning

Levels of use in organization

Leverage advantage



29/47

t

Process and Metric Maturity Model

That

1

Busii

consci ing of management.

i i l

core processes

2Alignment

CoreProcessesManaged

Metrics reinforce & leveragei

3 HorizontalAlignment

CoreProcessesIntegrated

4 TotalAlignment

EnablingProcesses

Integrated

Allorganizational, geog )

i j

& optimize the whole

ith

i

5Holisticimulated

g gy setting(e.g., nodal influence,

LevelProcess

Metrics are ad hoc andprimarily results oriented.Little or no process focus.

which exists is primarily directedinternally toward local operations Initial Initial

ness process management, whichbegins & ends w th the customer isestablished, in control, and in the

ous think

Process metrics added &ntegrated w th resu t metrics

Metrics aligned betweenstrategy & daily activities in

Vertical

Common process language & specsIntegrated core processes allow a

seamless flow of work across processboundaries

activ ties across all corebusiness processes

Local interests are subordinatedto the good of the whole

metrics (process, results,raphic, etc.

align with strateg c ob ectives,provide competitive advantage

Support processes are integrated wand enable core business processesto provide competitive advantage.

Customer-focused processmanagement is appliedunconsc ously

OptimizingMetric-driven actions sdurin strate process toensure organizational alignmentbefore metrics are implemented

Process management has providedworld-class competitiveadvantageagile & forward looking)

MetricMaturityMaturity


Ray heon Systems, 1998


30/47

Level One: Initial

• Enterprise does not manage its business

with a process focus• Many metrics sub-optimized by local organizational

interests rather than having them aligned with

customer interests and with the strategic

objectives of the enterprise

• Organizations measure the results of past actions

•

Results-oriented metrics cannot provide theleading indicators needed for timely corrective

action to change outcomes



31/47

Vertical alignment is the alignment and reinforcement of strategic

objectives with supportive goals and progress measures at alllevels of the organization.

Level Two: Vertical Alignment

Definition

Vertical alignment i li i ics the a gnment and re nforcement of strateg

objectives with supportive goals and progress measures at alllevels of the organization.

• The business enterprise applies a process focus so it can

measure leading indicators of the expected process output• Defective process output is viewed as a process-capability

problem, not a people problem

•

Carefully chosen metrics ensure that all levels of theorganization align with strategic objectives



32/47

A core process related to product development activities

might be documented, be in control (repeatable), be

consistently deployed across the organization, and havemeasurable improvement gains. If so, that process is

probably at or near Level 2 maturity. If the metrics

indicate variations in the process results, then they are

still at Level 1 because the process is not in control.

Level Two Example

), be

l

A core process related to product development activities

might be documented, be in control (repeatable

consistently deployed across the organization, and havemeasurab e improvement gains. If so, that process is

probably at or near Level 2 maturity. If the metrics

indicate variations in the process results, then they are

still at Level 1 because the process is not in control.



33/47

Level Three: Horizontal Alignment

Two phases:

1. The global optimization of work flow across all process

boundaries. These boundaries become transparent to the flowof work. Metrics are customer-focused and assess theenterprise-level capability of a process to provide value from thecustomer’s perspective.

2. The global optimization of work flow across all organizationalboundaries that support or use a particular process. Metrics arecustomer-focused and assess how well the infrastructure

enables execution of customer-focused processes.


L l Th E l


34/47

The enterprise may have several core customer-related processes

such as winning new business and developing new products. Also,

the enterprise may have many functions that support or executethese core processes.

Level Three Example

l

lso,

The enterprise may have several core customer-re ated processes

such as winning new business and developing new products. A

the enterprise may have many functions that support or execute

these core processes.

Level 3 characteristics include:

• Integrated core processes that customers see as seamless

• Minimized hand-offs or delays as work moves among

processes and sub-processes

• Management focus primarily on early process activities in a

product life cycle• Metrics insure local organizational interests (functional or

business unit) are subordinated to customer needs and what is

best for the entire business


L l F T t l Ali t


35/47

Total alignment is the synergistic interaction of metrics from all

support processes with metrics from all core process to reinforcethe strategy and to drive business excellence.

Level Four: Total Alignment

Definition

Total alignment is the synergistic interaction of metrics from all

support processes with metrics from all core process to reinforcethe strategy and to drive business excellence.

• All employees clearly see where the business is headed

and how they can make a difference.

• Horizontal integration (Level 3) provides employees with

“line of sight” to customer value. Dramatic performance

improvements can occur at this level.

•

Total enterprise-level alignment (Level 4) is required to

overcome the major systemic barriers to great performance

and to embed the long-term gains into the fabric of the

organization’s culture.


L l F E l


36/47

At Level 4, the enterprise begins asking howenabling processes create competitive

advantage for the core customer-related

processes, rather than what they do to improve

themselves.

Level Four Example

l

l

At Leve 4, the enterprise begins asking howenabling processes create competitive

advantage for the core customer-re ated

processes, rather than what they do to improve

themselves.

• Total enterprise alignment is required to overcome the major

systemic barriers to great performance


L l Fi O ti i i


37/47

Level Five: Optimizing

•

From a process perspective, the enterprise will have muchgreater influence on the market than its size might indicate. The

agile and forward-looking enterprise will be able to foresee

events and respond to those events before they occur.

• From a metrics perspective, the enterprise will be able not only

to simulate and predict the outcome of a strategy before its

deployment, but also to predict the effect of specific metrics onthe outcome of that strategy before choosing metrics.


Case Study: Nike


38/47

Case Study: Nike

• The study focuses on European Operations of Nike.

• The company is organized around three lines of business:Apparel – 60,000 SKU, Footwear – 25000 SKU, and Equipment 1000 SKU.

•

90% of the business comes from 6 countries in Europe.

• The product life cycles are short.

• Uncertainty of demand is an important characteristic.

•

Company was continuously improving its supply chainmanagement through a better integration of operations acrosssubsequent echelons and separate functions in the valuechain.

•

As part of these efforts management decided to assign someof their resources for improvement projects to performancemanagement. The integration of various local performanceindicators into a company-wide consistent system wasrequired.

Lohman, C., et al. (2004) “Designing a performance measurement system: A casestudy”. European Journal of Operational Research, v.156, pp.267-286.


Case Study: Nike


39/47

Case Study: Nike

The project objectives:

• To develop a set of high-level performance metricstailored to the specific business needs for use by thesenior supply chain management team, i.e. Operations,

while including existing local metrics as much as possibleand sensible.

• To design a format, i.e. a scorecard, displaying the metric

scores at the level of Nike as well as that of the businessunits.



Case Study: Nike


40/47

Case Study: Nike

• A structure with three layers is used for displaying the information

• High Level

• Mid Level

• Lower Level

•

Normalization

• All the metrics were normalized based on a linear 0–10 scale.

•

Usage

•

Senior supply chain management team : director Operations, thefunctional directors of Transportation, Warehousing, and CustomerService and the three business unit Operations directors. They will usethe scorecard on a monthly basis to facilitate review of theorganizations performance.

• General Manager Nike Europe to facilitate a quarterly review of

Operations

• Maintenance

• Monthly scorecard reviews

• Yearly redesign of the scorecard and its contents when launching new

business plans



Case Study: Nike


41/47

Case Study: Nike

Developing and Embedding Performance Measurement System

• Defined a metrics dictionary with some 100 fixed metrics combined with anoverview of existing initiatives, while being much more flexible regarding

the structure.•

Used a hierarchical structure within each cluster, thereby using so-called‘‘engineered indicators’’ i.e. metrics based on two or more lower-levelmetrics.

•

Brought together all people working on parallel initiatives in the field of PMwithin the organization.

• Experimented with different ways of clustering the metrics and presentingthe scorecards.

•

Gathered the data required for actually measuring and reporting all metrics

in a reliable way.•

One person of the cross-functional initiatives group managed theperformance reporting process.

• The PM manager has authority to make changes in the definition, to makesure that metrics remain consistent between various areas and with theglobal scorecard. Lohman, C., et al. (2004) “Designing a performance measurement system: A casestudy”. European Journal of Operational Research, v.156, pp.267-286.


Case Study: Air Force Sustainment


42/47

yDecline in Mission Capability Rates

“Material condition of an aircraft indicating it can perform at leastone and potentially all of its designated missions. Mission-capableis further defined as the sum of full mission-capable and partialmission-capable. Also called MC”. - DOD Definition of Mission Capability

S pe cial M is s ion

9 5

9 0

8 5

8 0

Fighters Bombers

SOURCE: Compiled from data in

National Research Council, Aging7 5 Avionics in Military Aircraft

(Washington, DC: National

Academy Press, 2001)7 0

6 5

6 0

5 5

5 0

Year 8 2

8 5

8 8

9 1

9 4

9 7


Air Force Sustainment System


43/47

i ise Page 43

Air Force Sustainment System

ll l

i ision:

/

(Fli line

i

i

CSI

le

thi i

)

No

) &

le

)

il le

Pi

l

p

No

i

i

i

ia

ll

i

w i

w i

i

i

i

i ipeli

A i

© Deborah N ghtingale, 2005 Massachusetts Institute of TechnologyESD.61J / 16.852J: Integrating the Lean Enterpr

Information F ows: Process:Physica Part F ows: Status:

Ent ty: Dec

LEGEND

ALC DEPOT-LEVE L

TRANSPORTATION

BASE-LEVELBase Repair ght

Mantenance)

Base

Supply

Base Traff c Mgmt

Express Carr er

CRI

Repaired

at Depot

ComponentFails

Depot

TMO

Repairab

s staton?

(RTS

(NRTS

Possib

MICA P

Yes

(RTS

Parts ava abat base?

No &

Possible

MICA P

Yes or

ece

Part

CA NN

SMAG

Requests New

Part Issuance

Parts avai able

at De ot?

Yes

How to

Repa r?

Offs te

Locaton

Contract

Out

Non-Job

Routed

Repair

Job

Routed

Repair

Order to

CSI v

SBSS

Component

Reinsta ed

Job Routed Serv ceable Parts

Carcass

thout

Backorder

Carcassthout

Backorder

CSI

Parts

A rriv e

AWP at

Depot

AWP atBase

Non-Job Routed Serv ceable Parts

SBSS-

Initiated

Order

to Base

Non-

Condemned

CarcassServ cable

Unit

Serv cable Part Re-Enters

Sus ta n ment P ne

rcraft

CA NN



44/47


Initial Field-Study Observations

• Current metrics generally foster local optimization rather than global optimization

•

Depot maintenance metrics are primarily focused on financial andrelated measures of performance

• Metrics conflict with delivery of best customer support

•

Current metrics generally do not allow measures of progress towards the achievement of system-wide goals•

Impossible to trace incremental impact on the warfighter ofimprovement actions at local level (e.g., incremental investment inmore parts and materials)

•

Informed tradeoff decisions cannot be made at system-level

• Current metrics drive the “wrong” behavior• Big example is cannibalization (removal of a serviceable component

from one end-item waiting for parts to repair another)


Regression Model:


45/47

Mission Capability Rate

F-16 fully-mission-capable (FMC) rate

Effect of key strategic metrics and action variables (below) on the

Fully-mission- Utilization rate 0.08

capable (FMC) rate

as a function of:

Î

Flying scheduling effectiveness 0.14

Break rate -0.06

Cannibalization rate -0.15 Adjusted R2 = 0.47 Repeat discrepancy 0.03

Unscheduled maintenance -0.12

Temperature (control variable) -0.09

Precipitation (control variable) 0.05

Active duty “dummy”* variable 1.36

National Guard “dummy”* variable -0.32

* “Dummy” variable captures the impact of special characteristics peculiar to each user group.


Complex Causal Loop Structure


46/47

Representation

Fix Rate Total Mission Capable Rate

Aircraft Utilization Rate

Flying Scheduling Effectiveness

Cannibalization Rate Break Rates

Sustainment Costs

+

-

+ -

+

++

-

-

-

-

-

+

+

--

B

R

R

R

R

R

B

RR

R = Reinforcing loop

B = Balancing loop

width = magnitude

+/- = Sign of coefficient

Active Duty Mission Reserve Mission Guard Mission


Research Findings


47/47

Research Findings

“Metrics Thermostat” research reveals a far more complex metrics structurethan indicated by top-down hierarchical metrics structure currently beingused

• AFMC Metrics Handbook: Presents hierarchical metrics structure (command;

mission element board (MEB); center-process metrics)

•

US Air Force Logistics Transformation Team “Balanced Scorecard” approach

Complex feedback loops show system behavioral dynamics not otherwise wellunderstood

• Increased fix rates lead to higher aircraft utilization, higher FMC rates, and increased

flying effectiveness• But this also results in increased break rates, leading to lower fix rates by tapping

limited resources, and therefore lowering FMC rates

• Model quantifies net effects of complex interactions

Model provides new insights into the prevalence and effects of “cannibalization”

not available before• Quantitative evidence provides new shows that cannibalization at field level has net

negative effect on the overall F-16 mission capability rate

• Shows how cannibalization (an emergent system property) is driving “wrong”behavior


metrics and performance measurement

Documents