key performance indicators (kpis) & metrics · key performance indicators (kpis): metrics vs....

Amit Chandak

Key Performance Indicators (KPIs) & Metrics

Director Senior Experience Owner

10/09/2018

2

Profile

Amit Chandak earned his LSSBB from USAA in 2016 and QAI Global

Institute in 2011. Amit is pursuing a Master Black Belt designation from

The Ohio State University and holds a Bachelor’s degree in Business

as well as an M.B.A.

Amit is currently working at USAA as Director of Experience Owners to

formulate and execute Process Excellence, Agile and Risk

Management strategies to deliver seamless members and employees

experiences.

Prior to USAA, Amit brings over 16 years Banking and Financial

Service Industry (BFSI) experience – worked at Apple as an IT Project

Manager, Process excellence & Business Reengineering consultant at

Accenture, TCS and Wipro while working with various global clients

across several business domains (Finance & Accounting, Investment

Banking, Supply Chain, Consumer Banking, Insurance, HR etc)

[email protected]

+1 210 240 1299 https://www.linkedin.com/in/amit-

chandak-62b0996/

Presented by Amit Chandak at ASQ San Antonio Section 1404

mailto:[email protected]

3

Agenda

• Difference Between KPIs vs. Metrics

• Types of Metrics

• Metric Test

• Challenges & Tips

• Identification Process & Tools


4

Key Performance Indicators (KPIs): Metrics vs. KPIs

• Measures components of an activities within a

process

• Aligns to core steps in the business process

• Informs KPIs

• Measures the completion of a specific

business objective (Output) or desired end

state within a certain timeframe

• Includes thresholds of acceptability

• Informs key business decisions

Not all metrics are KPIs, but all KPIs are metrics

METRIC

Measurement of an

activity’s performance

KEY PERFORMANCE INDICATOR (KPI) Measure indicating a

process's performance

Key

Ch

ara

cte

risti

cs K

ey

Ch

ara

cte

risti

cs


5

Levels of Metrics

METRIC

Measurement of an

activity’s performance

KEY PERFORMANCE INDICATOR (KPI) Measure indicating a

process's performance

• Identify and Measure metrics at each level of

Organization

• Ensure metrics are tied at each levels; e.g.

• Profitability at Executive Level

• Cost per application at Manager Level

• Productivity or Handle Time at Employee Level

Key

Ch

ara

cte

risti

cs

Metr

ic

Levels


6

Performance Measurement: Manage Inputs, Report Outputs

Speed

Direction

Altitude

Vital Few Outputs

What does the pilot control?

Leading Metrics Lagging Metrics Predict

Inputs

Throttle

Rudder

Elevator


7

Types of Metrics: Leading (Input & Process) and Lagging (Output)

Leading Metrics Predicts Performance (Output)

•Processing Time, Queue Time

•Exception Rate, Rework

•Backlog (Volume & Time)

•Utilization, Availability

•Average productivity per Agent

•Customer Complaints

•Skill – Cross Training

•Volume Received

•Timeliness (Delivery)

•NIGO

•Timeliness (Delivery)

•Accuracy

•Satisfaction (Customer & Emp)

•Cost of unit, Cost of Poor Quality

•Revenue

•Risk

On Time

Quality

Complete Step 1 Step 2 Step 4 Step 5

On Time

Quality

Cost

Satisfaction Step 3

Inputs Process Output

Leading Metrics Lagging Metrics

Inputs Metric Process Metric Output Metric


8

The performance metrics chosen must be “S.M.A.R.T.”

▪ Is it clearly defined?

▪ Can it be easily generated without complex calculations? Specific S

▪ Is it easy to collect the data needed?

▪ Can it be compared to other data?

▪ Can the measurement be defined in an

unambiguous way?

Measurable M

▪ Can stakeholders influence it?

▪ Do we understand what drives the metric? Actionable A

▪ If the metric changed, would anyone care?

▪ Does it tell the whole story from a value stream perspective? Relevant R

▪ Can it be measured often enough to identify problems while

they are still relevant? Time -

bound T

SMART Test


9

Specific S

Measurable M

Actionable A

Relevant R

Time -

bound T

Does Your Metric Qualifies SMART Test?

Poor measure

Timeliness: Number of wires

submitted by EOD SLO.

▪ All terms are clear

▪ Computing this is

simple addition

▪ The data is

straightforward to

collect

▪ The employee / unit is

in control on whether

they deliver on time

▪ Cannot tell whether

there were any wires

that were late

▪ The number can be

calculated at any time

SMART measure

Timeliness: % of wires

submitted by EOD SLO.

▪ No change

▪ No change

▪ No change

▪ The score incorporates

the total number of

wires

▪ No change


10

Specific S

Measurable M

Actionable A

Relevant R

Time -

bound T

Poor measure

Quality: Number of errors in

customer submissions per year

▪ Errors are clear

▪ The data is straightforward

to collect

▪ Does not provide context

(e.g. if 3 fail is that a lot or a

little)

▪ Associates cannot prevent

customers from submitting

error filled materials

▪ One year is required before

a poorly performing

process for detection will

be noticed

SMART measure

Quality: % of errors that are not

caught in the initial screen

▪ No change

▪ No change

▪ The % indicates the volume

of undetected to detected

errors

▪ Associates are able to

control the quality of their

screening

▪ The number can be

computed as any point

Does Your Metric Qualifies SMART Test?


11

Watermelon Effect


12

Process Integration – Backward and Forward

Marketing Acquisition Fulfillment Service

Design Raw Materials Production Distribution

Integrate your KPIs to get optimal business results


13

Best Practices for Identifying Metrics and KPIs

Define End to End Process

1

Brainstorm Metrics 2

Distinguish KPIs vs. Metrics 3

Measure & Specification Limits 4

•Identify Metrics (from list created in Step 2) and

document additional details to enable data

collection

•Baseline information and specification limits

•Identify most important indicators of

successful Process performance

•Identify Input-Process-Output metrics

•Brainstorm operational metrics that will help

monitor and manage the process

•Develop SIPOC or High-level process maps

•Identify customer (internal/external) and

suppliers

Identify and Remediate Gaps 5

•Identify measurement gaps in

current state and remediate

accordingly


Data Collection Plan

14

Process Name: Pizza Delivery Process Owner (PO): Pizza Store Manager

LoB: Suppliers Inputs Input Requirements Process Outputs Customer Requirements Customers

Who Provides the

inputs?

Materials, Information,

Ideas, Labor

How does the Process Owner

want the inputs? Starts with: Hunger

Documents, Information,

Services, Decisions

How does the Customer

want the outputs?

Who Uses or

Benefits from the

Output?

Store Manager (PO)

Store Manager (PO)

Hungry customer

Store Manager (PO)

Menu options

Order venue (phone, digital,

,com)

Delivery information (name,

address)

Stated delivery Service Level

Agreement (SLA= 45 minute

delivery)

Full menu options

Order venue (phone, digital, ,com)

Accurate information (name, address)

Time order is entered

Order Pizza

Documented order

Order is in queue

Estimated delivery time

Delivery information

Order recorded

Order is in queue

Accurate delivery lead time

estimate

Accurate delivery information

Baker

Baker/Hungry customer

Hungry customer

Driver (Task Owner)

Order Taker (Activity Owner)

Order Taker (Activity Owner)

3rd Party supplier

Pizza store Manager (PO) /

Baker (Activity Owner)

Documented order

Order in queue

Ingredients

Standard operation procedures

(SOP)

Accurate order

First-in-first-out (FIFO) order

Fresh ingredients available

Ingredients added in proper proportions

Assemble Pizza

Assembled pizza Pizza assembled accurately per

order (100% of the time)

Hungry customer


Store Manager (PO)


Assembled pizza

Pizza oven

Time in oven

Assembled pizza accurate

Pizza oven at baking temperature Bake Pizza Baked pizza ("done") Thoroughly baked pizza / not

burnt

Pizza visually appealing

Hungry customer

Scheduler (Activity Owner)

Scheduler (Activity Owner)

Store Manager (PO)

Store Manager (PO)

Estimated delivery time

Time in delivery queue

Volume of deliveries in queue

Thermal carrier

Delivery route

Delivery driver

Delivery information and vehicle

On time delivery

Minimize queue time

Minimize queue volume

Insulates content (keeps pizza hot)

Efficient route

Available delivery driver

Delivery staff

Reliable delivery vehicle

Deliver Pizza

Pizza delivered

On time delivery (45 minutes

from order)

Hot pizza

Correct toppings

Hungry customer

Ends with: Eat Pizza

Ends with: Eat Pizza

Step 1 - Define End to End Process using Gold SIPOC

15

Steps 2 - Data Collection Plan

Date

Captured

Associated

CTQ

Category

Metric Name Operational Definition Math Formula

Unit of

Measurement

Description of Business

Insight / Objective

XX/XX/XX CTQ from the VOC

Is the metric related

to Time, Cost,

Quality, or Member

Experience?

What should the metric

be called? Should be

easy to understand.

How would you describe the metric so there is no

ambiguity on what you are intending to measure?

How would you calculate the

measure? For instance, would it be

a sum of all products, a ratio, the

difference between two values, etc.

e.g., #, $ , % What would this metric enable you

to do? What will it be used for?


Step 2. Brainstorm operational metrics that will help monitor and manage the process, and identify KPIs that depicts performance.

16

Steps 3 - Data Collection Plan

KPI? Other Conditions / Filters to record

Included in a

report /

dashboard?

Name of report /

dashboard

Current Data Source &

Location

How is the data

collected?

Who collects

the data?

What

Frequency?

Yes / No

What other types of filters would be needed to

make this data more meaningful? For instance,

would you need to be able to filter by product or

by employee?

Is the measure

being captured in a

report/dashboard

today?

If the measure is

captured in a report,

please list the name of the

report .

Where is the data being

stored?

Is the data being

captured manually

or automatically?

List the name of

the individual or

system.

What frequency is

the data collected

?


Step 3. Identify Metrics (from list created in Step 2) and document additional details to enable data collection

17

Step 4 - Data Collection Plan

Industry

Benchmark Baseline

Target

State Lower Spec Limit

Upper Spec

Limit Data Available

Notes on Data

Availability What is the current

performance of the

metric in the industry?

What is the current

performance of the metric?

What is the target state?

(Should be informed by

VOM / VOB).

What is the lower limit?

What is the upper limit?

(There might not be an

upper limit)

Yes, No, Partially Available Note as GAP if data not

available


Step 4. Once data is gathered, enter in baseline information and specification limits.

18

CoSA: Process Owner: Pizza Store Manager

Process Name: Pizza Delivery Last Update:

3X3 Matrix

Y Y

(Output, Lagging, Reactive)

X1 (Input, Leading, Predictive)



Y1 Quality: Pizza toppings

correct Accurate order taken Pizza assembled per order Ingredients available

Y1 Quality: Hot pizza Delivery Lead Time Thermal carrier efficiency Time in delivery queue (wait/delay

time after baking)

Y2 Speed: On Time Delivery

(45 Minutes) Order queue time Time in delivery queue Delivery route efficiency

Y3 Cost: Cost per pizza Defect rate Delivery efficiency (routing) Missed delivery commitment

3 x 3 Matrix

•Vital few vs Trivial many – Defining and monitoring the vital few is essential so team is

focused

•Identify 3 most important Lagging Metrics (KPIs) and 3 most influential Leading Metrics

•Proactively control the inputs that significantly impacts the output metric


19

Best Practices for Classifying Metrics

To Be Retired To Be Migrated To Be Developed

Associated Risk

Imp

act

to C

usto

mer

Associated Risk

Imp

act

to C

usto

mer

Next Steps:

• Document as-is process for tracking

and reporting the metrics/KPIs

• Create streamlined process for

reporting metrics/KPIs

Next Steps:

• Meet to determine gaps in obtaining

metrics/KPIs not currently in

production

• Develop interim state method

for obtaining metric/KPI

Next Steps:

• Communicate decision to retire

metrics/KPIs to stakeholders who are

responsible for their production

Gap Remediation Strategies – (Step 5 of 5)


20

Continue The Cycle

What get measured, gets reviewed and improved!

Identify

Measure

Improve

Sustain & Recycle


21
