Baseline Data

• Measure• Kaizen Facilitation

2

Objectives

• Define data types and purpose• Explain concepts of efficiency and effectiveness• Provide tips on establishing baseline metrics• Understand importance of data collection• Review elements for data planning and usage

3

Types of Data (Definition)

A metric obtained by observing a population, product, process, or service...

• Qualitative • Job is expensive and takes too long

• Quantitative• ATTRIBUTE DATA

• (Categories, Yes/No, Pass/ Fail, Machine 1 vs. Machine 2, etc)• CONTINUOUS DATA

• (Time, Temperature, Weight, Pressure, etc)

4

Purpose of Data Collection

• The “generic” purpose of all data (metrics) is to help make better decisions

• A Metric is an input to a decision system which has the following elements:• Who looks at it?• When / Where / How do they look at it?• What do they compare it to?• What are their choices?• What actions are taken?

Better Data = Better Decisions

5

System Feedback

• How do you know if your system is working right?• All processes exist to fulfill needs• Your system is working ‘right’ if it fulfills those needs

• Who defines these needs?• The customer (recipient of good or service)

• Typically associated with the output of the process• The business (provider of good or service)

• Typically associated with the process itself

6

Establishing Metrics

• For each product or process requirement, there should be one or more objective measures

• These measures (metrics) allow us to verify that requirements have been met through the general relationship:

MeasuredPerformance

_ RequiredPerformance

= (Delta or Variance)

7

Sources of Process Feedback

• Typical organization has two main sources:• Metrics

• Quantitative performance indicators are compared to defined requirements• If these indicators do not meet or exceed the requirements, then a “problem” is said to

exist

• Squeaky Wheel• Anecdotal and experiential feedback for the process indicate the potential for a

“problem”• Squeaky wheels can be a trigger for the creation of a metric

8

Flow versus Performance

• Process mapping helps us understand process flow• Now we must collect data to more objectively evaluate and measure

performance• To be successful, we must meet both Customer and Business

requirements• Business (VOB) - efficiency measures• Customer (VOC) - effectiveness measures

• Measurement systems incorporate both

9

Business Feedback (VOB) - Efficiency• In order to remain profitable, a business must deliver goods and

services such that their cost is lower than their price• Business systems should operate at:• High speed• Low cost• With minimum resources (task sequence, strain, waste)

• Voice Of the Business are “Efficiency” measures

10

What to Measure? (VOB) - Efficiency

• Outputs• Highest level• Look for:• Quality

• Defects, Yield, Rework (Overall and by process step)• Cycle Time

• Lead time (Overall and by process step)• Cost / Bottlenecks (Overall and by process step)

11

VOB Metrics (How Capable is the Process?)• Quality: Does the product or service meet the customer requirements

(specifications)?

• Cycle time: How much time do various steps in the process take? Are there delays in some steps?

• Bottlenecks: What types of bottlenecks are you seeing? How frequently? How long is the delay?

12

Customer Feedback (VOC) - Effectiveness

• Goods and services that your Customer receives is an output of your process• Customers want goods and services:• On-time• With highest quality• At an excellent value (competitive price, et al)

• Voice Of the Customer are “Effectiveness” measures

13

What to Measure? (VOC) - Effectiveness• Unacceptable product or service• Customer complaints• High warranty costs• Decreased market share• Backlog• Redoing completed work (cost)• Late output• Incomplete output (yield)

14

VOC Metrics (How Satisfied is the Customer?)

• On-time: How many of your products or services out of the total that were made or delivered meet customer requirements?

• Quality: Does the product or service meet the customer requirements?

• Cost: How much does it cost to produce the product or service and how does the cost compare to your competitor’s costs?

Plan

Data Collection

USAGE

Elements of Data Collection• Data collection is the process of gathering the

information you need to be able to make a better decision

15

16

Planning: Clarifying Goals

• Decide why you are collecting the data• Determine factors that could cause the measurement of an item to

vary• Find ways to reduce the impact of those factors• Decide how the data will help you• Decide what you will do with the data once you have it

17

Planning: Consistency and Stability • Know your process• Decide what data you need to collect• Fresh (current) data or historical (past) data

• Develop an “operational definition” of metric• Test your data collection forms• Make it easy to collect data• Communicate the what and why to the data collectors and process

participants

18

Usage: Operational Procedures

• Decide what you are trying to evaluate• Decide how you will attach a value to what you are trying to measure• Decide if you need to collect new data• If so, decide how you will collect the data• Decide how you will record the data• Determine the period of time you will study• Estimate how many observations you will need

19

Usage: Oversight Required

• Train everyone who will be collecting data• Make data collection procedures error-proof• Be there in the beginning to oversee data collection• Confirm understanding of operational definitions• Check to make sure data measurements are stable and that the data

looks ‘reasonable’

• Entered as part of workflow• Mined with queries and reportsDatabase

• Sheets that move with product or fileTravelers

• Every item vs. selected ones• Links data to a specific recordLogs

• Easy to use• Aggregates dataCheck sheets

Collection: Sources

20

Collection: How Much ?

Too Little Data• Bad decision making• Extends project timeline• Extra work to collect additional

data

Too Much Data• Wasted time collecting data• Frustrated team members• Wasted time spent managing

data• Important information hidden in

piles of data

21

22

Types (Pros and Cons)

• Attribute Data• Few Tools for Analysis• Need more data

• Continuous Data• Gain more insight to behavior of the process• Can use many tools to analyze• Need less data

Whenever possible its generally better to use Continuous Data

Data

Continuous

Attribute

Binary

Measured

Categorical

Ordinal

Count

Types (Safety example)

23

Time of Incident

Gender

Type of Incident

Severity Rating

Number of LTI’s

(Can be sub-divided)

• Collecting data on a small subset of a population to make inferences about the characteristics of the whole population• Benefits• Lower cost• Saves time

• Drawbacks• Possibility of errors

Sampling uses this…

Sampling

24

…to predict

about all of this

Sampling

25

26

Integrity (Some things to think about)

• Is recorded data what we meant to record?• Does it contain information that was intended?• Does the measure discriminate between items that are

different?• Does it reliably predict future performance?• Does it agree with other measures designed to get at the

same thing?• Is the measure stable over time?• Is the data correct in the system?

Evaluate what’s right and wrong with your process

• The way baseline data is collected can have an adverse effect on the outcome of the project• A data collection plan is an organized, written strategy for gathering

information for your project• Data are reliable every time• Only relevant data are collected• All necessary data (VOB, VOC) are collected• Resources are used effectively

• Data collection form – a very important check sheet for your project…

Tips

27

28

Review

• Don’t assume people know how to collect data• Take a careful, methodical approach

• Develop a systematic method to collect, review and analyze the data • Chart data and Analyze for trends• Ask: Do the results pass the common sense test?

• Monitor process using your metrics• See how trends project into the future• Share the results with appropriate personnel