© 2007 breakthrough systems1 deming’s red bead experiment aice quality conference tuesday,...
TRANSCRIPT
© 2007 Breakthrough Systems 1
Deming’s Red Bead Experiment
AICE Quality ConferenceTuesday, 06MAR07
Jim ClausonBreakthrough Systems
http://jclauson.com/aice
© 2007 Breakthrough Systems 2
Why Are We Here?
To consider:
• what the Red Bead Experiment is,
• what it has to do with quality and
• how it can have an impact on your everyday quality activities
© 2007 Breakthrough Systems 3
Session Agenda
The Red Bead Experiment Numeracy System of Profound Knowledge Finding Red beads Impact of psychology Charting red beads “What will you do on Monday?”
© 2007 Breakthrough Systems 4
Introductions and Expectations
• If you are sitting with someone you know, please move
• You will interview, then introduce the person next to you
• Who they are, something unique or interesting, what industry they represent, and what their expectations are today
© 2007 Breakthrough Systems 6
Review: Why Are We Here?
To consider:
• what the Red Bead Experiment is,
• what it has to do with quality and
• how it can have an impact on your everyday quality activities
© 2007 Breakthrough Systems 7
Beyond Red Beads
• Red Beads Everywhere, oh my!!
• Finding those red beads– Red, white and non-red?
• Quantifying those read beads– Hit by a car –vs- bump a file cabinet
• Eliminating those red beads– 99.99999999999999999999999999%
© 2007 Breakthrough Systems 8
Numeracy
• an ability to handle numbers and other mathematical concepts
• in the US, it is somewhat better known as Quantitative Literacy
• innumeracy is a lack of numeracy
© 2007 Breakthrough Systems 9
3 Kinds of Numbers for Management.
• Facts of life. If we don't make this profit figure, we will go out of business.
• Planning, prediction and budget. Can be used to compare alternative plans.
• Arbitrary numerical targets. Generally used to judge workers.
Avoid the use of the 3rd kind of number
Henry Neave The Deming Dimension
© 2007 Breakthrough Systems 10
Data Sanity
We can either react to numbers, with explanations of every percent change, with the inherent frustrations, fear, and failure
OrWe can understand our data, put it to good use,
and apply valid management principles
The choice is ours.
© 2007 Breakthrough Systems 11
Through the Lens of SoPK
System of Profound Knowledge
• Appreciation for a system• Knowledge about variation• Theory of Knowledge• Psychology
From The New Economics, Deming
© 2007 Breakthrough Systems 12
1 of 4: Appreciation for a System
• Pay attention to interactions more so than components
• Knowledge of statistical variation more so than discrete numbers
• Long term focus more so than short term
• Cooperation more so than fear, blame and internal competition
© 2007 Breakthrough Systems 13
1 of 4: Appreciation for a System - II
• “94% of the outcome of any organization comes from the processes used, not the people”.
• “A fault in the interpretation of observations, seen everywhere, is to suppose that every event is attributable to someone (usually the one closest at hand), or is related to some special event. The fact is that most troubles with service and production lie in the system and not the people”.
© 2007 Breakthrough Systems 14
2 of 4: Knowledge of Variation
• You have experienced the Red Bead Experiment
• The Theory of Variation is at the core of cost savings, Kaizen, 6 sigma…
• Dr. Deming’s early works focused on statistical variation. He added the rest of the SOPK in the last 10 years of his life.
• Stable System versus Unstable System
© 2007 Breakthrough Systems 15
Variation: Deterministic –vs- Probabilistic
• Deterministic - linear, cause and effect sequences. If you do this, that will happen.
• Probabilistic - exact time, location, and effect is random. e.g. Number of Red Beads.
• Treating a probabilistic result as if it was deterministic will cause problems
• Past results will not guarantee future results
© 2007 Breakthrough Systems 16
Variation: Can we predict?
• Engineers often predict accidents. Their predictions are uncanny for correctness in detail. They fail in only one way – they can not predict exactly when the accident will happen.
- Dr. Deming, Out of the Crisis page 479• Calculations after the fact, using only data available
prior to the disaster, showed there was greater than a 10% chance of the Challenger explosion occurring, given the pre-launch temperatures and prior history of O-ring burn through.
© 2007 Breakthrough Systems 17
3 of 4: Theory of Knowledge - I
• Knowledge is based upon prediction
• Knowledge is built on theory– Chanticleer the barnyard rooster
– Actions taken without theory lead to losses
• Use of data requires prediction
• There is no true value of a measurement, it depends on methods, context, and use
• Operational definitions are necessary
© 2007 Breakthrough Systems 19
Operational Definitions - II
• Most arguments about conflicting data come down to the definition of how to count the data
• Try to be precise in your definitions, but likely something unforeseen will arise
• The beads were:– red & white or red, white & non-red?
© 2007 Breakthrough Systems 20
Deming said…
• “It’s absolutely vital for business that you settle this method of counting, measuring, definition of faults, mistake, defect, before you do business. It’s too late afterwards”
-Dr. W. Edwards Deming
© 2007 Breakthrough Systems 21
4 of 4: Psychology
• Extrinsic versus intrinsic motivation
• People will use the charts you make - up and down the organization
• If you do not understand the people & understand psychology, the charts will be ignored
• Competition, fear, perceptions, loss of control change the data and the chart’s message
© 2007 Breakthrough Systems 22
Through the Lens of SoPK
System of Profound Knowledge
• Appreciation for a system• Knowledge about variation• Theory of Knowledge• Psychology• Thinking about all 4, we’ll concentrate on
identifying and quantifying variation
© 2007 Breakthrough Systems 24
Take a Step Back: Systems Thinking
• Process viewed as a system
• SIPOC
• The “new” 4M’s
• Psychology: Suboptimization
• Remember the “94/6 Rule”
• Using SPC
© 2007 Breakthrough Systems 27
The “new” 4M’s
• Old: man, machine, material, method• Measurement added• Person or people replaces man• Equipment replaces machine• Supplies is used for material• Process is used for method• Environment added
– Physical and mental
© 2007 Breakthrough Systems 28
Group Activity - IV
• As individuals, draw a SIPOC diagram for your job
• See if you can identify all of the new 4M’s as inputs to your process
• Compare and discuss as teams
• Choose the 3 most interesting to report
• Save this work, it will be used later
© 2007 Breakthrough Systems 29
Psychology: Suboptimization
• We assume that optimizing a system considers all the sub-parts
• One unit may be selfish and take an action that makes them look good, but hurts others
• One process may shift problems down the line to let others have to worry about it
© 2007 Breakthrough Systems 30
No Gold Stars Here
• Awards, bonuses, gold stars can actually have a detrimental impact
• For a person driven extrinsically, each subsequent reward must be larger in order to have the same impact
• Creation of winners and losers
http://www.alfiekohn.org/index.html
© 2007 Breakthrough Systems 31
Group Activity - V
Game: Win As Much As You Can
A Decision Making Exercise, Illustrating the Effects of Human Behaviors and Psychology on Performance Measures
[link] to game
© 2007 Breakthrough Systems 32
The “94/6%” Rule
• It is critical to separate system causes from individual causes of variation
• Deming started at 85% systems and 15% worker and had moved up to 96% and 4% by his death
• What are the implications of the 96/4% Rule?
© 2007 Breakthrough Systems 33
Breather – We OK?
• What have we addressed?
• Numeracy
• System of Profound Knowledge
• Looking at your work as a system
• Suboptimization exercise
• 96/4% Rule
• We OK?
© 2007 Breakthrough Systems 34
Statistical Process Control
• Control Charting provides knowledge of variation
• a lens, providing a different way of viewing the world
• a significantly different view of what is happening than will other methods
© 2007 Breakthrough Systems 35
A Basic Control Chart
Injuries per Month - as a Control Chart
0
5
10
15
20
25
Jan-
03
Mar
-03
May
-03
Jul-0
3
Sep
-03
Nov
-03
Jan-
04
Mar
-04
May
-04
Jul-0
4
Sep
-04
Nov
-04
Jan-
05
Mar
-05
May
-05
© 2007 Breakthrough Systems 36
SPC Basics
• Common -vs- special
• System in control?
• System predictable?
• What about psychology?
© 2007 Breakthrough Systems 37
Charting the Red Beads
• In the Red Bead Experiment, we reacted to the random noise from result to result.
• Rewards, punishments, ranking of the workers, feedback to the workers had no effect on the results of the process.
• The process was stable and needed to be changed!
© 2007 Breakthrough Systems 38
Stable –vs- Unstable
• Stable processes contain only common cause variations and are predictable
• Unstable processes contain special cause variations and are not predictable.
• Only predictable processes may be used to plan effectively
© 2007 Breakthrough Systems 39
Cause: Common –vs- special
Special Cause Variation: If a statistically significant trend occurs, find the
special cause of this trend. Use this information to correct or reinforce these special causes.
Common Cause Variation: If no trends exist, you must look at the long run
performance of the process and fundamentally change the process in order to improve the process.
© 2007 Breakthrough Systems 40
Defining Trending in Charting
• One point outside the control limits• Two out of Three points two standard deviations
above/below average• Four out of Five points one standard deviation
above/below average• Seven points in a row all above/below average• Ten out of Eleven points in a row all above/below
average• Seven points in a row all increasing/decreasing
© 2007 Breakthrough Systems 42
Constructing a Control Chart
• Plot the actual data by month (or whatever time interval you are using)
• Plot at least 25 points (when available)
• Calculate a baseline average rate
• Add 3 standard deviation control limits
• Incorporate a set of trend rules
© 2007 Breakthrough Systems 43
Why 3 Standard Deviations?
• Dr. Shewhart established 3 standard deviations as an economic balance between failure to detect and false alarms.
• Economic Control of Quality of Manufactured Product (1931!)
© 2007 Breakthrough Systems 44
Just in Case…
• Many courses incorrectly teach that the control limits cover 99.7% of the normal distribution
• Not all data are normal, “real data” can cause the rate to be as low as 95% (Dr. Wheeler)
• The Tchebychev Inequality states up to 11% can be outside three standard deviations
© 2007 Breakthrough Systems 45
Breather – We OK?
• stable –vs- unstable
• common –vs- special
• constructing a control chart
(or a process behavior chart)
• 3 Standard Deviation limits
© 2007 Breakthrough Systems 47
Group Activity – VI
• As individuals, take your SIPOC & 4M exercise, and– Identify a performance indicator on the input and
then the output of the process you have diagrammed and explain why you chose it
– Share and discuss your choice with your team– Choose 2 or 3 examples from the team to share
with the class– Remember: you are hunting red beads
© 2007 Breakthrough Systems 48
Activity Debrief
• What indicators did you choose?
• Why?
• What actions would you take if one developed an adverse trend?
© 2007 Breakthrough Systems 49
Choosing the Right Measures
“Managers who don’t know how to measure what they want settle for wanting what they can measure.”
Dr. Russ Ackoff
© 2007 Breakthrough Systems 50
Performance Indicator Introduction
• It is more important how the measure is used than what the measure is
• Self-fulfilling prophecies can prevent us from gathering any data
• We are drowning in data, but little knowledge is derived
• Context and Operational Definitions are crucial
© 2007 Breakthrough Systems 51
5 Critical Issues ( 1-3)
• Managers suffer from overabundance of irrelevant information.
• Managers don’t know what information they need. Need to look at the decision process to determine this.
• Even if given the information they need, decision making will not necessarily improve
© 2007 Breakthrough Systems 52
5 Critical Issues (4-5)
• More communication does not necessarily lead to better performance. Information can be used destructively.
• Managers do need to know how the information system works. Just because it came from a computer doesn’t mean it is right.
© 2007 Breakthrough Systems 53
Designing a Management System
• The information system should be designed as an integral part of the management system
• Most information systems are designed independently, leading to failure
• Information systems should serve management, not vice versa
© 2007 Breakthrough Systems 54
PI Barriers
• higher ups will use it as a “hammer”• subjected to quotas and targets imposed from
above• fear (“accountability”) used as a “motivator”• actions and explanations as a result of
random fluctuations• perceived loss of control over portrayal of
performance• must develop “perfect” indicator the first time• use of SPC can minimize these fears
© 2007 Breakthrough Systems 55
Ackoff on Performance Indicators
• We do need to know the context within which the performance indicators will be used
• Forecasting and living with the forecasted future is important, but what about designing a better future?
© 2007 Breakthrough Systems 56
Context of PI
• Do not look at a chart in a vacuum
• Reconcile any differences between the data and “gut feeling”
• Combine experience and the data
• Lessons from the data should lead to insight in the field, and vice versa
© 2007 Breakthrough Systems 57
PI Evolution
As a process matures, one may end up evolving the indicators used. For example, if interested in completing actions by commitment dates, one may end up using (as the process matures):
• Percent of Actions completed by due date in effect at time of completion
• Percent of Actions completed without missing any due dates during their life
• Percent of Actions completed by the original due date
• Average days Actions completed ahead of original due date
© 2007 Breakthrough Systems 58
Trying for the “Perfect” PI
• When committees get together and try to table-top the perfect indicator, paralysis often sets in.
• Realize all data are flawed, there is no “true value”, indicators can always be “gamed.”
• Putting the right culture of HOW to use performance indicators in place minimizes adverse impacts.
• Gain experience with simple indicators, then move on to more complex indicators if needed.
• With proper analysis, flaws with existing data can be detected and fixed. If you never look at the data, there will never be an incentive to fix the data.
© 2007 Breakthrough Systems 59
Just Do It!
• All data are flawed• Make good use of your data• Endless conference table
discussions won’t cause any data to appear
• Initial prototype successes will lead to experience, and will further the spread of the use of indicators
© 2007 Breakthrough Systems 60
Data Gathering
• Plan ahead
• Establish Operational Definitions
• Check data quality
• Avoid bias
• K.I.S.S.
© 2007 Breakthrough Systems 61
Data Quality
• Data should be replicable• Operational Definitions are a must• Source Data must be defined• There is no “true value” of any measure, but a
good operational definition can save much trouble in the future
• ANYONE at ANYTIME in the future should be able to apply the same operational definition to the same source data, and get the same results.
© 2007 Breakthrough Systems 62
Choose a Reporting Interval
• If a trend developed, how long could you go without needing to know it?
• Longer intervals imply more risk
• Need sufficient volume of points (25)
• Costs increase as reporting interval decreases
• What is current reporting interval?
© 2007 Breakthrough Systems 64
Creating the Baseline
• The Baseline on a control chart consists of the average (center) line, a three-standard deviation Upper Control Limit (UCL) and a three-standard deviation Lower Control Limit (LCL).
• The Baseline allows us to predict the future, and evaluate for trends.
© 2007 Breakthrough Systems 65
A Good Baseline
• A “good” baseline detects future trends with a minimum of false alarms.
• If a trend is detected, we don’t want it to be due to too few data points in the baseline, causing the baseline to have been inaccurate.
© 2007 Breakthrough Systems 66
To get a Good Baseline
Do show all data, but change the average and control limit calculations by:
• Dropping data off of the beginning
• Dropping data off of the end
• Dropping individual datum point(s) and circling them
© 2007 Breakthrough Systems 68
Establish Expectations
• “Stable” performance is not necessarily good
• Management needs to determine if the current stable baseline is “acceptable” or “unacceptable”
• Recall that the # of red beads was unacceptable, but process was stable and in control
© 2007 Breakthrough Systems 69
Monitoring
• Update charts on the required time interval• Check for trends against the trending rules• Circle any trends, inform owning
management and look for special cause(s)• Do not shift a baseline unless there is a trend
(baseline proven guilty)
© 2007 Breakthrough Systems 70
That’s Charting Performance Indicators in a Nutshell
• PI as part of an overall management system
• What makes a good performance indicator
• Data gathering• Establishing a good baseline• Establishing Expectations• Monitoring
© 2007 Breakthrough Systems 72
Session Summary
The Red Bead Experiment Numeracy System of Profound Knowledge Finding Red beads Impact of psychology Charting red beads “What will you do on Monday?”
© 2007 Breakthrough Systems 75
Wrap & Roll…
Thanks for your time and attention
Jim Clauson
http://jclauson.com/aice