understanding variation: statistical process control...
TRANSCRIPT
Understanding Variation:Statistical Process Control (SPC)
Brent C. James, M.D., M.Stat.Executive Director, Institute for Health Care Delivery ResearchIntermountain HealthcareSalt Lake City, Utah, USA
A process
... a series of linked steps, often but not necessarily sequential, designed to ...
some set of outcomes to occurtransform inputs into outputsgenerate useful informationadd value
cause
Walter Shewhart: a system of causes
A system of causesWalter Shewhart was a student of, above all, causes. He believed that results in complex systems did not just happen but were the consequences of lawful relationships; maybe it was because he was a physicist that he chose to interpret production that way. He believed that, properly analyzed, experience in real causal systems could teach a great deal about those systems, and he devoted much of his professional career to developing methods through which the study of variation in measured results could teach the observer about the causal systems that led to those results. If he had been a physician, he would have been called an applied epidemiologist, or a clinical researcher—and a master at it.
The causal systems that intrigued Shewhart (the most) he called "systems of chance cause," but he used the word "chance" in a most unusual way: to Shewhart, "chance causes" meant, exactly, "unknown causes." It dawned on him that real, unknown causes were of two distinct types: as he put it, not all systems of chance causes are alike. In particular, some such causal systems produced effects that obeyed understandable mathematical laws. That was fortunate, since, because they obeyed mathematical laws, that permitted one to make predictions based on experience. He called these "constant systems of chance causes," and they are the same as Deming later called "common causes" and Juran called "random causes."
1. Shewhart WA. Economic Control of Quality of Manufactured Product. New York, NY: D. Van Nostrand Company, Inc., 1931. (Available from Quality Press, American Society for Quality Control, 310 West Wisconsin Avenue, Milwaukee, WI 53203)
2. Deming WE. Out of the Crisis. Cambridge, MA: MIT Center for Applied Engineering Studies, 1986.3. Juran JM, Gryna FM, eds. Juran's Quality Control Handbook, 4th ed. New York, NY: McGraw-Hill, 1988.
1
2 3
Berwick, DM. Controlling variation in health care: a consultation from Walter Shewhart. Medical Care 1991; 29(12):1212-25.
Constant (convergent) systems
follow the laws of mathematical probability:
How the process behaved in the pastpredicts how it should behave in the future
non-constant (divergent) systems follow the laws of chaos theory:
How the process behaved in the pastdoes not predict how it should behave in the future
Random variation
different processes have different levels of random variationrandom variation is a matter of measurement, not goal setting
represents "appropriate" variation
is a physical attribute of the process
represents the sum of many small variations, arising from real but small causes that are inherent in -- and part of -- any real processfollows the laws of probability -- behaves statistically as a random probability functionbecause random variation represents the sum of many small causes, it cannot be traced back to a root cause
Assignable variation
represents "inappropriate" variation
represents variation arising from a single cause that is not part of the process (system of causes)
therefore can be traced, identified, and eliminated (or implemented)
Managing assignable variation
Find a data point that probably represents assignable variation (usually a statistical outlier)
track it to root causes
eliminate (or implement) the assignable cause
(React to individual fluctuations in the data)
Managing random variation
Act (either implement the tested alternative, modify it and test again, or discard it)
Study the results (does the new process have a level of performance and/or random variation that is superior to that displayed by the old process?)
Do it in a trial (on a small test group)
Plan a change (design a new process)
(The Shewhart PDSA cycle is a simple application of the scientific method)
Usually, the new process is a variant of the old process. Therefore
The level of random variation is a physical attribute of a process. Therefore, in order to reduce random variation one must find a new process with a new level of random variation, that is superior to that of the original process.
Tampering:
using assignable methods in an attempt to manage
random variation
Shewhart proved that tampering does not just waste time and effort --
it seriously harms process performance
A frequency distribution
tracks the performance of a process across a group of observations / measurements
shows the number of times (y axis -- count, rate, percentage,
proportion) each possible value occurred (X axis)
while it is not possible to exactly predict any single future observation for the process, the frequency distribution gives an envelope within which nearly all of the process's future measures should fallwhen Shewhart talked about processes that obeyed "understandable mathematical laws," he meant convergent processes for which it is possible to generate a frequency distribution:
How the process behaved in the past, predicts how it should behave in the future
Empiric frequency distribution
Value observed
Num
ber o
f tim
es o
bser
ved
(Num
ber,
rate
, per
cent
age,
pro
porti
on)
Parametric frequency distribution
Value observed
Num
ber o
f tim
es o
bser
ved
(Num
ber,
rate
, per
cent
age,
pro
porti
on)
Parametric frequency equation
Value observed
Num
ber o
f tim
es o
bser
ved
(Num
ber,
rate
, per
cent
age,
pro
porti
on)
Statistical resolution
How well a statistical test can detect true differencescalled "statistical power"directly related to Type II (beta) statistical error
Determined bydata typespecific statistical testsample size (statistical significance vs. clinical significance)
In general, parametric statistical tests are more powerful than non-parametric statistical tests
(but they make assumptions about underlying frequency distributions, which may or may not be true)
Parameters: mean and variance
center (mean, median)
spread (variance, standarddeviation, range)
Value observed
Num
ber o
f tim
es o
bser
ved
(Num
ber,
rate
, per
cent
age,
pro
porti
on)
Probability-based boundaries
2.575 std. devs. 2.575 std. devs.
0.5% 0.5%
99%
Value observed
Num
ber o
f tim
es o
bser
ved
(Num
ber,
rate
, per
cent
age,
pro
porti
on)
Frequency Distribution
Time
Observed value
Statistical Process Control Chart
Random variation
Time
(How the process behaves over time)
T1 T2 T3 T4 T5 T6 T7 T8 T9
Observed value
Process Control Chart
Assignable variation
TimeT1 T2 T3 T4 T5 T6 T7 T8 T9
(How the process behaves over time)
Observed value
Process Control Chart
Statistical process control charts
Show the probability that an observation arose from the underlying process -- that is,
the probability that a particular point's deviation from the center represents only "random" variation arising from the system of causes that make up the process, as opposed to "assignable" variation representing an identifiable, intruding cause.
Theyseparate random from assignable variationbased on statistical probabilityusing control limits, runs, trends, and other patterns in the longitudinal data.
are action / decision thresholds (along with runs, trends, and other patterns in the data)
are measured in units of standard deviations: 5% limits 1.96 std. devs. 1% limits 2.575 std. devs..1% limits 3.08 std. devs.
must balance three costs:The Cost of Tampering -- Type I ( ) statistical error: the
probability that a point is actually random, when the SPC chart classifies it as assignable
The Cost of Failure to Detect -- Type II ( ) statistical error: the probability that a point is actually assignable, when the SPC chart classifies it as random
The Cost of Analysis (usually relatively unimportant)
Control limits
Two types of control limits
a priori control limits ("standards given"):
empiric control limits:there is no a priori knowledge of the process's center or
spread;so we estimate them from the observed data themselvesthen update them each time we obtain more observations
(eliminating assignable points when calculating center and spread)
the process's measured center or spread are known before hand, a priori, from some outside source --
so the control limits can be calculated before hand, then each new performance observation added to the graph as it is generated, over time.
Outpatient anticoagulation
0
1
2
3
4
5
6
7
0
1
2
3
4
5
6
7
INR
process capability = 60%
5.02.5
4.13.0
2.53.3
3.04.0
3.53.0
Patient LL: warfarin anticoagulation
warfarindose
courtesy of Dr. Larry Staker
INR
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
2.9 6.0 2.8 2.7 4.7 3.7 3.4 3.3 2.3 2.6 2.9 3.4 3.4 3.7 2.0 2.7 3.3 1.8 4.7 3.8 2.7 3.2 3.1 5.0 2.3
Anticoagulation XmR Chart - before
UCL 6.25
LCL 0.36
3.304
UCL 3.623
LCL 0.000
1.108
Patient LL: warfarin anticoagulation
INRs(X)
movingRange
(anterior MI - spec range = 2.5 - 3.5)
courtesy of Dr. Larry Staker
Stop tampering!
0
1
2
3
4
5
6
7
0
1
2
3
4
5
6
7
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
2.9 2.8 2.7 3.7 3.4 3.3 2.6 2.9 3.4 3.4 2.7 3.3 2.7 3.2 3.1 2.5
INR
process capability = 94%Patient LL: warfarin anticoagulation
warfarindose
3.0
courtesy of Dr. Larry Staker
INR
Sources of measured INR variation
1. the INR test itself- variation between different types of analyzers / batches of analyte- variation within a particular analyzer / batch of analyte
2. individual pill drug content, batch potency
3. patient differences- compliance- diet (e.g., green, leafy vegetables high in Vitamin K)- prescription medications, OTC medications, herbals- underlying genetic and physiologic differences (e.g., liver metabolism rates)
4. clinician dose tamperingIn this example, the initial process capability shortfall was 40% (100% - 60%); eliminating just one source of assignable variation - clinician tampering -
dropped the shortfall to 6% (a move of 34 percentage points); in other words, clinician tampering accounted for 85% (34/40) of the total shortfall
Choosing an SPC chart
Choice of SPC chart depends on the underlying frequency distribution of the data being analyzed
Parametric distributions give better power/resolution than nonparametric distributions, but contain risks associated with underlying assumptions
Frequency distribution usually depends on data type
Types of data
Data come in different "flavors" (types):NominalOrdinalIntervalRatio
Data type determines analysis
Different types of data contain different amounts of information
Choosing a parametric SPC chartBinary attribute data (nominal or ordinal) = binomial distribution
- asymtotically Gaussian (normal; bell-shaped) at large sample sizes;- turns into Poisson distribution if np < 5
use a P ("proportion") chart
Discrete ratio "# of between" data = geometric distribution- useful for very rare events; estimates point proportions at every event
use a g / h ("geometric") chart
Discrete ratio "# of per" data = Poisson distribution- asymptotically Gaussian (normal; bell-shaped) when mean > 25-30
use a C / U (think "count per unit") chart
Continuous ratio data = normal (Gaussian) distribution- very often doesn't fit, though ...
use an X ("mean") chart
Binomial distribution
y = Pr{X=k} = p (1-p)k n-kn!k!(n-k)!
Binary attribute data
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
# observed
0
0.05
0.1
0.15
Prop
ortio
n
mean = 0.19204n = 100
Red bead game
67
89
1011
1213
1415
1617
1819
2021
2223
2425
2627
2829
3031
3233
# red / 100
0
0.05
0.1
0.15
Prop
ortio
n
mean = 0.19204 n = 100
Red bead game
Sample #0
10
20
30
40
# re
d be
ads
/ 100
mean = 0.19204 n = 100
Red bead game
Sample #0
10
20
30
40
# re
d be
ads
/ 100
mean = 0.19204 n = 100
What if nothing fits?
Transform the datalog transformspower transformsseverity transformslinear, cyclic, or non-linear transforms
1.
Use Shewhart's method2.
Use a non-parametric control chart3.
generate control limits directly from the known frequency distribution
Use some other known frequency distribution4.