data management for quality improvement: tools you should ... · data management for quality...

Data Management for

Quality Improvement:

Tools You Should Be Using Now

Munish Gupta, MD MMSc

Heather Kaplan, MD MSCE

Neonatal Quality at Hot Topics

December 6, 2015

Disclosure Statements

Munish Gupta has no relevant financial relationships with the manufacturer(s) of any commercial product(s) and/or provider(s) of commercial services discussed in this CME activity.

Heather Kaplan has the following financial relationships with the manufacturer(s) of any commercial product(s) and/or provider(s) of commercial services discussed in this CME activity:

• Consultant for Vermont Oxford Network NICQ quality collaboratives.

We do not intend to discuss an unapproved/investigative use of a commercial product/device in our presentation.

We Want to Cover

1. Run charts

2. Control charts

What they are and how to use them!

Introductions

Munish Gupta

Heather Kaplan

All of you?

I am a…..

A. Neonatologist

B. Fellow or resident

C. Nurse practitioner, physician assistant

D. Nurse

E. Student

F. Other

A. B. C. D.

25% 25%25%25%

My experience level with data for quality improvement (QI)…. A. What’s QI?

B. Some, but I’ve never made a control chart

C. A fair amount, I make control charts now and then

D. A good amount, I use control charts fairly regularly

E. I’m a Jedi master A. B. C. D.

25% 25%25%25%

We assume you’re comfortable with…

Model for improvement

• Setting aims

• Developing measures

• Testing changes using PDSA cycle

Measuring for improvement

• Outcome, process, and balancing measures

• Measurement over time

The Model for Improvement

AIMS

MEASURES

CHANGES

Testing Changes

Figure from Institute for Healthcare Improvement (www.ihi.org)

A (Real) NICU Example

You would like to reduce the incidence of NEC

in your NICU.

You have identified two evidence-based

strategies for reducing risk of NEC:

• Increasing the use of human milk; and

• Standardizing feeding practices.

Key Driver Diagram

SMART Aims Primary Drivers Secondary Drivers/Interventions

Use of Human Milk (HM)

Process Measure: % of VLBW infants receiving HM at discharge

Decrease rate of NEC in VLBW infants by 25% by January 2014

Outcome measure: NEC rate per 100 VLBW days

Human Milk Initiation

Process Measure: % of VLBW infants w/ first feeding of HM Process Measure: time to first use of HM for oral care

Milk Continuation

Process Measure: # of days held skin-to-skin in first month

Standardized Feeding Protocol

Process Measure: % of infants who followed feeding protocol

Standardized feeding advancement

Standardized fortification

Donor milk use

First Measure: First Feeding as HM

Month First Feeding HM Total Infants

Jan-11 15 53

Feb-11 9 25

Mar-11 15 47

Apr-11 17 55

May-11 17 42

Jun-11 18 45

Jul-11 17 43

Aug-11 21 54

Sep-11 18 41

Oct-11 20 49

Nov-11 13 39

Dec-11 14 40

What’s good about this approach?

What’s missing?

Statistical Process Control Theory

History of SPC

Manufacturing origins

1920s - Walter Shewhart,

W.E. Deming (Bell Labs)

Easy for non-statisticians

detect process changes

Ramped up extensively during

WWII, post-war Japan, U.S. mfg

Used in all industries, including health care

Walter Shewhart

Statistical Process Control (SPC) and QI

Measurement over

time critical for QI

But all things vary

SPC: analysis of data

over time

Understand variation

Measuring Change in Variation

We are looking for improvement / change in

key data

But natural background variation in all things

we do – fact of life

Need tools to interpret process changes (in

data) versus natural variation in data

We would like to detect true change fast

Definitions

1. Common Cause Variation: Causes inherent as

part of usual process (good or bad).

2. Special Cause Variation: Specific causes not

part of usual process (good or bad).

3. Stable Process: Predictable variation within

natural common cause bounds.

4. Unstable Process: Both special and common

cause variation, variation unpredictable.

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Min

ute

s

Time to Get to Work, Daily

Why is this Important

Type of variation type improvement action

Type of

variation

Reduce unnatural

variation

Reduce natural

variation, improve

basic process

Common

cause

Special

cause Establish stable

work process

Improve overall

outcomes

SPC Tools for Measurement

1. Run charts – minimal standard

2. Control charts

Keys:

Plot and evaluate over time

Interpret visually and statistically

Run Charts

Run Charts

Visual display of data over time

Center line: median of data

Can include annotations, goal line

Why Important

Interpreting Run Charts

Perla et al, BMJ Qual Saf 2011; 20:46-51

≥ 6 points ≥ 5 points

Too many or too few

Too few, too many runs

Perla et al, BMJ Qual Saf 2011; 20:46-51

Key Driver Diagram








Milk Continuation






Donor milk use

First Measure: First Feeding as HM

Month First Feeding HM Total Infants

Jan-11 15 53

Feb-11 9 25

Mar-11 15 47

Apr-11 17 55

May-11 17 42

Jun-11 18 45

Jul-11 17 43

Aug-11 21 54

Sep-11 18 41

Oct-11 20 49

Nov-11 13 39

Dec-11 14 40

How would you interpret this chart?

A. There is a shift – 6 points on one side of median.

B. There is a trend - 5 points increasing or decreasing.

C. There seem to be too few or too many runs.

D. There is an astronomical data point.

E. There is no special cause.

A. B. C. D.

25% 25%25%25%

Run Charts

“Minimum standard” for QI project data

(particularly to publish)

Can start with first few data points!

Need at least 10 data points to use rules for

detecting special cause

Simple to create (no software needed)

Can be used with all types of data

But… not as powerful as a control chart

Control Charts

Control Charts

The Shewhart chart (a.k.a. control chart) is a

statistical tool used to distinguish between

common cause and special cause variation

Chart Title

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Run Order

Me

as

ure Center Line

Upper Limit

7.86

Lower Limit

Provost, LP and Murray S. The Data Guide. 2008

Val

ue

of

Re

sult

Unit of Time (e.g. days, weeks, months, quarters)

A control chart is a run chart with some differences.

Run chart: Center line is the median.

Control chart: Center line is often the mean.

“Control limits” that reflect inherent variability in data – need to be calculated, but key to effectiveness

Mean

Upper Control Limit (UCL)

Lower Control Limit (LCL)

Slides Courtesy of Yiscah Bracha, PhD. CCHMC

From Run Charts to Control Charts

Relationship to Probability Theory

9181716151413121111

65

60

55

50

45

40

35

Time

_X=49.77

UCL=58.77

LCL=40.77

1

1

1

1

11

1

Constructing a control chart

Underlying data distribution dictates population parameters. Parameters dictate:

• Measure of central tendency (the “centerline”)

• Measure of variability standard deviation values for the upper and lower control limits.

Underlying distribution depends on type of data being observed (e.g., normal/Gaussian, Poisson, binomial, geometric)

Need to know what type of data you have to construct the proper type of control chart!

Continuous Data

1. Numerical value for each unit in a group

Discrete (Integer) Data

2. Classification: Presence or not of an attribute

3. Count: How many attributes occur in sample

Type of data

Sample Size

Type of Chart

Math (software)

Constructing Control Charts

Types of Data & Control Charts

Common cause probability

model Example

Dis

cre

te

Classification: Binomial

Parameter: p

Patient develops an SSI (Y/N)

Count: Poisson

Parameter: l

Number of catheter-associated

HAIs

Continuous

Normal

Parameters: m, s

Time to deliver thrombolytics

Healthcare Systems Engineering Institute

Which Control Chart To Use

Adapted from Provost & Murray, The Health Care Data Guide, 2011, and Carey, Improving Healthcare with Control Charts, 2003.

Type of

Data Discrete /

Attribute (data is counted or

classified)

Continuous /

Variable (data is measured

on a scale) Count

(events/errors

are counted;

numerator can be

greater than

denominator)

Classification (each item is

classified;

numerator cannot

be greater than

denominator) Equal or

fixed area

of

opportunit

y

Unequal

or variable

area of

opportunit

y

Equal or

unequal

subgroup

size

Subgroup

size = 1 (each subgroup

is single

observation)

Subgroup

size > 1 (each subgroup

has multiple

observations)

C chart Count of

events

U chart Events per

unit

P chart Percent

classified

X and MR

charts Individual

measures and

moving range

X-bar and S

charts Average and

standard

deviation

Control Charts for Attribute Data (1)

Classification data

• P chart: Percent of observations with a given attribute

• Measure of variability comes from binomial distribution

Example: Late-Onset Sepsis

Performance Metric: Late-Onset Nosocomial Sepsis

What it means operationally: Of all infants discharged in a given month, the percent that had at least one infection during their hospitalization.

Subgroup: The variable number of infants.

Summary stats:

P: The percent of infants with an infection

What we want to see: Percent of infants with a nosocomial infection.

Month

Infants

with Late

Infection Patients

Discharged

4/1/2006 10 61

5/1/2006 13 81

6/1/2006 19 94

7/1/2006 20 78

8/1/2006 7 77

9/1/2006 18 77

10/1/2006 16 84

11/1/2006 12 83

12/1/2006 15 76

1/1/2007 17 90

2/1/2007 16 73

3/1/2007 16 100

4/1/2007 13 75

5/1/2007 16 99

6/1/2007 12 88

7/1/2007 22 105

8/1/2007 16 91

9/1/2007 19 93

Example: Late-Onset Sepsis

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

4/1

/06

5

/1/0

6

6/1

/06

7

/1/0

6

8/1

/06

9

/1/0

6

10

/1/0

6

11

/1/0

6

12

/1/0

6

1/1

/07

2

/1/0

7

3/1

/07

4

/1/0

7

5/1

/07

6

/1/0

7

7/1

/07

8

/1/0

7

9/1

/07

1

0/1

/07

1

1/1

/07

1

2/1

/07

1

/1/0

8

2/1

/08

3

/1/0

8

4/1

/08

5

/1/0

8

6/1

/08

7

/1/0

8

8/1

/08

9

/1/0

8

10

/1/0

8

11

/1/0

8

12

/1/0

8

1/1

/09

2

/1/0

9

3/1

/09

4

/1/0

9

5/1

/09

6

/1/0

9

7/1

/09

8

/1/0

9

9/1

/09

1

0/1

/09

1

1/1

/09

1

2/1

/09

Month

Y Axis: Proportion of

Infants D/C with Late-Onset

Infection

Centerline: Average Proportion

of Infants with Late-Onset

Infection (over 45 months)

Control Limits

P-chart: % of VLBW infants with Late-Onset Infection

Centerline = p-bar = Average of the Statistic

UCL = CL + 3 σs

LCL = CL - 3 σs

100 n

dp

in

pppUCL

1003

in

pppLCL

1003

σs from the

binomial

distribution

Provost, LP and Murray S. The Data Guide. 2008 Slide courtesy of Terri Byczkowski, PhD, CCHMC

P-chart Calculations

Control Charts for Attribute Data (2)

Count Data

• Chart names contain

• “C”, as in Count

• “U”, as in Unit

• Points on chart represent

• Raw number of instances

• Number of instances per opportunities to observe

• Measure of variability comes from Poisson distribution

Example: Catheter-Associated Infections

Metric: Catheter-Associated infections

How it is observed: Data obtained from infection control as reported to CDC. Each day, number of catheters is counted. This is used to obtain catheter days each month. Number of infections (catheter-associated) occurring each month is also reported.

How often it is observed: For 24 months

Subgroup: Monthly

Summary stats: Unit Count: number of infections per opportunity (catheter day)

What we want to see: Number of catheter-associated infections per 1000 catheter days

Month #

Infections Catheter

Days

10/1/2008 8 2212

11/1/2008 15 3064

12/1/2008 15 3007

1/1/2009 6 2783

2/1/2009 14 2499

3/1/2009 4 2692

4/1/2009 8 2784

5/1/2009 14 2772

6/1/2009 9 2690

7/1/2009 10 3145

8/1/2009 16 3171

9/1/2009 12 3209

10/1/2009 11 3076

11/1/2009 17 2749

12/1/2009 7 2759

Example: Catheter-Associated Infections

Y Axis: CA-Infection Rate

per 1000 line days

Centerline: Average CA-

Infection rate (over 24 months)

Control Limits

U-Chart

Control Charts for Continuous Data

Chart names contain “X”, as in individual value or sample mean

• X-MR (individual value & moving range)

• Xbar-S (average & standard deviation)

Points on chart represent

• Individual values

• Averages

Measure of variability comes from normal (Gaussian) distribution

Example: C-section Decision to Incision

Performance Metric: Time to incision following decision for emergent c-section

What it means operationally: Minutes between decision to do c-section and time of incision

How it is observed: 10 charts sampled per week

Subgroup: The 10 charts sampled each week

Summary stats for the subgroups: X-bar: The average decision to incision time for

the 10 charts sampled each week S: The standard deviation of decision to incision

times for the 10 charts sampled each week.

What we want to see: process behavior over 30 weeks

Week

Decision to

Incision Time

(minutes)

X-barStandard

Deviation

40

45

36

49

50

35

34

50

39

47

49

52

36

49

50

42

38

50

44

50

44

52

36

49

46

41

36

52

43

51

1

2

3

42.5 6.4

46 5.6

45 6.1

Example from Benneyan, Int J Six Sigma and Competitive Advantage, 2008

C-section Incision: X-Bar & S Charts

S chart looks at the variation within subgroups. High variation within subgroups it makes it difficult to interpret variation between subgroups.

X-bar chart looks at the variation between subgroups.

Example from Benneyan, Int J Six Sigma and Competitive Advantage, 2008

Why two charts (Xbar & S)?

2 types of possible

process changes

(unnatural variation)

Mean or standard

deviation

Either can change

without the other

One chart to detect

each type of change

Change in mean

Change in SD

Why two charts (Xbar & S)?

Mean change, SD not SD change, Mean not

//upload.wikimedia.org/wikipedia/commons/9/93/Xbar_chart_for_a_paired_xbar_and_R_chart.svg

//upload.wikimedia.org/wikipedia/commons/9/9f/S_chart_for_a_paired_xbar_and_s_chart.svg

How to Interpret a Control Chart

Similar to run charts

Probability-based rules

Goal to detect non-random patterns

Rules designed to balance Type I (alpha

error, p<0.05) and Type II errors

“Rules” for Detecting Special Cause

TEST 1: 1 point outside outer control limit

TEST 2: 2 out of 3 points more than 2 SD from center line

TEST 3: 4 out of 5 points more than 1 SD from center line

TEST 4: Run of 8 points in a row on one side of center line

TEST 5: Trend of 6 points in a row increasing or decreasing

TEST 6: 14 points in a row alternating up and down

“Rules” for Detecting Special Cause (2)

Quiz: Interpretation

Points outside control limits?

Runs of 8 or more consecutive points on one side of the centerline?

Trends of 6 or more consecutive points increasing or decreasing?

Two of three consecutive points near the outer control limits?

Yes

Yes

No

Benneyan JC, et al. Qual Saf Health Care. 2003;12:458-464.

LCL

UCL

Yes

Quiz: Interpretation

This process appears to be in control, i.e. no special cause variation, only common cause variation.

Points outside control limits?

Runs of 8 or more consecutive points on one side of the centerline?

Trends of 6 or more consecutive points increasing or decreasing?

Two of three consecutive points near the outer control limits?

No

No

No

Benneyan JC, et al. Qual Saf Health Care. 2003;12:458-464.

No

ACTION

NO Special Cause

is occurring in

System

Special Cause is

occurring in System

Take action on

individual outcome

(treat special)

MISTAKE 1

OK

Treat outcome as

part of system;

work on changing

the system (treat

common)

OK

MISTAKE 2

ACTUAL SITUATION

Provost, LP and Murray S. The Data Guide. 2008

Using Control Charts to Guide QI

Key Driver Diagram








Milk Continuation






Donor milk use

For our measure of the percent of infants whose first feeding is human milk, what type of control chart(s) would you use?

A. U-chart

B. C-chart

C. P-chart

D. X-bar and S charts

A. B. C. D.

25% 25%25%25%


Type of

Data Discrete /


classified)

Continuous /


on a scale) Count

(events/errors

are counted;

numerator can be

greater than

denominator)


classified;

numerator cannot

be greater than


fixed area

of

opportunit

y

Unequal

or variable

area of

opportunit

y

Equal or

unequal

subgroup

size

Subgroup


is single

observation)

Subgroup


has multiple

observations)

C chart Count of

events

U chart Events per

unit

P chart Percent

classified

X and MR

charts Individual

measures and

moving range

X-bar and S

charts Average and

standard

deviation


Any evidence of special cause?


A. There is no special cause variation.

B. There are one or more points outside control limits.

C. There is a shift of 8 points above the median.

D. There is a trend of 6 increasing points. A. B. C. D.

25% 25%25%25%

Key Driver Diagram








Milk Continuation






Donor milk use

Measure: Time to Oral Care with HM

Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11

21 88 88 72 60 40 24 110 28 46 30 63

4 150 69 64 75 40 35 54 25 27 28 45

100 168 60 44 90 14 22 100 60 90 18 44

75 102 33 126 14 24 40 28 42 60 44 57

56 132 34 78 25 16 40 35 25 36 25 96

168 104 6 24 56 50 84 92 45 20 10 54

63 160 68 40 40 138 60 60 30 48 40 15

130 111 16 69 55 30 40 80 55 32 85 10

81 81 80 48 12 105 120 84 21 100 32 72

20 60 140 21 54 35 35 40 15 42 20 100

10 18 80 60 68 27 40 48 30 125 51 110

70 21 84 36 27 22 34 42 18 38 42 12

102 45 6 48 30 95 48 30 16 24 24 54

135 81 40 84 34 42 38 60 18 36 90 48

18 27 48 54 48 14 64 92 44 8 100 18

120 120 69 14 110 90 48 22 56 15 28

60 12 36 36 75 64 27 42 20 90

95 120 65 33 18 95 72 102 65 44

140 45 76 84 36 52 95 50 42

70 38 78 100 57 9

120 28 72 48

28 36

64

Time to Oral Care with Colostrum/Human Milk (hours)

For our measure of the time to oral care with human milk, what type of control chart(s) would you use?

A. U-chart

B. C-chart

C. P-chart

D. X-bar and S charts

A. B. C. D.

25% 25%25%25%


Type of

Data Discrete /


classified)

Continuous /


on a scale) Count

(events/errors

are counted;

numerator can be

greater than

denominator)


classified;

numerator cannot

be greater than


fixed area

of

opportunit

y

Unequal

or variable

area of

opportunit

y

Equal or

unequal

subgroup

size

Subgroup


is single

observation)

Subgroup


has multiple

observations)

C chart Count of

events

U chart Events per

unit

P chart Percent

classified

X and MR

charts Individual

measures and

moving range

X-bar and S

charts Average and

standard

deviation


Focusing just on the X-bar chart, is there evidence of special cause variation?

A. There is no special cause variation.

B. There are one or more points outside control limits.

C. There is a shift below the median.

D. There is a trend of decreasing points.

E. B and C. A. B. C. D. E.

20% 20% 20%20%20%


A. Neither mean nor

variation has improved.

B. Mean is improving but

variation is unchanged.

C. Variation has improved

(reduced), but mean is

unchanged.

D. Both variation and

mean have improved. A. B. C. D.

25% 25%25%25%

Why Control Charts Over Run Charts?

More sensitive / more powerful for detecting

special cause

Estimate capability of a stable process more

accurately predict performance

But… more difficult to generate

The Goal: Standardize then Improve

Standard

process

Improved

process

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58

Subgroup number

1 2 3 Sm

alle

r is

bett

er

Unstable

process

c/o J. Benneyan

Testing your New

Knowledge

What type of data is this? Time to first dose of surfactant among 10 VLBW infants intubated in delivery room.

A. Discrete - count

B. Discrete - classification

C. Continuous/variable

D. I wonder what the football score is…

A. B. C. D.

25% 25%25%25%


Type of

Data Discrete /


classified)

Continuous /


on a scale) Count

(events/errors

are counted;

numerator can be

greater than

denominator)


classified;

numerator cannot

be greater than


fixed area

of

opportunit

y

Unequal

or variable

area of

opportunit

y

Equal or

unequal

subgroup

size

Subgroup


is single

observation)

Subgroup


has multiple

observations)

C chart Count of

events

U chart Events per

unit

P chart Percent

classified

X and MR

charts Individual

measures and

moving range

X-bar and S

charts Average and

standard

deviation


Time to first dose of surfactant among 10 VLBW infants intubated in delivery room.

What type of data is this? Number of unplanned extubations each month per number of ventilator days.

A. Discrete - count



D. Wow, I really can’t wait for the reception tonight…

A. B. C. D.

25% 25%25%25%


Type of

Data Discrete /


classified)

Continuous /


on a scale) Count

(events/errors

are counted;

numerator can be

greater than

denominator)


classified;

numerator cannot

be greater than


fixed area

of

opportunit

y

Unequal

or variable

area of

opportunit

y

Equal or

unequal

subgroup

size

Subgroup


is single

observation)

Subgroup


has multiple

observations)

C chart Count of

events

U chart Events per

unit

P chart Percent

classified

X and MR

charts Individual

measures and

moving range

X-bar and S

charts Average and

standard

deviation


Number of unplanned extubations each month per number of ventilator days.

What type of data is this? Percent of infants admitted each month with initial temperature < 36.0 ⁰C.

A. Discrete - count



D. Are they really still talking up there?

A. B. C. D.

25% 25%25%25%


Type of

Data Discrete /


classified)

Continuous /


on a scale) Count

(events/errors

are counted;

numerator can be

greater than

denominator)


classified;

numerator cannot

be greater than


fixed area

of

opportunit

y

Unequal

or variable

area of

opportunit

y

Equal or

unequal

subgroup

size

Subgroup


is single

observation)

Subgroup


has multiple

observations)

C chart Count of

events

U chart Events per

unit

P chart Percent

classified

X and MR

charts Individual

measures and

moving range

X-bar and S

charts Average and

standard

deviation


Percent of infants admitted each month with initial temperature < 36.0 ⁰C.

What type of data is this? Average initial temperature of all VLBW infants admitted each month.

A. Discrete - count



D. I wonder what the football score is…

A. B. C. D.

25% 25%25%25%


Type of

Data Discrete /


classified)

Continuous /


on a scale) Count

(events/errors

are counted;

numerator can be

greater than

denominator)


classified;

numerator cannot

be greater than


fixed area

of

opportunit

y

Unequal

or variable

area of

opportunit

y

Equal or

unequal

subgroup

size

Subgroup


is single

observation)

Subgroup


has multiple

observations)

C chart Count of

events

U chart Events per

unit

P chart Percent

classified

X and MR

charts Individual

measures and

moving range

X-bar and S

charts Average and

standard

deviation


Average initial temperature of all VLBW infants admitted each month.

Review

Run charts: minimum standard for QI

Control charts: powerful tool for QI,

somewhat complex but not that hard

Other Topics We Cannot Cover

Topic Possible approaches

Other SPC Tools • Comparison or funnel charts

• Pareto charts

Sample Size • Sample size charts

Fixing and revising

limits

• Rules for “fixing limits” and

comparing new points to existing

mean vs. “updating limits”

• Setting new limits when evidence of

special cause

Other Topics We Cannot Cover

Topic Possible approaches

Rare Event Charts • G and T Charts (NEC example)

Non-normal data • Larger samples

• EWMA, Cusum charts

• Transformations, probability limits

Natural trending

of cyclical data

• Trend control charts

• Auto-correlated charts

Thanks!

References

For more information on this topic, see the following publications:

1. Benneyan, J.C., R.C. Lloyd, and P.E. Plsek, Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care, 2003. 12(6): p. 458-64.

2. Benneyan, J.C., The design, selection, and performance of statistical control charts for healthcare process improvement. Int J Six Sigma and Competitive Advantage, 2008. 4(3):p.209-239.

3. Carey, R.G., Improving healthcare with control charts : basic and advanced SPC methods and case studies. 2003, Milwaukee, WI: ASQ Quality Press. xxiv, 194 p.

4. Langley, G.J., R.D. Moen, K.M. Nolan, T.W. Nolan, C.L. Normal, and L.P. Provost, The Improvement Guide. 2nd ed. 2009, San Francisco, CA: Jossey-Bass. 490 p.

5. Lee, K. and C. McGreevey, Using control charts to assess performance measurement data. Jt Comm J Qual Improv, 2002. 28(2): p. 90-101.

6. Lee, K.Y. and C. McGreevey, Using comparison charts to assess performance measurement data. Jt Comm J Qual Improv, 2002. 28(3): p. 129-38.

7. Perla, R.J., L.P. Provost, and S.K. Murray, The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf, 2011. 20(1): p. 46-51.

8. Provost, L.P. and S.K. Murray, The health care data guide : learning from data for improvement. 1st ed. 2011, San Francisco, CA: Jossey-Bass. 445 p.

data management for quality improvement: tools you should ... · data management for quality...

Documents