first steps in how to do a quality improvement project. dr s.a.cullis frcgp. associate postgraduate...

First Steps in how to do a Quality Improvement Project.

Dr S.A.Cullis FRCGP.Associate Postgraduate Dean

HEEM.Dr Susan Hadley

Programme DirectorHEEM

Acknowledgements

• Dr Martyn Diaper NHSIQ

• Alison Tongue NHSIQ

• Orlando Hampton HEEM

• Prof. Jeany Penny O.B.E University of Derby.

Learning Objectives

• To learn about Quality and Safety and to understand the basic structure of a Quality Improvement Project.

• To find out about BMJ quality website as a tool to help

• To learn where to find more information/resources.

Do registrars need to learn this?

The Gold Guide says• ‘take part in systems of quality assurance and

quality improvement in their clinical work and training.’

RCGP curriculum says• Understanding how and when to apply tools and

metrics to improve the quality of care is a key skill that can and should be learnt during your training, as well as enhanced in lifelong learning

Why do we reliably fail?

What causes harm to patients?Where should we focus our improvement

efforts?

Be a good personTrain well

Maintain good intentionsWork hard

What is wrong with this plane?

Lessons from Human Factors Research

‘You can’t change the human condition,but you can change the conditions

under which humans work’

James Reason

Error is normal... what are you going to do about

it?

Be a good personTrain well

Maintain good intentionsWork hardMake it easier to do the right thing

Make it harder to do the wrong thing

Spot & stop inevitable errors

So How Can We Improve Quality?

What are we trying to accomplish?

How will we know that change is an improvement?

What change can we make that will result in improvement?

The Model for Improvement

Langley, G., Nolan, K., and Nolan, T., 1994. The Foundation of Improvement, Quality Progress, June 1994

Change through small steps

Change ...• with a clear purpose• you can learn from (without fear of failure)• which is less exhausting• with fewer unintended consequences• which builds engagement and optimism

What are we trying toaccomplish?

How will we know that achange is an improvement?

What change can we make thatwill result in improvement?

Model for Improvement

Act Plan

Study Do

Understanding the problem. Knowing what you’re trying to do - clear and desirable aims and objectives

Measuring processes and outcomes

Change ideas: What have others done? What hunches do we have? What can we learn as we go along?

Langley G, Moen R, Nolan K, Nolan T, Norman C, Provost L, (2009), The improvement guide: a practical approach to enhancing organisational performance 2nd ed, Jossey Bass Publishers, San Francisco

PDSA cycle for learning and improvement

Act

• what changes are to be made?

• next cycle?

Plan•objective• questions and predictions (why)• plan to carry out the cycle (who, what, where, when)

Study

•complete the analysis of the data

•compare data to predictions

•summarise what was learned

Do

• carry out the plan• document problems and unexpected observations• begin analysis of the data

Repeated PDSA cycles work towards the AIM

PDSA

PDSA

PDSA

PDSA

PDSA

Data Driven Change

Hunches

Theories

Ideas

Aim•What am I trying to achieve?•How will I know a change is an improvement?•What changes can I make that will result in the improvement

Need to start small!!

Benefits of this approach

• easier to start• produces better solutions more quickly• engages people better• reduces waste• easier to continue

How to decide what to improve?

Significant Event Audit.

• Pringle’s SEA definition:• A process in which individual episodes (when

there has been a significant occurrence either beneficial or deleterious) are analysed in a systematic and detailed way to ascertain what can be learnt about the overall quality of care, and to indicate any changes that might lead to future improvements

Safety Walkrounds™

Try it yourself ...

The Pareto Principle

How not to change everything at once!

Making better decisions about prioritiesFocusing your effort where it matters

The Pareto Principle

The Principle

• We don’t have time to improve everything• A small number of issues account for the majority of

the challenge

• 20% of causes account for 80% of the problem

• We should focus on the ‘critical few’, not the ‘trivial many’

What does it look like?

Interruptions in surgeries

Tally by GPs of the causes of interruptions while seeing patients.

Category Count % of Total Cumulative %Cancellation msg 588 38.9 38.9Admin info 312 20.6 59.5Chaperone 198 13.1 72.6Sign script - urgent 78 5.2 77.8Other 72 4.8 82.6Sign script - contraception 72 4.8 87.4Clinical query - GP 66 4.4 91.8Equipment search 60 4 95.8Clinical query - NP 24 1.6 97.4Clinical query - learner 18 1.2 98.6Sign script - minor illness nurse 18 1.2 99.8Panic button 6 0.4 100.2TOTALS 1512 100

Cancel

lation m

sg

Admin info

Chapero

ne

Sign sc

ript -

urgent

Other

Sign sc

ript -

contra

ception

Clinica

l query

- GP

Equipmen

t sea

rch

Clinica

l query

- NP

Clinica

l query

- lea

rner

Sign sc

ript -

minor illness

nurse

Panic b

utton05

1015202530354045

0102030405060708090100

% of TotalCumulative %

What does it look like?

Cancel

lation m

sg

Admin info

Chapero

ne

Sign sc

ript -

urgent

Other

Sign sc

ript -

contra

ception

Clinica

l query

- GP

Equipmen

t sea

rch

Clinica

l query

- NP

Clinica

l query

- lea

rner

Sign sc

ript -

minor illness

nurse

Panic b

utton0

5

10

15

20

25

30

35

40

45

0

10

20

30

40

50

60

70

80

90

100

% of TotalCumulative %

Three categories of interruption (17%) account for 73% of the problem

Driver Diagrams

How to generate and organise ideas for improvement.

Driver Diagrams

AIM – an improved system

Primary driver 1

Primary driver 2

Secondary driver 1

Secondary driver 2

Secondary driver 3

Secondary driver 4

Secondary driver 5

CP1

CP2

CP3

CP4

CP5

CP6

CP7

AIM PRIMARYDRIVERS

SECONDARYDRIVERS

CHANGEPROJECTS

Pedometer

Gym work out 3 days

Squash weekends

No pub weekdays

Take packed lunch

Low fat meals

Buy only 1

sandwich

Water bottle for work bag

Fruit for dessert

Put away large wine

glasses

Put cycling days in diary

Cycling kit out night before

Get rid of Oyster card

Take stairs

2 stone weight loss in

6/12

Generate Change Ideas

Pedometer

Gym work out 3 days Squash

weekends

No pub weekdays

Take packed lunch

Low fat meals

Buy only 1

sandwich

Water bottle for work bagFruit for

dessert Put away large wine

glassesPut cycling days in diaryCycling

kit out night before

Get rid of Oyster card

Take stairs

Be more active during

the dayDo sport

Drink less alcohol

Substitute lower calorie

foods

Eat less

Marshall the mass of ideas

2 stone weight loss in

6/12

Driver DiagramsWeight loss example

Pedometer

Gym work out 3 days

Squash weekends

No pub weekdays

Take packed lunch

Low fat meals

Buy only 1 sandwich

Water bottle for work bag

Fruit for dessert

Put away the large

glasses

Put cycling days in diaryCycling kit

out night before

Get rid of Oyster card

Be more active during the day

Do sport

Drink less alcohol

Substitute lower calorie foods

Eat lessReduce calories

in

Increasecalories

out

Take stairs

2 stone weight loss in 6/12

Software for driver diagrams

–PowerPoint

–Dia–Freemind–VUE

–MindMeister–MindJet

Reliable Design

Deliberate reliable design

Segment?

What’s happening?

What’s the priority?

What’s going on?

eg How do at-risk infants get Vitamin D?

Deliberate reliable design – ???

Measurement for Improvement.

•70% of change fails

The traditions of measurement

• eg A-B comparison, average, huge dataset

Research

• eg one-to-many benchmarking comparison, average, large dataset

Judgement

• eg continual analysis of single changing process over time

Improvement

What mindsets are at play here?

Mindsets

Research

Improvement

Judgement

Research Judgement Improvement

Goal New knowledge (not its applicability)

ComparisonReward / punishmentSpur for change

Process understanding

Evaluating a change

Hypothesis Fixed None Multiple and flexible

Measures Many Very few Few

Time period Long, past Long/medium, past Short, current

Sample Large Large Small

Confounders Measure or control Describe and try to measure

Consider but rarely measured

Risks in improvement

settings

Ignores time based variation

Over-engineers data collection

Ignores time based variation

Over-reaction to natural variation

Incorrectly perceived as ‘inferior statistics’

Measurement mindsets

Based on L Solberg, G Mosser and S McDonald (1997) The Three Faces of Performance Measurement: Improvement, Accountability and Research, Journal on Quality Improvement, 23 (3): 135 - 147.

What to measure?

Structure Process Outcome

Avedis Donabedian

‘Outcomes remain the ultimate validators of the effectiveness and quality of medical care’ but they ‘must be used with discrimination’

The environment in which care occurs

What care is delivered, and how

The impact on patients and the

population

Balancing measures

OU

TPU

TS

HUMAN FACTORS

HUMANFACTORS

INTERNAL PROCESSES & PROCECEDURES

INPU

TSEXTERNALFACTORS

Process Measure(s)

Out

com

e M

easu

re(s

)

Balancing Measure(s)

What to measure?

Is it being done?

Is it working?

Unintended consequences?

Stru

ctur

e M

easu

re(s

)

system fit for use?

Period 1 Period 2

Poor performance

Good performance

Period 1 Period 2

Change made

Period 1 Period 2

Poor performance

Good performance

Improvement!

Improvement!

p<0.05

With summary data we make judgements about

improvementsWe may use summative statistics to justify it (e.g.

confidence intervals)But processes can change

over time and tell a different story!

Imagine this is mortality

data

Approaches to monitoring performance

Target

A change was introduced between the two periods. Was it a success?

48

Measurement for learning

Run charts: measurement for learning

0102030405060708090

Day

1 4 7 10 13 16 19

Seco

nds

to

answ

er p

hone

Seven one side

Seven down (or up)

DO

Look for a run of seven points all above or all below the centre line or all increasing or all decreasing

Just like a TPR chart

Average length of pre-ward stayStroke Ward

from 01/2007 to 07/2007

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7Months

How often you measure can also have an effect on how you look at the results

Mike Davidge NHS Institute for Innovation and Improvement

Average length of pre-ward stayStroke Ward

from 01/2007 to 07/2007

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

Weeks

Patient length of pre-ward stay Stroke Ward

from 01/2007 to 07/2007

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

Patients

Control charts

Statistical process control (SPC)

Understand

Improve

Motivate

Frequent measures

Average line

(mean/median)

Upper control

limit(3 SDs)

Lower control

limit (3 SDs)

0.13th percentile

97

.4%

C.I

.

Understanding variation

“Common Cause”

• inherent in the design of the process

• variation is predictable and the process is stable

• affects process most of the time

• many factors, some “unknowable”

• “noise in the system”

“Special Cause”

• not part of the process• unpredictable variation• intermittently apparent• “assignable” causes• can usually be

identified

The five rules

Rule 1: Any point outside a control limit.

Rule 2: A run of 8 points all above or below the centre line.

Rule 3: A run of 6 points increasing/decreasing.

Rule 4: 2 out of 3 consecutive points beyond +/- 2 sigma.

Rule 5: A run of 15 points all within +/-1 sigma of the centre line.

Software suggestions

There is a huge number of software packages which perform analysis for statistical process control.

Among them, some of the most accessible are:• Very cheap.

• Baseline www.valuesystemdesign.com/Baseline/Baseline_01.htm

• Cheap. • Winchart prismeurope.co.uk/software/

• More comprehensive. • Minitab. www.minitab.com

• Chart Runner. www.pqsystems.co.uk

BMJ Quality

• This is an online guide to doing a quality improvement project with links to BMJ Learning modules on relevant topics.

• It guides you through the process from start to finish and you can publish your project in their online Quality Journal.

• Licenses are available on a first come first served basis by discussing with your PDs.

BMJ Quality

• www.quality.bmj.com

Links to Other Useful Resources.

• http://www.ihi.org• http://

www.vle.eastmidlandsdeanery.nhs.uk/mod/folder/view.php?id=15885

• http://www.qihub.scot.nhs.uk/default.aspx• (this is a link on the VLE to QI powerpoint

slides and other useful info. It will soon be updated and moved to a specific QI location on Moodle)

GP Primary Care Educator Leads

• Dr Sue Cullis. APD scullis@nhs.net• Dr Susan Hadley. PD susan.hadley@nhs.net• Dr Christine Johnson.

christine.johnson@nottingham.ac.uk• Dr Graham Todd todds@doctors.org.uk• Dr David Young davidj.young@nhs.net• Dr Helen Tallantyre helen.tallantyre@nhs.net

Areas Covered by the PCELs

• Overall Lead- Sue Cullis• Leicester, Kettering, Northampton- Susan

Hadley & Sue Cullis• Nottingham, Lincoln & Boston- Christine

Johnson• Chesterfield (and North Notts)- Graham Todd• Derby ( and Mansfield)- David Young• (Mansfield & North Notts- Helen Tallantyre)

Looking Forward?

• Write down 3 things you are going to do as a result of today’s session?

• And by when are you going to do them?

• Please share 1 action with the group.

• “ The most important single change in the NHS in response to this report would be for it to become, more than ever before, a system devoted to continual learning and improvement of patient care, top to bottom and end to end.”

• Don Berwick. A promise to learn - a commitment to act. August 2013.

A Final Thought from Don Berwick