boards, dashboards, and data from the top: getting the board on board 1-3 p.m., june 11, 2007...
Post on 01-Jan-2016
231 Views
Preview:
TRANSCRIPT
Boards, Dashboards, and Data
From the Top: Getting the Board on Board1-3 p.m., June 11, 2007
Boston, Massachusetts
James L. Reinertsen, M.D.
Boards ask two types of questions about quality and safety
1. How good is our care?─ How do we compare to others like us?
2. Is our care getting better? ─ Are we on track to achieve our key quality
and safety objectives?─ If not, why not? Is the strategy wrong, or is it
not being executed effectively?
For all of these questions…
In God we trust.
All others bring data.
Yes, but what data?
Purpose of Measurement
Research Comparison or Accountability
Improvement
Key question “What is the truth?”
“Are we better or worse than…?”
“Are we getting better?”
Penalty for being wrong
Misdirection for the profession
Misdirected reward or
punishment
Misdirection for an initiative
Measurement requirements
and characteristics
Complete, accurate, controlled,
glacial pace, expensive
Risk adjusted, with denominators, attributable to
individuals or orgs, validity
Real time, raw counts,
consistent operational
definitions, utility
Typical displays Comparison of control and
experimental populations
Performance relative to
benchmarks and standards…
Run charts, control charts, time between
events…
Adapted from Solberg,Mosser, McDonald Jt Comm J Qual Improv. 1997 Mar;23(3):135-47.
Example of an answer to “How good is our care?”
Date of this report is October 24, 2006
Hospital could be “green” but still worse than median of comparison group
Compared to others
Another example of “How do we compare?”Hospital Adverse Events per 1,000 Patient Days
Adverse Events Include (but are not limited to):• Allergic rash• Excessive bleeding, unintentional trauma of a blood vessel• Respiratory depression requiring intubation due to pain medications• Hyperkalemia as the result of overdose of potassium• Lethargy/shakiness associated with low serum glucose• Drug-induced renal failure• Surgical site infection, sepsis, infected lines, other hospital-acquired infections• Internal bleeding following the first surgery and requiring a second surgery to stop the bleeding• Atelectasis, skin breakdown, pressure sores• DVT or pulmonary embolism during a hospital stay
Source: Roger Resar, John Whittington, IHI Collaborative
150
Number of Adverse Events per 1,000 Patient Days
Using IHI Global Trigger Tool
0 25 50 75100
125
Current IHI BestIHI Average
5 40
Our Hospital, May 2007
What Boards should know about data on “How good are we and how do we compare
to others?”
Upside• Often risk adjusted• Apples to Apples• Source of pride• Source of energy for
improvement
Downside• Time lag (months)• Static (no data over time)• If you look bad, energy is
wasted on “the data must be wrong”
• If you look good, you become complacent
• How you look depends on how others perform
• Standards and Benchmarks are full of defects (“The cream of the crap”)
Recommendations for Board use of “How do we compare to others?”
1. Ask this question to help you set aims, and perhaps annually thereafter, but don’t use these sorts of reports to oversee and guide improvement at each meeting.
2. Compare to the best, not the 50th %tile• e.g. Toyota Specs
3. Always make sure you know how “Green” is determined
Boards ask two types of questions about quality and safety
1. How good is our care?─ How do we compare to others like us?
2. Is our care getting better? ─ Are we on track to achieve our key quality
and safety objectives?─ If not, why not? Is the strategy wrong, or is it
not being executed effectively? Where dashboards and scorecards
can be helpful to boards
1.1 Satisfy Our Patients D
eath
s per
1000 D
isch
arg
es
YrMon.
2006200620062006200520052005200520042004200420042003200320032003DecSepJunMarDecSepJunMarDecSepJunMarDecSepJunMar
40
35
30
25
20
15
10
Monthly12mo rolling rate
Variable
Inpatient Mortality
Benchmark
8/7/2006; Prepared by Immanuel St. Joseph's-Mayo Health System Quality Resources Department
Immanuel St. J oseph's
Example: Immanuel St. Joseph’s Mayo Health System Board’s
answer to the question “Is our mortality rate getting better?”
Available in January 2007!
Is our quality and safety getting better?Are we going to achieve our aims?
• To answer these questions for Boards…─ The aims should be clearly displayed and understood─ A few system-level measure(s) should be graphically
displayed over time ─ The measures should be displayed monthly, at worst,
and should be close to “real time”─ Measures do not necessarily need to be risk adjusted─ Measures of critical initiatives (projects that must be
executed to achieve the aim) should be available if needed to answer the Board’s questions
The Board question “are we going to achieve our aims?” requires management to have a strategic theory
Big Dots(Pillars, BSC…)
Drivers(Core Theory of
Strategy)
Projects(Ops Plan)
What are your key strategic aims? How good must we be, by when? What are the system-level measures of those aims?
Down deep, what really has to be changed, or put in place, in order to achieve each of these goals? What are you tracking to know whether these drivers are changing?
What set of projects will move the Drivers far enough, fast enough, to achieve your aims? How will we know if the projects are being executed?
The ideal dashboard will display a cascaded set of measures that reflect the “theory of the strategy.”
Example Dashboard for Harm(for 5M Lives Campaign)
0
20
40
60
80
100
120
J an Feb Mar Apr May
Global HarmTrigger Tool
0
10
20
30
40
50
60
70
J an Feb Mar Apr May
Handwashing
42
44
46
48
50
52
54
56
58
J an Feb Mar Apr May
Culture ofdisciplineon safetyrules
0
10
20
30
40
50
60
70
J an Feb Mar Apr May
Teamwork
0
20
40
60
80
100
120
J an Feb Mar Apr May
Harm fromhigh alertmeds
0
2
4
6
8
10
12
14
16
18
20
J an Feb Mar Apr May
Surgicalcomplications
0
5
10
15
20
25
30
35
J an Feb Mar Apr May
PressureUlcers
0
2
4
6
8
10
12
J an Feb Mar Apr May
MRSA
0
2
4
6
8
10
12
14
16
J an Feb Mar Apr May
CHFReadmissions
System Level Measure: Global Harm Trigger Tool
Drivers: Handwashing, culture of discipline, and teamwork
Projects: High alert meds, surgical complications, pressure ulcers, CHF, MRSA
Board
The full Board should review the System-level Measures (Big Dots.) The Board
Quality Committee should review both the System-level Measures and the Key Drivers
of those Measures. Occasionally, but not often, the Board will need to see measures of Key Projects, but these are generally the
responsibility of management to oversee and execute.
Common Flaws in Dashboards• No system-level measures or aims (so it’s possible to quality
and safety to be worse, and yet to achieve “green” on all the measures the Board sees!)
• Hodge-podge of system, driver, and project measures (so the Board doesn’t know what’s important)
• Static measures (so the Board has to take management’s word that “we’re on track to achieve our aims”
• Too many measures (so the Board doesn’t understand any of them)
• Mixture of “How do we compare to others” and “are we getting better?” measures (so the Board doesn’t know what questions to ask)
• Low, unclear standards for “green” (so the Board becomes complacent despite significant opportunities for improvement!)
Can you identify the flaws in the following “dashboard?”
Measure Current Performance Goal for 2007 Acute MI Core Measures
6h Decile National, 4h decile State 2nd state decile or above
Congestive Heart Failure Core Measures
4th Decile National, 2nd decile State 2nd State decile or above
Pneumonia Core Measures
3rd Decile National, 1st Decile State 2nd State decile or above
Press-Ganey Patient Satisfaction
57% Rate us “Excellent” Statistically significant improvement i.e 62% “Excellent” rating
OR Turnover Time 22 minutes 15 minutes Falls 7 per 1000 patient days Less than 5 per 1000 patient days Medication Errors 5.1 per 1000 patient days (from Nurse
Variance Reports) Less than 7 per 1000 patient days
Total Knee and Hip Infection Rates
1.2% Less than 4.1 % i.e. Better (lower) than 50th %tile for NNIS
Surgical Site Infection Rates for Cardiac Surgery
4.2% Less than 10.4% i.e. Better (lower) than 50th %tile for NNIS
Time to answer nurse call lights on all Med/Surg Units
We are developing a standard measure, and will report in future meetings to Board on this initiative
We are aiming to achieve significant improvement in timeliness of response to patients concerns.
Measure Current Performance Goal for 2007 Acute MI Core Measures
6h Decile National, 4h decile State 2nd state decile or above
Congestive Heart Failure Core Measures
4th Decile National, 2nd decile State 2nd State decile or above
Pneumonia Core Measures
3rd Decile National, 1st Decile State 2nd State decile or above
Press-Ganey Patient Satisfaction
57% Rate us “Excellent” Statistically significant improvement i.e 62% “Excellent” rating
OR Turnover Time 22 minutes 15 minutes Falls 7 per 1000 patient days Less than 5 per 1000 patient days Medication Errors 5.1 per 1000 patient days (from Nurse
Variance Reports) Less than 7 per 1000 patient days
Total Knee and Hip Infection Rates
1.2% Less than 4.1 % i.e. Better (lower) than 50th %tile for NNIS
Surgical Site Infection Rates for Cardiac Surgery
4.2% Less than 10.4% i.e. Better (lower) than 50th %tile for NNIS
Time to answer nurse call lights on all Med/Surg Units
We are developing a standard measure, and will report in future meetings to Board on this initiative
We are aiming to achieve significant improvement in timeliness of response to patients concerns.
No display over time
Low standards for “Green”
Mix of system, project measures
Mostly comparison measures
Summary of Best Practices for Quality and Safety Dashboards for Boards
• Separate the two types of oversight questions─ How good is our quality? How do we compare to others?─ Are we getting better? Are we on track to achieve our aims?
• Ask the comparison question annually, when setting quality and safety aims. Avoid use of comparative data to track improvement.
• Frame your aims with reference to the theoretical ideal, and to the “best in the world,” not to benchmarks
• Ask the ‘improvement question’ at every meeting, and track with a dashboard that shows real-time data on system level and driver measures displayed on run charts
• Demand that management develop a “theory of the strategy to achieve the annual quality and safety aims
• Do not put project-level measures (often about one unit, disease, or department) on the Board’s dashboard
top related