key performance indicators, a spy in the camp or a useful tool?

Paul A Blackett NPAG Clinical Engineering Conference 2011

Performance Indicators –

a Spy in the Camp or a Useful Tool?

Paul A Blackett

Lancashire Teaching Hospitals NHS Foundation Trust


Overview

• Definitions of KPIs

• How other industries are using them

• Tips for good KPIs

• Suggestions for Clinical Engineering KPIs & Discussion


What are Key Performance

Indicators?

A specific measure of some aspect of an organisation's performance.

“A quantitative tool (for example rate, ratio, index,

percentage) that provides an indication of an organisations performance in relation to a specified

process or outcome.” [1]

“”


Why do we need to use them?

• Controls Assurance (up until 2004)

• SfBH now CQC

• CNST now NHSLA

• ISO 9001:2008

• How do you know how well you are doing?


A look at how others do it!


Aircraft Safety


A few examples….. [2]


-


Railway Safety


Signals Passed at Danger [3]


Tips for good KPIs

[4]


RelevantFocus on the right topics

• What is important to your dept?

• What is it you are wanting to achieve?• What are your core objectives?

The 3 E’s

• Economy (eg cost of repairs this year)• Efficiency (cost per repair)

• Effectiveness (users satisfied with cost of repair)

Tips for good KPIs


Tips for good KPIs

Relevant

BalancedA Balanced Scorecard

• Originally from the private sector

• Four perspectives;

Servi

ce u

sers

Inte

rnal

Man

agem

ent

Con

tinuo

us im

prov

emen

t Fin

anci

al (t

axpa

yers

)


Relevant

Understandable

Balanced

Tips for good KPIs

Easy to understand• Avoid Jargon

• Straightforward

Clearly defined

• How the data is obtained

Verifiable

• Have you the evidence?


Tips for good KPIs

Relevant

Understandable

Balanced

Comparable

My KPI Your KPI

Between Organisations

Over time

My Old

KPI


Relevant

Understandable

Balanced

Timely

Comparable

Tips for good KPIs

Nobody wants to read yesterdays news!

• How responsive are your PI figures?

• Weekly data may be too

late if it takes a week to calculate them!


Relevant

Action focused

Understandable

Balanced

Timely

Comparable

Tips for good KPIs

What are you going

to do with the data?

Don’t stifle

Innovation!

Avoid perverse

incentives


Action focused

If you are setting targets -are they reasonable?

Tips for good KPIs

Is it a pass/fail or is there

some middle ground?


Relevant

Action Focused

Understandable

Balanced

Timely

Comparable

Viewpoint

Tips for good KPIs

Who is the data for?• The Public

• Technical Staff• Your Manager

What are their needs?

Tailor your KPIs to your audience!


Table form

Presenting KPIs


GraphicallyPresenting KPIs


DashboardsPresenting KPIs


From Theory to Practise!

Requirements for Clinical Engineering KPIs

• Simple

• Common functions• Suitable for all to use

• Not onerous to produce

• Allow for local adjustment• Reported once a year


Suggested National Set of Clinical and Medical Engineering KPIs

for discuss

ion!

for discuss

ion!

for discuss

ion!

for discuss

ion!

Service User Internal Management

Continuous Improvement Financial

• Response Time - from requesting phone call to first action.• Time to Completion – from requesting phone call to completed job.

• Scheduled maintenance completion – percentage attempted in a 12 month period.• Repair jobs open for more than 3 months.

• Annual Customer Satisfaction survey

• Time spent on Engineer training – hrs per year

• Annual cost per device

• Annual cost per bed


Response Time

The time taken from the first contact reporting an unscheduled job to

the time that it is first attended to. Measured in days. Expressed as an

average for the financial year.

Option - Exclude non-working days

Option – Categorise into priorities, for example Urgent, Routine, Low

Local Targets set to indicate, KPI met, borderline, or fail


Time to Completion

The time taken from the first contact reporting an unscheduled job to

the time that it is completed. Measured in days. Expressed as an

average for the financial year.

Option - Exclude non-working days

Option - Categorise into priorities, for example Urgent, Routine, Low

Option – Exclude jobs where external influences delay work



Customer Satisfaction Survey

An annual survey should be undertaken to determine user satisfaction

with the service. This survey should be tailored to the organisations

needs.

Pass / Fail

Time spent on Engineer Training

The number of hours over the financial year that a engineers have

been undergoing training or other educational process. Expressed as

hours per full time equivalent.

Option – Categorise into external and in-house providers.

Local target - Compare to other departments


Scheduled Maintenance Completion

The number of scheduled jobs attempted over the financial year.

Expressed as a percentage of those due in that year.

Option – Categorise, eg completed, not available, missing.


Repair Jobs open over 3 months

The number of ongoing unscheduled jobs whose current time to

completion is over 3 months at the financial year end. Expressed as a

simple count.

Option – Exclude incident investigations



Annual Cost Per Device

The average cost of maintaining a medical device over the financial

year. Expressed as cost per device.

State whether systems are considered single devices


Annual Cost Per Bed

The average cost of maintaining a medical device over the financial

year. Expressed as cost per bed.

State type of Trust eg Teaching, Acute, DGH



Finally…a quote…

“We are not in competition with each other. Rather, we are all

striving toward a theoretically correct performance. We each

get to set our own individual goals within the larger scope ..”


[1] http://www.jcaho.org/pms/core+measures/appendixd.pdf[2] https://www.directives.doe.gov/directives/archive-directives/440.2b-EGuide-1a2/at_download/file[3] http://www.rail-reg.gov.uk/upload/pdf/370.pdf[4] http://www.audit-commission.gov.uk/SiteCollectionDocuments/AuditCommissionReports/NationalStudies/archive_mptarget.pdf[5] http://www.hfmmagazine.com/hfmmagazine_app/jsp/articledisplay.jsp?dcrpath=HFMMAGAZINE/Article/data/02FEB2010/1002HFM_FEA_Biomedical&domain=HFMMAGAZINE

Product and Quality Performance Indicators for Services in Engineering. Peter Smithson IPEM Scope. Vol 11#3 Sept 2002

Benchmarking – The new ‘Soldering Iron’ for Effective Equipment Support. Paul Robbins. IPEM Scope. Vol 11#3 Sept 2002

References…

key performance indicators, a spy in the camp or a useful tool?

Technology

medical engineering

good kpis relevant balanced

good kpis suggestions

good kpis easy

dashboardspresenting

overview definitions

cost of repair tips

relevant action