Monitoring clinical quality through indicators, metrics and dashboardsNational perspectiveDanny Keenan

National Clinical Advisor

Care Quality Commission

2

NHS Performance Framework; Implementation guidance April 2011Technical Guidance for the 2011/12Operating Framework; Jan 2011The NHS Outcomes Framework 2011/12

• Quality metrics and indicators explained: what’s the difference between a metric and an indicator

• Discussion re Clinical Quality

• Augmenting national indicators with local measurements of quality

• How the Care Quality Commission will monitor improvements in quality

• Using clinical dashboards to monitor improvement in quality indicators and metrics

3

Metric or indicator?

A metric is a measure

Patients in primary care have their blood pressure measured

An indicator is a way of using a metric for purposes of comparison: (Latin: indicare = to point out)

4

Metric versus indicator

Unit A has a caesareanSection rate of 15%

A metric An indicator

5

What is clinical quality?

6

Three domains of quality

Safety

Effectiveness

Patient experience

7

Outcomes and indicators for the first framework

Preventing people from dying prematurely

Enhancing the quality of life for people with long-term conditions

Helping people to recover from episodes of ill health or following injury

Ensuring people have a positive experience of care

Treating and caring for people in a safe environment and protecting them from avoidable harm

EFFECTIVNESS

PATIENT EXPERIENCE

SAFETY

Clin

ical qu

ality

The NHS Outcomes Framework 2011/12

8

Assessing quality

Measuring clinical quality:

• Structure- Refers to the inherent characteristics which are associated with higher

quality (e.g):• Procedure volumes• Advanced IT• Nurse staffing ratios• Stroke service

• Processes- Do you comply with evidence-based care guidance?

• Administration of aspirin and B blockers with acute coronary syndrome• Compliance with desirable practices such as frequent determination of

HbA1c in diabetics

• Outcomes- some general ones- some sector or condition specific

9

Example metrics for assessing quality

Inputs(Policies and inputs that ensure quality)

Process(Pathways and best-practices for quality)

Outcome(Ensuring that patients are better-off)

Safety (Ensuring that providers ‘do no harm’)

Clinical effectiveness (Evaluating clinical quality of care)

Patient experience(Ensuring good quality of non-clinical care)

• Health and safety regulations understood

• Ensure staff qualifications

• Systems and structures in place for setting standards and monitoring quality of care

• Systems and structures for monitoring

• Patient advocate in place

• Use of double identifiers in drug prescription process

• Breast cancer screening rates

• Thrombolysis rates

• Confidentiality• Waiting times• Access to

information • Customer

satisfaction training

• Rate of adverse events, serious untoward incidents

• Functional capability post routine surgery

• Survival rates for major surgery

• Patient satisfaction rates with services

• Number of complaints

Structure

10

Implementing the NHS Performance Framework

www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_127007.pdf

11

Augmenting national indicators with local measurements of quality

Preventing people from dying prematurely

Enhancing the quality of life for people with long-term conditions

Helping people to recover from episodes of ill health or following injury

Ensuring people have a positive experience of care

Treating and caring for people in a safe environment and protecting them from avoidable harm

EFFECTIVNESS

PATIENT EXPERIENCE

SAFETY

Clin

ical qu

ality

The NHS Outcomes Framework 2011/12

There are a set of 50 indicators underneath these domains

12

Augmenting national indicators with local measurements of quality

The Commissioning Outcomes Framework

Derived from the NICE Quality Standards, among other things.

There will always be room for local measures:

North West :

• Advancing Quality ---AQuA (Advancing Quality Alliance):- Heart Attack- Hip or knee replacement- Heart failure- Pneumonia- Coronary artery bypass surgery- Next iteration:

Mental health, including dementia COPD Diabetes

13

NHS Performance Guidance-Implementation Guidance. April 2011

Measuring for Quality Improvement launched the development of a menu of Assured Quality Indicators to enable local clinical teams to identify indicators that support their improvement work and allow benchmarking with other clinical teams.

Quality indicators from this Assured Menu will have a number of uses, including in commissioner contracts (particularly the Commissioning for Quality and Innovation (CQUIN) payment framework), publication of Quality Accounts, and information for the public through NHS Choices.

14

Indicators for Quality Improvement

15

How CQC will monitor improvements in quality

The essential standards of quality and safety

16

CQC and registration

CQC regulates under the Essential Standards of Quality and Safety

The essential standards are focused on outcomes - the experiences people have as a result of the care they receive

• Regulation 9; Outcome 4;Care and welfare of people who use services

People experience effective, safe and appropriate care, treatment and support that meets their needs and protects their rights.

• Regulation 10; Outcome 16;Assessing and monitoring the quality of service provision

People benefit from safe, quality care because effective decisions are made and because of the management of risks to people’s health, welfare and safety.

• Regulation 12; Outcome 8;Cleanliness and infection control

People experience care in a clean environment, and are protected from acquiring infections.

17

Quality and Risk Profile (QRP)

18

Quality Risk Profiles (QRP) are at the centre

Bring together information about a care provider so as to estimate risk and prompt front line regulatory activity

Act as a guide and aid CQC’s inspectors

Not a judgement (or a rating, or a league table) in themselves

Build over timeCapable of using both numeric (quantitative) and textual (qualitative) data

Essential tool to support monitoring of compliance

Help inform subsequent judgements and add to the knowledge base about a care provider

Rejects the use of a small number of numeric indicators to make judgements about service quality

19

…but builds on 10 years development

19

The philosophy draws on the work of Klein and Carter in the 1980s

The mathematical approaches have been being developed by CQC and its predecessors since the early 2000s and have substantial academic input

The qualitative analysis techniques allow us to treat textual information consistently draw on best practice in social research techniques

The approach draws on and addresses Bevan and Hood’s reflections of the “measurement frenzy” in public service reform in the 2000s

This approach is at the cutting edge of use of information, but we believe transferable between sectors

20

Updating and sharing of QRPs with providers

20

QRPs are updated regularly (monthly or more often depending upon data feeds)

The QRPs will be shared with Providers as they are updated so that we are all working from the same data

NHS trusts have had access to their QRP since September via a .pdf. This has been made available to commissioners, SHAs, Monitor and DH since October

This summer we will release an online service which will give access to data as live and which will have vastly increased functionality

21

22

How the Care Quality Commission will monitor improvements in quality

23

How CQC will monitor improvements in quality

Our main tool will be using Registration

and……….

Compliance with Registration

Previously we had the Annual Health Check…proved to be a blunt tool

Now that Registration has been in place, with Trusts, for a year compliance is becoming more important

24

Site visits

The aim of site visits is to gather evidence of compliance

They are short, focussed unannounced site visits, rather than set piece inspections that require the provider to spend a lot of time in preparation

Site visits are direct checks of compliance rather than assessing compliance through the assurance systems the organisation has in place.

Therefore site visits may include direct observation of care and spending time with people who use the service, their families and carers, unless not appropriate to do so. We may also talk to managers and staff. Experts by experience will join us on some site visits to help us engage with people who use services.

25

Judgement framework

Regulatory judgement and response

Stage 2: Does the evidence show compliance?

Stage 3: What is the impact on people who use services and the likelihood of this happening? Is there:

No concern

Minor concern

Moderate concern

Major concern

Stage 4: Validation

Regulatory response

Maintain registration - no further action

Improvement actions:

eg improvement letter

Enforcement actions:

Statutory warning notice

Imposition or variation of conditions

Fines

Prosecution

Suspension of registration

Cancellation of registration

Regulatory judgementJudgement of compliance or concerns

Translates minor, moderate or major concerns into regulatory judgement

Takes account of the provider’s capability to improve

Action will be proportionate

Many Trusts have some compliance problems

With enforcement to

ensure compliance

we believe this will d

rive im

provement

26

Using clinical dashboards to monitor improvement in quality indicators and metrics

This will be determined locally, bearing in mind that there will be requirements laid down by the Department (including QIPP local agenda); CQC and Monitor.So a dashboard will have these through it:

•Measures to reassure the Board/Managers that the service they provide is effective, safe and patient responsive•Measures negotiated with local commissioners•Measures suggested by clinicians and for use in revalidation•Measures suggested by patients, including patient outcomes and surveys

27

Using clinical dashboards to monitor improvement in quality indicators and metrics

Pitfalls:

• Avoid that which is easy to measure but not useful

• The adverse effects of any measure which is that the measured item improves at the expense of other important areas

• The burden of measurement

28

Local dashboards

Some overarching themes:• Process:

- Board Issues (Having a culture of quality and safety in an organisation and amongst staff)

- Quality of Records

- Information / HES quality/ NHS number

- Commitment to safety

- Issues relating to staff

• Sickness/stress, staff satisfaction, absence, training, vacancies

- Commitment to Audit and Service Improvement

- Commitment to Research

29

Local dashboards

Some overarching themes:• Outcomes:

- Mortality - Healthcare associated infections- Patients:

• Satisfaction, complaints- Prescribing, administration and reconciliation of medicines - Patient Safety First outcomes- Never events- National audits;

• Cardiac• Cancer

- HES based outcomes

30

Thank you

Danny.Keenan@cqc.org.uk

31

Annex to the NHS Performance Framework; Implementation guidance April 2011Technical Guidance for the 2011/12Operating Framework; Jan 2011http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_123660.pdf

32

Mental Health

www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_127007.pdf