predictive models and data linkage

© Nuffield Trust

Predictive Models and Data Linkage Sharing international experience: Linking disease registry information and predictive modelling to improve quality and efficiency

September 2012

Martin Bardsley Head of Research The Nuffield Trust

© Nuffield Trust

Applications of predictive risk in the UK

• Case finding for people at high risk of admission seen as increasingly important for people with LTCs and complex conditions

• Examples of predicting across health and social care

• Scope to make the most of linked data sets in describing care pathways

• Evaluation and risk adjustment

© Nuffield Trust

Predictive risk and case finding

© Nuffield Trust

© Nuffield Trust

Predictive modelling in UK

• BMJ in paper* in 2002 showed Kaiser Permanente in California seemed to provide higher-quality healthcare than the NHS at a lower cost. Kaiser identify high risk people in their population and manage them intensively to avoid admissions

• • Modelling aims to identify people at risk of high costs in future • Relies on exploiting existing information

+ve: systematic; not costly data collections; fit into existing systems -ve: information collected may not be predictive

• *Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143

© Nuffield Trust

Predictive Models Identify who will be where on next year’s Kaiser Pyramid

© Nuffield Trust

Regression to the mean: Change in average number of emergency bed days

Predictive models try to identify people here

© Nuffield Trust

Extending models beyond healthcare

© Nuffield Trust

Information flows

© Nuffield Trust

Protecting individuals identities

© Nuffield Trust

Looking at and individuals history of care One person’s story

© Nuffield Trust

Model Risk threshold PPV (%) Sensitivity (%) PARR (England) 50 65.3 54.3

70 77.4 17.8

80 84.3 8.1

SPARRA (Scotland) 50 59.4 18.0

70 76.1 3.3

S Care model (Pooled £1K)

50

70

55

60

19

10

Typical accuracy models currently used to predict hospital admission

© Nuffield Trust

Range of case finding models available

SPARRA PARR (++) SPARRA MD Combined Predictive Model PRISM PEONY AHI Risk adjuster LACE ACGs (John Hopkins) MARA (Milliman Advanced Risk

Adjuster) DxCGs (Verisk) Dr Foster Intelligence SCOPE RISC (United Health Group)

Variants on basic admission/readmission predictions: Short term readmissions Social care costs Condition specific tools

© Nuffield Trust

Wider applications of linked data

© Nuffield Trust

© Nuffield Trust

Using the data available

© Nuffield Trust

Testing for gaps in care

© Nuffield Trust

North West e-lab

© Nuffield Trust

Accident and emergency 350,000 records

Outpatients 1,680,000 records

Inpatients 360,000 records

Social care 240,000 records

Community matrons 20,000 records

GPs 60 practices 48.5 million records

Relative size of data sets collected For one WSD area

March 2011

© Nuffield Trust

Data linkage Social & secondary care interface

© Nuffield Trust

Inpatient and Social Care costs per person in final year of life by age band over two lines

£0

£2,000

£4,000

£6,000

£8,000

£10,000

£12,000

40 50 60 70 80 90 100

Age Band

Social care

Hospital IP care

SC+ Hosp

© Nuffield Trust

Number of inpatient admissions (with 95% confidence intervals) per person by age according to type of social care received

Bardsley M, Georghiou T, Chassin L, Lewis G, Steventon A, and Dixon J. Overlap of hospital use and social care in older people in England J Health Serv Res Policy jhsrp.2011.010171; published ahead of print 23 February 2012,

© Nuffield Trust

Describing patterns of social care around cancer diagnosis. Linkage to cancer registry

© Nuffield Trust

What was the average cost of hospital care?

© Nuffield Trust

GP visits around cancer diagnosis

© Nuffield Trust

Risk adjustment and Evaluation a. Prospective Trials b. Retrospective evaluations

© Nuffield Trust

© Nuffield Trust

Using risk scores within a randomised trial

March 2011

Ensuring even mix of patients Analysis by risk subgroup

© Nuffield Trust

Information flows for this analysis

Secondary Uses Service

GP

Community systems

Social care

Local operational systems

Encrypted client-event based

Encrypted client-event based

Encrypted client-event based

Encrypted client-event based

Link to create Combined Model

Nuffield Trust Linked datasets Hospital

Episodes Statistics

Encrypted client-event based

HES-ONS mortality data

Encrypted client-event based

© Nuffield Trust

Distribution of Combined Model risk scores Importance of risk adjustment

General population

Top 0.5%

0.5% - 5%

5% - 20%

20% - 100%

WSD participants

Top 10%

10% - 45%

45% - 85%

85% - 100%

Very high risk High risk Moderate risk Low risk

© Nuffield Trust

Exploiting admin data within an RCT- trends in emergency hospital admissions

Start of trial

Able to chart hospital use before recruitment

© Nuffield Trust

Linked data in RCTS

• Enables larger sample sizes as its relatively cheap information

• Able to generate multiple outcome measures

• Track patient histories before baseline – and inform risk adjustment

• Generate intermediate points

• BUT • Constrained by type of information collected and quality

• May exclude care from some sectors

© Nuffield Trust

Retrospective evaluations The Partnership for Older People Projects (POPPs)

“We recommend expanding the Partnerships for Older People Projects (POPPs) approach to prevention across all local authorities and PCTs.”

•£60m investment by DH with aim to: “shift resources and culture away from institutional and hospital- based crisis care” •146 interventions piloted in 29 sites. •National evaluation of whole programme found £1.20 saving in bed days per £1 spent.

© Nuffield Trust

From the 146 interventions offered under POPP, we selected 8 for an in-depth study of hospital use

Support workers for community matrons Intermediate care service with generic workers Integrated health and social care teams Out-of-hours and daytime response service

+ 4 different short term assessment and signposting services

© Nuffield Trust

Our preferred option for this evaluation: link participants to HES through a trusted third party

March 2011

Collate files and add NHS numbers

Derive HES ID

Collate patient lists

Patient identifiers (e.g. NHS number)

Trial information (e.g. start and end date)

Non-patient identifiable keys (e.g. HES ID, pseudonymised NHS number)

Participating sites Information Centre

Nuffield Trust

© Nuffield Trust

Prevalence of health diagnoses categories in intervention and control groups

© Nuffield Trust

Overcoming regression to the mean using a control group

March 2011

0.0

0.1

0.2

0.3

-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num

ber

of e

mer

genc

y ho

spit

al a

dmis

sion

s pe

r he

ad p

er m

onth

Month

Intervention

Start of intervention

© Nuffield Trust

Overcoming regression to the mean using a control group

March 2011

0.0

0.1

0.2

0.3

-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num

ber

of e

mer

genc

y ho

spit

al a

dmis

sion

s pe

r he

ad p

er m

onth

Month

Intervention

Start of intervention

© Nuffield Trust

Overcoming regression to the mean using a control group

March 2011

0.0

0.1

0.2

0.3

-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num

ber

of e

mer

genc

y ho

spit

al a

dmis

sion

s pe

r he

ad p

er m

onth

Month

Intervention

Start of intervention

© Nuffield Trust

Overcoming regression to the mean using a control group

March 2011

0.0

0.1

0.2

0.3

-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num

ber

of e

mer

genc

y ho

spit

al a

dmis

sion

s pe

r he

ad p

er m

onth

Month

Control Intervention

Start of intervention

© Nuffield Trust

Impact of eight different interventions on hospital use

© Nuffield Trust

Summary

• Predictive modelling practical case finding tool for identifying high risk patients

• Possible to screen large populations using existing data

• Scope to extend linkage over time and across data sets to give a broader view of patients’ journey

• Large data sets can be used in both prospective studies (RCTs) and enable retrospective analyses using matched controls

• Biggest weakness with existing administrative data is the limited level of clinical information – yet greater use of clinical records, audits and registries is possible

© Nuffield Trust

www.nuffieldtrust.org.uk

Sign-up for our newsletter www.nuffieldtrust.org.uk/newsletter

Follow us on Twitter (http://twitter.com/NuffieldTrust)

© Nuffield Trust