predictive models and data linkage
TRANSCRIPT
© Nuffield Trust
Predictive Models and Data Linkage Sharing international experience: Linking disease registry information and predictive modelling to improve quality and efficiency
September 2012
Martin Bardsley Head of Research The Nuffield Trust
© Nuffield Trust
Applications of predictive risk in the UK
• Case finding for people at high risk of admission seen as increasingly important for people with LTCs and complex conditions
• Examples of predicting across health and social care
• Scope to make the most of linked data sets in describing care pathways
• Evaluation and risk adjustment
© Nuffield Trust
Predictive risk and case finding
© Nuffield Trust
© Nuffield Trust
Predictive modelling in UK
• BMJ in paper* in 2002 showed Kaiser Permanente in California seemed to provide higher-quality healthcare than the NHS at a lower cost. Kaiser identify high risk people in their population and manage them intensively to avoid admissions
• • Modelling aims to identify people at risk of high costs in future • Relies on exploiting existing information
+ve: systematic; not costly data collections; fit into existing systems -ve: information collected may not be predictive
• *Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143
© Nuffield Trust
Predictive Models Identify who will be where on next year’s Kaiser Pyramid
© Nuffield Trust
Regression to the mean: Change in average number of emergency bed days
Predictive models try to identify people here
© Nuffield Trust
Extending models beyond healthcare
© Nuffield Trust
Information flows
© Nuffield Trust
Protecting individuals identities
© Nuffield Trust
Looking at and individuals history of care One person’s story
© Nuffield Trust
Model Risk threshold PPV (%) Sensitivity (%) PARR (England) 50 65.3 54.3
70 77.4 17.8
80 84.3 8.1
SPARRA (Scotland) 50 59.4 18.0
70 76.1 3.3
S Care model (Pooled £1K)
50
70
55
60
19
10
Typical accuracy models currently used to predict hospital admission
© Nuffield Trust
Range of case finding models available
SPARRA PARR (++) SPARRA MD Combined Predictive Model PRISM PEONY AHI Risk adjuster LACE ACGs (John Hopkins) MARA (Milliman Advanced Risk
Adjuster) DxCGs (Verisk) Dr Foster Intelligence SCOPE RISC (United Health Group)
Variants on basic admission/readmission predictions: Short term readmissions Social care costs Condition specific tools
© Nuffield Trust
Wider applications of linked data
© Nuffield Trust
© Nuffield Trust
Using the data available
© Nuffield Trust
Testing for gaps in care
© Nuffield Trust
North West e-lab
© Nuffield Trust
Accident and emergency 350,000 records
Outpatients 1,680,000 records
Inpatients 360,000 records
Social care 240,000 records
Community matrons 20,000 records
GPs 60 practices 48.5 million records
Relative size of data sets collected For one WSD area
March 2011
© Nuffield Trust
Data linkage Social & secondary care interface
© Nuffield Trust
Inpatient and Social Care costs per person in final year of life by age band over two lines
£0
£2,000
£4,000
£6,000
£8,000
£10,000
£12,000
40 50 60 70 80 90 100
Age Band
Social care
Hospital IP care
SC+ Hosp
© Nuffield Trust
Number of inpatient admissions (with 95% confidence intervals) per person by age according to type of social care received
Bardsley M, Georghiou T, Chassin L, Lewis G, Steventon A, and Dixon J. Overlap of hospital use and social care in older people in England J Health Serv Res Policy jhsrp.2011.010171; published ahead of print 23 February 2012,
© Nuffield Trust
Describing patterns of social care around cancer diagnosis. Linkage to cancer registry
© Nuffield Trust
What was the average cost of hospital care?
© Nuffield Trust
GP visits around cancer diagnosis
© Nuffield Trust
Risk adjustment and Evaluation a. Prospective Trials b. Retrospective evaluations
© Nuffield Trust
© Nuffield Trust
Using risk scores within a randomised trial
March 2011
Ensuring even mix of patients Analysis by risk subgroup
© Nuffield Trust
Information flows for this analysis
Secondary Uses Service
GP
Community systems
Social care
Local operational systems
Encrypted client-event based
Encrypted client-event based
Encrypted client-event based
Encrypted client-event based
Link to create Combined Model
Nuffield Trust Linked datasets Hospital
Episodes Statistics
Encrypted client-event based
HES-ONS mortality data
Encrypted client-event based
© Nuffield Trust
Distribution of Combined Model risk scores Importance of risk adjustment
General population
Top 0.5%
0.5% - 5%
5% - 20%
20% - 100%
WSD participants
Top 10%
10% - 45%
45% - 85%
85% - 100%
Very high risk High risk Moderate risk Low risk
© Nuffield Trust
Exploiting admin data within an RCT- trends in emergency hospital admissions
Start of trial
Able to chart hospital use before recruitment
© Nuffield Trust
Linked data in RCTS
• Enables larger sample sizes as its relatively cheap information
• Able to generate multiple outcome measures
• Track patient histories before baseline – and inform risk adjustment
• Generate intermediate points
• BUT • Constrained by type of information collected and quality
• May exclude care from some sectors
© Nuffield Trust
Retrospective evaluations The Partnership for Older People Projects (POPPs)
“We recommend expanding the Partnerships for Older People Projects (POPPs) approach to prevention across all local authorities and PCTs.”
•£60m investment by DH with aim to: “shift resources and culture away from institutional and hospital- based crisis care” •146 interventions piloted in 29 sites. •National evaluation of whole programme found £1.20 saving in bed days per £1 spent.
© Nuffield Trust
From the 146 interventions offered under POPP, we selected 8 for an in-depth study of hospital use
Support workers for community matrons Intermediate care service with generic workers Integrated health and social care teams Out-of-hours and daytime response service
+ 4 different short term assessment and signposting services
© Nuffield Trust
Our preferred option for this evaluation: link participants to HES through a trusted third party
March 2011
Collate files and add NHS numbers
Derive HES ID
Collate patient lists
Patient identifiers (e.g. NHS number)
Trial information (e.g. start and end date)
Non-patient identifiable keys (e.g. HES ID, pseudonymised NHS number)
Participating sites Information Centre
Nuffield Trust
© Nuffield Trust
Prevalence of health diagnoses categories in intervention and control groups
© Nuffield Trust
Overcoming regression to the mean using a control group
March 2011
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num
ber
of e
mer
genc
y ho
spit
al a
dmis
sion
s pe
r he
ad p
er m
onth
Month
Intervention
Start of intervention
© Nuffield Trust
Overcoming regression to the mean using a control group
March 2011
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num
ber
of e
mer
genc
y ho
spit
al a
dmis
sion
s pe
r he
ad p
er m
onth
Month
Intervention
Start of intervention
© Nuffield Trust
Overcoming regression to the mean using a control group
March 2011
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num
ber
of e
mer
genc
y ho
spit
al a
dmis
sion
s pe
r he
ad p
er m
onth
Month
Intervention
Start of intervention
© Nuffield Trust
Overcoming regression to the mean using a control group
March 2011
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Num
ber
of e
mer
genc
y ho
spit
al a
dmis
sion
s pe
r he
ad p
er m
onth
Month
Control Intervention
Start of intervention
© Nuffield Trust
Impact of eight different interventions on hospital use
© Nuffield Trust
Summary
• Predictive modelling practical case finding tool for identifying high risk patients
• Possible to screen large populations using existing data
• Scope to extend linkage over time and across data sets to give a broader view of patients’ journey
• Large data sets can be used in both prospective studies (RCTs) and enable retrospective analyses using matched controls
• Biggest weakness with existing administrative data is the limited level of clinical information – yet greater use of clinical records, audits and registries is possible
© Nuffield Trust
www.nuffieldtrust.org.uk
Sign-up for our newsletter www.nuffieldtrust.org.uk/newsletter
Follow us on Twitter (http://twitter.com/NuffieldTrust)
© Nuffield Trust