risk scoring - cardiff picu scoring.pdf · zprism (pediatric risk of mortality) zpim (paediatric...
TRANSCRIPT
![Page 1: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/1.jpg)
Risk scoring
Allan Wardhaugh
![Page 2: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/2.jpg)
A tale of two units
![Page 3: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/3.jpg)
Holby City, Holby
![Page 4: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/4.jpg)
Sacred Heart
![Page 5: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/5.jpg)
PICU mortality last year
Holby City500 admissions15 deaths
Sacred Heart1200 admissions90 deaths
![Page 6: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/6.jpg)
‘Crude’ mortality
Holby City 3%
Sacred Heart 7.5%
![Page 7: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/7.jpg)
What else do we need to know?
How sick were the patients?Did the patients at Sacred Heart undergo more complex procedures?Does Holby City send sick patients away?Does Holby City discharge patients to die somewhere else?What is the mortality in other units?
Case Mix
![Page 8: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/8.jpg)
Standardised Mortality Ratio
Measured mortalityPredicted mortality – risk adjustment tool
SMR = Measured/ Predicted
∴ SMR > 1 performing poorlySMR < 1 performing well
![Page 9: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/9.jpg)
Predicted mortality
Historical dataNational ‘average’
What about case mix?
![Page 10: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/10.jpg)
Risk adjustment
Predicts mortality for a population based on a statistical model derived from a similar populationExamples
APACHE (Acute Physiology and Chronic Health Evaluation )
CRIB (Clinical Risk Index for Babies)
PRISM (Pediatric RISk of Mortality)
PIM (Paediatric Index of Mortality)
![Page 11: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/11.jpg)
Regression statistics
Target variable – ‘dependent variable’Predictors – ‘independent variables’Regression statistics use association between variables to predict one (DV) from another (IV).Simplest form y = b0 + b1(x)where y = predicted value, b0= regression constant, b1= regression coefficient
Multiple regressiony = b0+b1(x1)+b2(x2)+…bn(xn)
![Page 12: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/12.jpg)
PICU
Mortality is target variable
Predictor variables?
![Page 13: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/13.jpg)
Example
1000 patients 100 died900 survivedProbability of death 0.1
Odds of death 0.11Admitted electively
10 of 100 deaths500 of 900 deathsProbability of death 0.02
Odds of death 0.02
![Page 14: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/14.jpg)
Probability and Odds
ppOdds−
=1
Probabilty = 0 – 1
Odds = 0 - ∞
![Page 15: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/15.jpg)
Regression statistics - logistic
For non-quantitative DV (e.g. dead/alive), logistic regression is usedFor each IV, odds are calculated for likelihood of having DVOdds assymetrical
very small number (0 – 1) if event unlikelyvery large if event likely (>1 - ∞)
Rectified by using natural log of odds – called logit – makes it a linear function
![Page 16: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/16.jpg)
Regression statistics - logit
Log odds = b0+b1(x1)+b2(x2)+…bn(xn)
Probability = odds/(1 + odds)Logit = ln odds
logit
logit
e1ep+
=
![Page 17: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/17.jpg)
PRISM
Pediatric Risk of Mortality Score14 physiological variables
Worst measurement in first 24 hoursNow on PRISM III – relies on scores in first 12 or 24 hours
Probability of PICU death= eR/1 + eR
Where R = 0.207 × PRISM – 0.005 × age(mo) – 0.433 × operative status – 4.782
![Page 18: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/18.jpg)
PRISM III
![Page 19: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/19.jpg)
PRISM – example60 month old non-surgical patient
PRISM Score Mortality Risk (%)
3 1.6
6 2.7
9 5
12 8.9
15 15.3
18 25.2
21 38.6
24 46.1
27 68.5
30 80.2
![Page 20: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/20.jpg)
PRISM - disadvantages
Data collection cumbersome (14 variables over a 24 hour period)May diagnose death rather than predict it (40% deaths occur in first 24 hours)Score may not allow comparison between units – patients poorly managed in first 24 hours will develop high PRISM score, so disease severity will appear to be greater
![Page 21: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/21.jpg)
PIM – Paediatric Index of Mortality –initial cohorts
678 consecutive admissions PICU RCHM 1988814 consecutive admissions RCHM 19901412 consecutive admissions 1994–5 RCHM
![Page 22: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/22.jpg)
PIM – identifying variables
Data collected for admission (for most) and first 24 hours34 Physiological Stability Index measurementsMAP, PIP, PEEP, and othersWorst value in first 24 hours for all
![Page 23: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/23.jpg)
PIM – derivation of model
All PRISM data collected plus additional informationUnivariate analysis carried out on all factors to test for association with mortalityFactors not associated (p>0.1) excluded from further analysisLogistic regression analysis used to derive preliminary model.
![Page 24: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/24.jpg)
PIM – testing the model
Learning and Test cohorts1994 – 96 5695 patients in 8 PICUs (Australia, Birmingham)
Enough patients in each unit to include 20 deaths.Learning sample data analysed to calculate regression coefficientsModel then tested on test sample, and examined for goodness of fit.Regression coefficients re-estimated using all 8 units for final model.
![Page 25: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/25.jpg)
PIM - coefficients
![Page 26: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/26.jpg)
![Page 27: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/27.jpg)
PIM – final equations
elogit/(1+elogit)
Logit = (2.357.pupils)+(1.826.specified diagnosis)+(–1.552.elective admission)+(1.342.mechanical ventilation)+(0.021.(SBP–120))+(0.071.Baseex)+(0.415.(100.FiO2/PaO2))–4.873
![Page 28: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/28.jpg)
PIM - recalibration
PICU outcomes change with timeReferral patterns change with timeAttitudes to withdrawing and limiting care may change with time
![Page 29: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/29.jpg)
Calibration findings
Specific diagnosisResp illness O:E 160:212Non-cardiac post-op O:E 48:82
293 coded diagnostic categories examinedIn-hospital cardiac arrest associated with increaedrisk of deathAsthma. Bronchiolitis, croup, obstructive sleep apnoea, DKA associated with reduced risk
New ‘high risk’ and ‘low risk’ categories introducedPost – op subdivided into with or without CBP.IQ <35 omitted (difficult to code reliably)
![Page 30: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/30.jpg)
SMR
Australia and New Zealand SMR 0.84 (0.76–0.92) UK 0.89 (0.77–1.00).
![Page 31: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/31.jpg)
![Page 32: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/32.jpg)
PRISM vs PIM
PRISM predicted 66% more deaths in this sampleScore altered by treatment in the first 24 hoursMay diagnose rather than predict deathPRISM III data requires 96 measured variablesLicense requiredNote that neither are adequate for individual case prediction – apply to populations only
![Page 33: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/33.jpg)
United KingdomPaediatric Intensive Care Outcome Study
UK PICOS
![Page 34: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/34.jpg)
PIM mortality ratio (observed/expected unit deaths) by unit. Generated using UK PICOS recalibration
.5
1
2M
orta
lity
ratio
200 400 600 800 1000Number of admissions
Your unit Other units Control limits
Mortality ratio calculated using the UK PICOS calibration of PIM in the UK.
Upper and lower control limits represent a 99.9% confidence interval around a mortality ratio of 1 based on the UK PICOS overall mortality of 6.2%..
![Page 35: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/35.jpg)
Outcome
PRISM III 24 hour score re-calibrated for UKPerformance of PIM-2 and PRISM III very similarPIM – 2 recommended as model of choice as data easier to collect
![Page 36: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/36.jpg)
DoH/ WAG fundedRun from Universities of Sheffield, Leicester and LeedsFirst annual report March 2003 – February 2004
![Page 37: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/37.jpg)
![Page 38: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/38.jpg)
![Page 39: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/39.jpg)
![Page 40: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/40.jpg)
2009-11 Unadjusted
![Page 41: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/41.jpg)
2009-11 adjusted (PIM-2R)
![Page 42: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/42.jpg)
League tables
Governments like themJournalists like themLocal politicians like themPatients groups like them
Do any of the above understand them?
![Page 43: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/43.jpg)
9 NICUs over 6 yearsCrude and risk adjusted (CRIB score) mortalityHospitals ranked in league tables each year according to W score
W= 100 × (observed - expected deaths)/No of admissions.Mortality lower than expected if W < 0
![Page 44: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/44.jpg)
Results
![Page 45: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/45.jpg)
Conclusions
Hospitals varied annually in their league positionConfidence intervals for W scores overlapped for all hospital every year except year 3‘Overall, hospital 1 did perform significantly better than expected but it is debatable whether this makes it a model hospital since its performance was inconsistent’.
![Page 46: Risk scoring - Cardiff PICU scoring.pdf · zPRISM (Pediatric RISk of Mortality) zPIM (Paediatric Index of Mortality) Regression statistics ... {For each IV, odds are calculated for](https://reader033.vdocuments.net/reader033/viewer/2022051720/5a775c457f8b9a0d558de315/html5/thumbnails/46.jpg)
Summary
PIM/ PIM 2 data easy to collect Useful in comparing unit performanceInterpret with care if number of deaths low (especially <20).Not for use as an individual prediction testImportant to complete as accurately as possiblePICANET randomly check to ensure data qualityLeague tables are unreliable