external validation of prognostic model of tbi
TRANSCRIPT
External Validation of a Prognostic Model to Predict Mortality After Traumatic Brain Injury
Dhaval Shukla*, Akhil Deepika*, GS Umamaheshwar Rao#, DK Subbukrishna@
Departments of Neurosurgery*, Neuroanesthesiology#, and Biostatistics @
NIMHANS, Bangalore
Prediction Models
• Statistical models that combine two or more items of patient data to predict outcome
• Two requirements– clinically valid– methodologically valid
• More reliable than what doctors can foretell• Influence patient management
Hierarchy of Prediction Models
• Univariate Analysis
• Multivariate Analysis
• Logistic Regression Analysis
• Discriminant Analysis
• Web Based Calculator
Clinical Predictors
• GCS
• Motor Response
• Pupillary Reaction
• Ocular Movements
• Blood Pressure
CT Scan Predictors
• Midline Shift
• Cisterns
• Ventricles
• Hematomas
• Petechial Hemorrhages
Biochemical Predictors
• Oxygen
• Hemoglobin
• Glucose
102 Prediction Models of TBI
• Small Sample Size
• Logistic Regression
• 93% High Income Countries
• 11% External Validation
• 19% User-friendly
Perel, et al. BMC Medical Informatics and Decision Making 2006
External Validation of CRASH Prediction Model from IMPACT Dataset
Perel, et al. BMJ 2008
External Validation
King: Tell me about my futureSoothsayer 1: Your all relatives will DIE in front
of your eyesKing punishes him.
Soothsayer 2: You will LIVE longestKing rewards him.
Its only human to cross check if someone predicts bad about you
External Validation
• The performance of a model on a different population
(‘generalizability’ or ‘transportability’)
• CRASH model not validated in middle/ low income country
BMJ 2008;336:425
• Review of clinical and CT Scan data of consecutively admitted TBI patients in ICU over 6 months
DATA COLLECTION
• Univariable Logistic Regression Analyses (LRA)
• Multivariate LRAMODEL
CONSTRUCTI ON
• Discrimination• Calibration
MODEL PERFORMANCE
• Bootstrap MethodVALIDATION
Indicates how closely predicted outcomes match observed outcomes
Resample from the sample data at hand for approximating sampling distribution of a statistic and bias correction
Describes how well a model distinguishes between those who die from those who survive0.90-1 is Excellent
Demographics
• Total no of patients: 150
• Male : Female :: 5.5 : 1
• Age range: 1 to 85
• Mean ICU stay: 8.3±7.2 days
• Mortality: 15.3%
• Time to death: 7.52±4.56 days
Variable CRASH OR (95%CI)
NIMHANS OR (95%CI)
p Value
Age 1.46 (1.39 to 1.54) 0.99 (0.95 to 1.04)
GCS 1.27 (1.24 to 1.31) 2.14 (1.27 to 3.60) 0.004
Pupil Reaction Both One None
11.45 (1.14 to 1.86)3.12 (2.46 to 3.97)
11.30 (0.3 to 43.7)1.23 (0.4 to 32.66)
Extracranial Injury 1.08 (0.91 to 1.28)
CT Scan
Petechial Hemorrhages 1.26 (1.07 to 1.47) 0.81 (0.16 to 4.00)
Obliteration of 3rd Ventricle/ Basal Cisterns
1.99 (1.69 to 2.35) 7.32 (1.27 to 42.14) 0.026
SAH 1.33 (1.14 to 1.55) 0.98 (0.23 to 4.17)
Midline Shift 1.78 (1.44 to 2.21) 0.42 (0.06 to 2.63)
Non Evacuated Hematoma
1.48 (1.24 to 1.76) 0.70 (0.00 to 1.70)
Complications in ICU 0.04 (0.00 to 0.29) 0.001
Model Construction
• CRASH Variables • 3 variables from univariate analysis
– (Pre ICU GCS, Intubation, Complication )
Result• Pre ICU GCS (P_GCS)• Obliteration of 3rd ventricle/ Basal Cistern (OB)• Complication during stay (CD)
DiscriminationVariables Adjusted OR 95% Confidence Interval
Lower Upper
OB 3.565 1.069 11.882
P_GCS 1.819 1.242 2.662
CD 0.053 0.011 0.262
Performance of Model
Area : 0.925, 95% CI : 0.877 – 0.973
Calibration
Calibration: Chi2 : 4.796, Significance: 0.685
Internal Validation
Predi cted group * OUTCOME Crosstabul at i on
13 3 1672. 2% 4. 0% 17. 2%
5 72 7727. 8% 96. 0% 82. 8%
18 75 93100. 0% 100. 0% 100. 0%
Count% wit hin OUTCOMECount% wit hin OUTCOMECount% wit hin OUTCOME
DEAD
SURVI VED
Predict edgroup
Tot al
DEAD SURVI VEDOUTCOME
Tot al
Overall Accuracy 91.4%
Explanation
• CRASH model never validated for developing countries
• Only ICU patients were sampled• Many patients underwent surgery• Pre ICU GCS
Conclusions
• Prediction models based on large population studies may not be valid for a selected group of patients
• Each intensive care should have their own prediction models, which should be revised when services improve
Don't ever prophesy;
for if you prophesy right nobody will remember you,
and if you prophesy wrong, nobody will forgive you
- Josh Billings