an analysis of rapid response team calling algorithms for ......an analysis of rapid response team...

An Analysis of Rapid Response Team Calling Algorithms for ClinicalDeficit Evaluation

by

David Samuel Chartash

A thesis submitted in conformity with the requirementsfor the degree of Master of Health Science

Graduate Department of Institute of Biomaterials and Biomedical EngineeringUniversity of Toronto

© Copyright 2013 by David Samuel Chartash

Abstract

An Analysis of Rapid Response Team Calling Algorithms for Clinical Deficit Evaluation

David Samuel Chartash

Master of Health Science

Graduate Department of Institute of Biomaterials and Biomedical Engineering

University of Toronto

2013

This research examines the activation of the Rapid Response Team (RRT) through the Early Warning

Score (EWS) model as a system of predicting adverse event outcomes. Modeling the input parameters

of this system concluded that although conventional parameters associated with EWSs were predictive,

the most predictive clinical and laboratory parameters are those of hematological and nephritic function,

related to the model of multi-organ system decompensation. Upon examining different EWSs, the Modified

Early Warning Score exhibited superior operating characteristics, however, it was not statistically different

than other common EWSs from literature. Accounting for temporal features of the dataset shows that

the International Normalized Ratio is the most predictive parameter, however, the hazard model exhibits

poor discriminative ability. Therefore, clinically, parameters outside the EWS models are predictive of the

outcomes in question, and their incorporation into future policy would serve to better inform the prevention

of adverse events.

ii

Dedication

Dedicated to Faro Chartash, 1999-2013.

iii

Acknowledgements

Prior to any of those professors who have shaped my graduate career, I would like to thank the professors

who have contributed to my desire to pursue research and post-graduate education: Drs. Charles McKenzie,

Samir Gupta, James Lacefield and Krishna Nayak.

At the University of Toronto, I would like to thank my supervisor, Dr. Joseph Cafazzo for his support

throughout my thesis. In addition, Peter Picton for his invaluable editing, advice, and support at University

Health Network and within the Centre. I would also like to thank Dr. Nicholas Mitsakakis at the Toronto

Health Economics and Technology Assessment Collaborative, for his assistance with the thought process

behind the statistical analysis.

Of my committee, Dr. Stephen Lapinsky for his introduction to the project, and support in managing the

realities of literature with a project of appropriate scope and content. Dr. Brian Cuthbertson, for his support

in detailing the complexities of algorithm analysis as performed medical literature. Drs. John Granton and

Damon Scales, for all the time put into participating in the committees and examinations for this thesis.

At Toronto East General, Adrian Harrington, Andrea Wang and Marilyn Lee, without you, this project

would not have been possible. Drs. Marcus Kargel, and Pieter Jugovic also deserve mention as our collab-

orating physician partners. Without our partners at Toronto East General, this project would never have

been able to collect and appropriately analyze the data necessary to produce any content, for that I am

immeasurably grateful.

To the National Sciences and Engineering Research Council Strategic Research Network “Healthcare

Support through Information Technology Enhancements”, and both the philosophical support from their

meetings, as well as financial support throughout my degree.

The group and resources supporting PHYSIONET at MIT also deserve mention, particularly Ken Pierce,

for his assistance in accessing the MIMIC II database, as well as Drs. Peter Szolovits and Caleb Hug for

their advice and masters thesis content respectively.

To the resources and support staff at Indiana University who maintain the Mason large memory computer

cluster, for the use of their tools to preserve local computing resources during the final stages of data analysis.

iv

Contents

1 Introduction 1

1.1 The Rapid Response Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 The Deployment of Rapid Response Teams . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.2 Consensus on the use of the Rapid Response Team . . . . . . . . . . . . . . . . . . . . 3

1.2 Patient Care and Safety Compromising Events . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.1 The Adverse Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2.2 The Patient Deterioration or Decompensation Event . . . . . . . . . . . . . . . . . . . 5

1.2.3 The Failure-to-Rescue Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2.4 Contextual Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3 The Rapid Response System: Putting the Pieces Together . . . . . . . . . . . . . . . . . . . . 8

1.3.1 The Afferent Limb: Early Warning Scores . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.2 The Efferent Limb: Rapid Response Teams . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Analytic Purpose 14

2.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.1.2 Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2.1 Summary of Input Data and Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2.2 Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Characteristics of Data 17

3.1 Patient Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 Administrative Data: Critical Care Secretariat . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3 Clinical Data: Toronto East General Electronic Health Record Query . . . . . . . . . . . . . 21

3.4 Pre-Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.4.1 R Libraries Required . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.4.2 Step I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.4.3 Step II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.4.4 Step III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.4.5 Step IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.4.6 Step V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.4.7 Step VI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.4.8 Step VII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.4.9 Step VIII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

v

4 Input Analysis 24

4.1 Introduction and Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.2 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.2.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3 Random Forest Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.4 Empirical Copula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.5 Summary of Results and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5 System Analysis 38


5.2 Receiver Operating Characteristic Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.2.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.3 Entropy Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.4 Hosmer Lemeshow Goodness of Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44


6 Output Analysis 50


6.2 Kaplan-Meir Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6.2.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6.3 Cox-Proportional Hazards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6.3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

6.4 Gray’s Method of Cumulative Incidence of Competing Risk . . . . . . . . . . . . . . . . . . . 56

6.4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

6.5 General Discrimination Indices for Censored Data . . . . . . . . . . . . . . . . . . . . . . . . 61

6.5.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

6.5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61


7 Conclusions 66

7.1 Summary of Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

7.1.1 Input Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

7.1.2 System Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

7.1.3 Output Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.1.4 Clinical Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

vi

7.2 Limitations and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

A Cerner Command Language Queries 69

A.1 Blood Pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

A.2 Laboratory Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

A.3 Vital Signs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

A.4 Age on Admission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

B Figures 73

Bibliography 155

vii

List of Tables

3.1 Laboratory and Biomaker Variables Present in Query from TEGH Clinical Information System

and Capable of Passthrough for Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.1 Expert Opinion as to Directionality of Selected Parameters . . . . . . . . . . . . . . . . . . . 25

5.1 Positive and Negative Test Characteristic Descriptions for Receiver Operating Characteristic

Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.8 Distribution of Early Warning Scores Across Encounters . . . . . . . . . . . . . . . . . . . . . 42

6.11 Table of Hosmer Lemeshow Goodness of Fit Tests for Receiver Operating Curve Parameters

Sorted by Chi-Squared Test Statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

B.1 Selected Critical Care Secretariat Reporting Database Field Headers . . . . . . . . . . . . . . 74

B.2 Data Description of Critical Care Secretariat Database . . . . . . . . . . . . . . . . . . . . . . 75

B.3 Description of Clinical and Laboratory Variables Present in Query from TEGH Clinical In-

formation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

B.4 Description of Numeric Clinical and Laboratory Variable Values Present in Query from TEGH

Clinical Information System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

B.6 Encounter Data Characteristics: Variables Selected by Number of Encounters Threshold . . . 108

B.7 Estimators of Entropy for Early Warning Scores Values, Sorted by Sample Entropy . . . . . . 108

B.8 Summary of Discrimination Indices Per Outcome . . . . . . . . . . . . . . . . . . . . . . . . . 108

B.9 Table of Area Under the Curve Summary Across Receiver Operating Curves Sorted by Area

Under the Curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

B.10 Table of Logistic Regression LASSO Optimization for Minimum Cross-Validation Error per

Encounter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

B.11 Table of Logistic Regression LASSO Optimization for Minimum Cross-Validation Error per

Encounter, Accounting for Directionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

B.12 Table of Stepwise Optimized Logistic Regression Model Parameters Accounting for Direction-

ality for Outcome: CodeBlue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137


ality for Outcome: CardiacArrest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

B.14 Table of Base Logistic Regression Model Parameters Accounting for Directionality for Out-

come: CardiacArrest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139


ality for Outcome: Death . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139


ality for Outcome: Composite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

viii


come: Composite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141


come: CodeBlue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142


ality for Outcome: ICUTransfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143


come: ICUTransfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144


come: Death . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

B.22 Table of Stepwise Optimized Logistic Regression Model Parameters for Outcome: CardiacArrest146

B.23 Table of Base Logistic Regression Model Parameters for Outcome: CardiacArrest . . . . . . . 147

B.24 Table of Stepwise Optimized Logistic Regression Model Parameters for Outcome: CodeBlue . 148

B.25 Table of Base Logistic Regression Model Parameters for Outcome: CodeBlue . . . . . . . . . 149

B.26 Table of Base Logistic Regression Model Parameters for Outcome: Death . . . . . . . . . . . 150

B.27 Table of Stepwise Optimized Logistic Regression Model Parameters for Outcome: Composite 151

B.28 Table of Base Logistic Regression Model Parameters for Outcome: Composite . . . . . . . . . 152

B.29 Table of Stepwise Optimized Logistic Regression Model Parameters for Outcome: ICUTransfer 153

B.30 Table of Base Logistic Regression Model Parameters for Outcome: ICUTransfer . . . . . . . . 154

B.31 Table of Stepwise Optimized Logistic Regression Model Parameters for Outcome: Death . . . 155

ix

List of Figures

1.1 Root Cause Analysis of Medical Errors Addressed by the Rapid Response Team [1] . . . . . . 11

B.1 Unified Modeling Language Representation of Rapid Response System (Activity Diagram) . . 73

B.2 Parameters Selected from Predictive Health Scores and Other Literature . . . . . . . . . . . . 109

B.3 Modified Early Warning Scores (MEWS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

B.4 VitalPAC™Early Warning Score (ViEWS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

B.5 National Early Warning Score (NEWS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

B.6 Cuthbertson Discriminant Functions (CDF) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

B.7 Cardiac Arrest Risk Triage Scoreing Criteria (CART) . . . . . . . . . . . . . . . . . . . . . . 111

B.8 Ministry of Health and Long Term Care Calling Criteria (MOH) . . . . . . . . . . . . . . . . 112

B.9 Parameters with Amount of Data within First Quantile . . . . . . . . . . . . . . . . . . . . . 113

B.10 Parameters with Amount of Data within Second Quantile . . . . . . . . . . . . . . . . . . . . 114

B.11 Parameters with Amount of Data within Third Quantile . . . . . . . . . . . . . . . . . . . . . 115

B.12 Parameters with Amount of Data within Fourth Quantile . . . . . . . . . . . . . . . . . . . . 116

B.13 Cleveland Dot Plots of Entropy Estimators Grouped by Early Warning Score . . . . . . . . . 117

B.14 Cleveland Dot Plots of Entropy Estimators Grouped by Early Warning Score Value Type . . 118

B.15 Forest Plots of Cox Proportional Hazard Results: Hazard Ratios . . . . . . . . . . . . . . . . 119

B.16 Cleveland Dot Plot of Cox Proportional Hazard Results: p-Values . . . . . . . . . . . . . . . 120

B.17 Cleveland Dot Plot of Cumulative Incidence from Comparative Risk: p-Values . . . . . . . . 121

B.18 Receiver Operating Curves of CardiacArrest Prediction by Time to Event Thresholds for Early

Warning Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

B.19 Receiver Operating Curves of CardiacArrest Prediction by Early Warning Score for Time to

Event Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

B.20 Receiver Operating Curves of CodeBlue Prediction by Time to Event Thresholds for Early

Warning Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

B.21 Receiver Operating Curves of CodeBlue Prediction by Early Warning Score for Time to Event

Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

B.22 Receiver Operating Curves of Composite Prediction by Time to Event Thresholds for Early

Warning Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

B.23 Receiver Operating Curves of Composite Prediction by Early Warning Score for Time to Event

Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

B.24 Receiver Operating Curves of Death Prediction by Time to Event Thresholds for Early Warn-

ing Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

B.25 Receiver Operating Curves of Death Prediction by Early Warning Score for Time to Event

Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

x

B.26 Receiver Operating Curves of ICUTransfer Prediction by Time to Event Thresholds for Early

Warning Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

B.27 Receiver Operating Curves of ICUTransfer Prediction by Early Warning Score for Time to

Event Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

B.28 Dependogram of Five Most Predictive Variables for Logistic Regression . . . . . . . . . . . . 156

B.29 Dependogram of Five Most Predictive Variables for Logistic Regression, Accounting for Di-

rectionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

B.30 Dependogram of Five Most Predictive Variables for Random Forest Classification . . . . . . . 158

B.31 Dependogram of Five Most Predictive Variables for Random Forest Classification, Accounting

for Directionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

B.32 Cleveland Dot Plot of p-Values for Random Forest Classification . . . . . . . . . . . . . . . . 160

B.33 Cleveland Dot Plot of p-Values for Random Forest Classification, Accounting for Directionality161

B.34 Cleveland Dot Plot of p-Values for Logistic Regression, Accounting for Directionality . . . . . 162

B.35 Cleveland Dot Plot of p-Values for Stepwise Optimized Logistic Regression, Accounting for

Directionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

B.36 Cleveland Dot Plot of p-Values for Logistic Regression . . . . . . . . . . . . . . . . . . . . . . 164

B.37 Cleveland Dot Plot of p-Values for Stepwise Optimized Logistic Regression . . . . . . . . . . 165

xi

List of Abbreviations

RT Respiratory Therapist

MD Medical Doctor

RN Registered Nurse

PT Physiotherapist

CCRT Critical Care Response Team

MET Medical Emergency Team

RRT Rapid Response Team

CCO Critical Care Outreach Team

RRS Rapid Response System

ICU Intensive Care Unit

FTR Failure-to-Rescue

EWS Early Warning Score

PCSCE Patient Care and Safety Compromising Events

ALF Afferent Limb Failure

AHRQ Agency for Healthcare Research and Quality

JC The Joint Commission

NPSG National Patient Safety Goals of the Joint Commission

NHS National Health Service

NICE National Institute for Health and Clinical Excellence

JAMA Journal of the American Medical Association

NLM National Library of Medicine

IOM Institute of Medicine

MEDLINE National Library of Medicine Journal Citation Database

EMBASE Elsevier Indexed Search Database including MEDLINE

CINAHL Cumulative Index to Nursing and Allied Health Bibliographic Database

PsycINFO American Psychological Association Abstracting and Indexing Bibliographic Database

HMIC Health Management Information Consortium Bibliographic Database

EED National Health Service Economic Evaluation Database

HTA National Health Service Health Technology Assessment Database

OMNI Organizing Medical Networked Information Database

TRIP Turning Research Into Practice Evidence Based Medicine Database

CCIS Critical Care Information System (Ontario)

TEGH Toronto East General Hospital

APACHE Acute Physiology and Chronic Health Evaluation score

SAPS Simplified Acute Physiology Score

MPM Mortality Prediction Model score

xii

MEWS Modified Early Warning Score

NEWS National Early Warning Score

ViEWS VitalPAC Early Warning Score

CART Cardiac Arrest Risk Triage score

CDF Cuthbertson Discriminant Functions

AVPU Consciousness Scale: Alert, Reacting to Voice, Reacting to Pain and Unresponsive

PSI Patient Safety Indicators

ICD International Classification of Diseases

ROC Receiver Operator Characteristic curves

AUC Area Under the Curve

CDS Clinical Decision Support

EMR Electronic Medical Record

EHR Electronic Health Record

CIS Clinical Information System

CPH Cox-Proportional Hazards

HR Hazard Ratio

dT Threshold Time-to-Event Threshold for Failure-to-Rescue Analysis

OOB Out of Bag

MSE Mean Squared Error

GLM Generalized Linear Models

AIC Akaike’s Information Criterion

LASSO Least Absolute Shrinkage and Selection Operator

GFR Glomerular Filtration Rate

xiii

Chapter 1

Introduction

1.1 The Rapid Response Team

Critical Care Response Team (CCRT), Medical Emergency Team (MET), Critical Care Outreach Team

(CCOT), and Rapid Response Team (RRT); these are many names are for the same unit, which is described

as an intervention intended to generally mitigate adverse events [2] and improve patient safety [3, 4].

Winters et al [4] describes these teams as operating to solve:

“The failure of our current system to adequately monitor patients in the general ward, recognize

the signs and symptoms of deterioration, rescue deteriorating patients and deliver optimal care

rapidly through escalation and triage.” [4]

1.1.1 The Deployment of Rapid Response Teams

Contextually, the Rapid Response Team is deployed as a component of the Rapid Response System;

a mechanism to “identify the hospitalized patient in crisis” [5]. The consensus definition [6] of the Rapid

Response System is described in Figure B.1, which illustrates the concept of a ‘track and trigger’ system.

This system takes clinical signals from the patient and processes them through an event detection scheme

(tracking). Following the detection of an event, the response is activated, and specialized resources are

deployed (trigger). The track and trigger systems are alternatively referred to as the afferent and efferent

ends of the Rapid Response System respectively. The Rapid Response Team serves as the intervention

component, acting to prevent decompensation when prompted by the trigger system.

Commentary in 2006 in The Journal of the American Medical Association (JAMA) by Winters et al [7]

suggests that Rapid Response Teams are evidence of a negative information cascade; RRTs were prioritized

and lauded as patient safety tools yet were associated with uncertain evidence as to their effectiveness. In

support of this commentary, Esmonde et al [8] conducted a systematic review of studies published from 1996

to 2004 via MEDLINE, EMBASE, CINAHL, PsycINFO, HMIC (Health Management Information Consor-

tium) National Research Register, Cochrane Database of Systematic Reviews, Cochrane Library Central

Register of Controlled Trials, Database of Abstracts of Reviews of Effectiveness (via Cochrane Library),

NHS EED (via Cochrane library), NHS HTA (via Cochrane Library), Citation indexes (Science and Social

Sciences), OMNI, TRIP Database. Of the outcomes studied in this review, mortality, cardiac arrest rate

and admission rates to critical care were defined to be pertinent. Mortality was found to be significantly

reduced in seven papers, however, there was no ubiquitous demonstration of reduced mortality and most

papers described no effect on mortality by the service. Cardiac arrest rate reductions were demonstrated

in four studies, however, there was no significant consistent improvement. Admission rates to critical care

1

Chapter 1. Introduction 2

were demonstrably reduced in five studies, however, again there was no significant consistent across the

board improvement. Numerical meta-analysis was not conducted in this paper, therefore the pooled risk

ratio change for each outcome is not detailed. All of the studies identified to improve outcomes were of poor

quality except one, resulting in poor robustness of evidence identified by the systematic review.

Since 2006, drawing on new publications, multiple systematic reviews of the subject have changed the

body of evidence. The first of these, Ranji et al [9] searched MEDLINE, CINAHL and BIOSIS through 2006

as well as the abstract lists from the 2004 and 2005 American Thoracic Society and Society of Critical Care

Medicine annual meetings. The pertinent outcomes identified by this study were cardiopulmonary arrest,

unscheduled ICU admission, and inpatient mortality. Inpatient mortality was reported as having an effect

in nine studies, additionally nine reported positively an effect on cardiopulmonary arrests and six studies

reported an effect on unscheduled ICU admission. From these trials, inpatient mortality was improved via

a risk ratio of 0.82 and a 95% confidence interval from 0.74-0.91. Cardiopulmonary events observed a risk

ratio of 0.73 and 95% confidence interval from 0.65-0.83. Unscheduled ICU admissions observed a risk ratio

of 1.08 and a 95% confidence interval of 0.96-1.22. Overall, the studies identified were described as being of

poor quality, resulting in the conclusion that Rapid Response Systems were not conclusively impactful.

McGaughey et al [10] searched the EPOC Specialized Register, the Cochrane Central Register of Con-

trolled Trials (CENTRAL) and other Cochrane databases from the Cochrane Library, MEDLINE, EMBASE,

CINAHL, First Search and CAB Health, in addition to reference lists of articles, abstracts known to the

authors. This search resulted in two cluster randomized trials being included in the review. In one trial, a

composite outcome of hospital mortality, unanticipated ICU admission, length of hospital stay and adverse

events following outreach services resulted in no statistically significant difference between control and out-

reach activation, with a p-value of 0.64, and adjusted odds ratio of 0.98 and a 95% confidence interval of

0.83-1.16. The second study only found that in-hospital mortality was reduced, with an adjusted odds ratio

of 0.52, 95% confidence interval 0.32-0.85 compared to control. This review concluded with the statement

that:

“The evidence from this review highlights the diversity and poor methodological quality of most

studies investigating outreach. The results of the two included studies showed either no evidence

of the effectiveness of outreach or a reduction in overall mortality in patients receiving outreach.

The lack of evidence on outreach requires further multi-site RCT’s to determine potential effec-

tiveness.” [10]

Chan et al [11] conducted a systematic review of studies published from January 1st 1950 to November

31st 2008 via specific medical databases: PubMed, EMBASE, Web of Knowledge, and CINAHL. Additionally,

“all evidence-based medicine reviews” were consulted, an undescribed block of databases including Cochrane

Review Databases. Of the two outcomes intended to be studied in this project, the two pertinent study

outcomes analyzed by this systematic review were cardiopulmonary arrest and pooled total mortality. Of

the 18 studies, however, 16 reported on cardiopulmonary arrest as an outcome, defined as both cardiac

arrest and pulmonary arrest or either one in isolation depending on the study. The reduction in rates of

the cardiopulmonary arrest was determined to be 33.8%, with a relative risk of 0.66, on a 95% confidence

interval of 0.54-0.80. There was no association with lower pooled hospital mortality, given a relative risk of

0.96, on a 95% confidence interval of 0.84-1.09.

In summary, evidence shows that:

• Prospective studies on Rapid Response Teams are generally of poor [12] quality.

• Slight improvements to cardiac arrest and inpatient mortality are observed, although the quantity and

consistency of those findings is questionable.


Additionally, the authors of the previously mentioned 2006 commentary are evidence of the change in mental-

ity and literature towards accepting the rapid response team as an overall positive intervention. In Winters et

al [4], they describe rapid response teams as being capable of reducing rates of cardiovascular arrest outside

of the ICU, as well as in-hospital mortality.

1.1.2 Consensus on the use of the Rapid Response Team

Rapid Response Teams are associated with evidence based guidelines and as such have resulted in imple-

mentable standards of care for hospitals around the world. In building a consensus on the use of vital signs

monitoring and the general afferent arm of the Rapid Response System, DeVita et al [5] states that:

1. Vital sign aberrations predict risk.

2. Monitoring patients more effectively may improve outcome, although some risk is random.

3. The workload implications of monitoring on the clinical workforce have not been explored,

but are amenable to study and should be investigated.

4. The characteristics of an ideal monitoring system are identifiable, and it is possible to

categorize monitoring modalities. It may also be possible to describe monitoring levels.

In addition to identifying these features of the Rapid Response System, DeVita et al notes that in building

this consensus, there was a significant inclusion of patient safety components, which follow. Identifiable goals

were prefaced with the statement:

“Given the cost of technology and the desire to improve patient safety [...]” [5]

In addition, this conference selected participants for inclusion based on:

“Expertise in clinical healthcare practice or patient safety.” [5]

Alongside this consensus, the efferent arm of the Rapid Response System was identified as a component of

patient safety.

In the United States, Jones et al [13] note that Rapid Response Teams attempt to meet a goal described

in The Joint Commission’s 2009 National Patient Safety Goal sixteen: Improve recognition and response

to changes in a patient’s condition [14]. Since 2009, The Joint Commission’s goals have changed, and

now, in 2013, Goal fifteen is described as to: The hospital identifies safety risks inherent in its patient

population [15]. This change has been noted by The Joint Commission as being due to recommendations

that the 2009 National Patient Safety Goals were too prescriptive and detailed [16].

In addition to The Joint Commission’s statement of goals, the National Institute for Health and Clinical

Excellence (NICE) in the United Kingdom describes the management of decompensating patients in its

fiftieth clinical guidance Acutely Ill Patients in Hospital: Recognition of and Response to Acute Illness in

Adults in Hospital [17]. This guideline identifies the use of a track and trigger system such as a rapid response

team [18] to monitor adult patients in acute hospital settings. These track and trigger systems are described

as:

“Multi-parameter or aggregate weighted scoring systems, which allow a graded response. These

scoring systems should: define the parameters measured and frequency of observations [as well

as] include a clear and explicit statement of the parameters, cut-off points or scores that should

trigger a response.” [17]

The parameters described in the guidance as necessary are: heart rate, respiratory rate, systolic blood

pressure, level of consciousness, oxygen saturation and temperature. Optional parameters are: hourly urine


output, biochemical analysis (lactate, blood glucose, base deficit, arterial pH) and pain assessment. Within

this track and trigger system, the use of a response strategy to deal with patients identified by physiological

monitoring is described as having a:

“Team with critical care competencies and diagnostic skills. The team should include a medical

practitioner skilled in the assessment of the critically ill patient, who possesses advanced airway

management and resuscitation skills. There should be an immediate response.” [17]

This team would also be responsible for considering whether admission to a critical care area is necessary.

Such a described “clinical outreach team” matches descriptions of a Rapid Response Team, in DeVita et

al [5, 6].

In addition to both the United States and United Kingdom, Australia has also produced guidance in

the form of a document entitled National Consensus Statement: Essential Elements for Recognising and

Responding to Clinical Deterioration [19]. This document notes that an essential element of the clinical

process is the Rapid Response System, which operates to escalate care or provide emergency assistance/advice

via Medical Emergency Team. It is specifically mentioned that the Rapid Response System should be part

of an organization’s escalation protocol [19]. This escalation protocol specifies the level of physiological

abnormality, and options for graded escalation, alongside protocols for the transition of care.

Canada, as of writing, has yet to publish a national strategy for the management of clinical deterioration.

Ontario’s response in its critical care strategy [20] and the implementation of Critical Care Rapid Response

Teams [21] is a step in that direction, however concrete results on the subject have yet to be compiled. The

Rapid Response Team calling criteria developed by the Ontario Critical Care Secretariat can be found in

Figure B.8. Taking into account the culture of medical administration taught in academic circles, as well

as the health care business environment of the United States, The Joint Commission’s changing tack from

2009 to 2013 is of importance, particularly given the provision of care according to the rights of medical

administration [22], and the tracking of individual care providers’ performance. As such, it is no wonder

that organizational change intended to preserve business function by the individual has given rise to broad

spectrum safety based risk assessment in the place of a structured deep dive into specific root cause elements

of patient safety. In both the United Kingdom and Australia the uptake of Rapid Response Systems in a

more structured, evidenced and stepwise fashion is consistent with the timeline in which literature developing

quantitatively structured rapid response systems have been developed [23, 24, 25, 26, 27, 28, 29]. In addition

to the research conducted in those countries, their systems of medical care are regulated and standardized to

an extent not found within the fractured multi-payer/multi-party American medical system. With its single-

payer health system, Canada stands to achieve a similar level of quality in producing and implementing

guidance as Australia and the United Kingdom, particularly given more knowledge and evidence since both

2007 and 2010 when the aforementioned guidance packages were released.

1.2 Patient Care and Safety Compromising Events

The Rapid Response System (RRS) operates to predict and response to patient decompensation [5, 6].

The Rapid Response Team (RRT) operates as the part of the RRS’s intervention arm, responding to detected

patient decompensation and performing triage if necessary. The efficacy of this triage serves primarily to

prevent clinically significant events such as cardiac arrest, transfer of the patient to the ICU and death.

As discussed above, the Rapid Response System operates to improve patient safety and impact the events

associated with the decompensation of patients. These events serve as both the end result and complications

of inadequate attention to the second, third and fifth steps of the chain of prevention [30] (monitoring,


recognition and response). The events that stand out as significant when identifying the performance of the

RRS and RRT are:

1. The Adverse Event

2. The Patient Deterioration or Decompensation Event

3. The Failure-to-Rescue Event

The sections below describe the definition of each event in detail, as well as how it is detected or measured.

1.2.1 The Adverse Event

Rapid Response Systems“operate to mitigate clinically significant outcomes” [31], however that statement

does not encompass the scope of their intended practice. An additional function, in keeping with evidence,

is to identify precursors to adverse events, in particular “unintended injury that is due in part to delayed

or incorrect medical management and that exposes the patient to an increased risk of death and results in

measurable disability.” [32] In identifying these events, classification by peer review is unreliable, as discussed

by Forster et al [33] unless the particular adverse event probability is high.

The utility of adverse event detection is also in question, with methodological investigation as to the

best detection still ongoing [34, 35] in multiple hospital settings. Artificial intelligence methods in the ICU

environment have been used to detect adverse events [34, 36], in comparison to the following standard

mortality prediction models:

• Acute Physiology and Chronic Health Evaluation (APACHE) score1. Parameters developed from

expert consensus, and subsequently evaluated via statistical calibration and iterative remodeling.

• Simplified Acute Physiology Score (SAPS)2. Parameters selected via existing variables used in the ICU

as part of routine operation.

• Mortality Prediction Model [Multiple Logistic Regression] (MPM [MLR])score3. Variables drawn from

the admission form of Baystate Medical Center.

Academic methods used by both Hug [36] and Silva et al [34] exhibit a better predictive ability than the

comparator of SAPS II, however, a comparison against other scores has not been performed. Therefore, the

utility of these methods, while more powerful than SAPS II, is not thoroughly accurate. Further commentary

on the prediction of adverse events by Abenstein and Narr [48] suggests that objective decision support such

as that described in Taenzer et al [49] with statistically valid predictive methods are capable of providing

good utility of prediction, and thus the ongoing further investigation in literature is warranted.

1.2.2 The Patient Deterioration or Decompensation Event

Given that the patient deterioration or decompensation event is a central event that motivates triggering

of the Rapid Response Team (RRT), the definition of clinical deterioration is pertinent in understanding the

utility of the RRT. The frameworks used to identify patient deterioration from Jones et al [50] are itemized

below:

• Adverse events due to medical care and not due to underlying medical condition.

1Versions: APACHE [37], APACHE II [38], APACHE III [39], and APACHE IV [40].2Versions: SAPS [41], SAPS II [42] and SAPS III [43, 44].3Versions: MPM I [45], MPM II [46], MPM III [47]


• Clinical events are defined as adverse events e.g. myocardial infarction, thrombo-embolism,

renal failure.

• Other adverse events used include cardiac arrest and unplanned ICU admission. Patient

fulfils RRT criteria (single parameter trigger). Summed MEWS score (aggregate track and

trigger system). Considers organisational and system factors, patient factors, physiological

factors that change over time, and in response to therapy.

The first two of these frameworks require retrospective analysis to identify events, and the latter two, although

prospective, have not been completely validated to satisfaction with complex and true multi and intra-variate

mathematical modeling. The pertinent point identified at the end of the chain is the adverse event. The

classification of this event is informed by an analysis of the iatrogenesis and medical neglect associated with

the event. Due to this retrospective classification, the prospective detection of adverse events is linked to

the detection of a decompensation or deterioration event.

End state outcome selection also is required for the identification of patient deterioration. The three

significant outcomes related to patient decompensation in literature are ICU transfer, cardiac arrest and

death. These outcomes have been statistically validated by Churpek et al [51] to identify the discrimination

ability of vital signs to predict deterioration to outcome. With these, logistic regression of common selected

vital signs show the best predictors are of mortality, and the worst are of ICU transfer. A selection bias

is also described as being inherent in the vital signs used to validate the prediction of deterioration. This

bias is described as being “[...] introduced by clinicians who recognize these signs as markers of clinical

deterioration more readily than other signs” [51]. With an acknowledged selection bias of vital signs in

predicting decompensation to outcomes, the inclusion and exclusion of different parameters is still a topic of

much consternation. Recent literature has also suggested that beyond clinical vital signs, laboratory values

may be of import when predicting mortality for certain patient populations [25, 26, 27].

In identifying the deterioration event, the mechanism of prediction is also of import. Automated mecha-

nisms [52, 53, 54] have been developed, as well as clinical judgement aides [29, 23, 55, 56, 24, 21]. Deviation

from the normal levels by vital signs is the method by which patient deterioration is identified. Operating

principles allow for the alerting of clinical staff when these vital signs, or an aggregate sum drop below a

predefined threshold. Observation charts have also been investigated as a method of providing clinicians

with improved recognition of the deteriorating patient [57]. This chart exhibited color coded fields for the

quantitative trends in documented vital signs for deteriorating patients, and identified for staff when a Rapid

Response Team call should be placed. In summary, when designing the observational chart to mitigate error

rates and response times, there was a significant reduction in error rates between the control and re-designed

chart [57].

In the evaluation of deterioration prediction methods, the question of whether clinical judgement is a

required component of the detection scheme has been set as policy by most bodies acting to develop Rapid

Response Teams [19, 17, 21] Comparing provider judgement against objective scoring methods has resulted in

a difference between the selection of the moment of concern [58]. Additionally, there is a delay in recognizing

an adverse event noticed by both expert opinion and provider retrospection [58].

1.2.3 The Failure-to-Rescue Event

The failure-to-rescue (FTR) event is described by Silber et al [59] as f , “number of deaths in those patients

that have an adverse event occurrence” [59]. The rate of failure (F ) is the number of events f divided by

the number of adverse occurrences (a). This rate, or “probability that the hospital fails to rescue the patient

after the adverse occurrence” [59] is represented as the probability of death given adverse occurrence p(d|a).


In describing the concept of failure-to-rescue within mortality estimates, the following equation describes the

probability of death (d) as a function of adverse Event (a) [59]:

p(d) = p(d|a)p(a) + p(d|!a)(1− p(a)) (1.1)

Therefore, in describing f , the equation can be rearranged to be [59]:

p(f) ∼ p(d|a) =p(d)− (p(d|!a)(1− p(a)))

p(a)(1.2)

In Schmid et al’s [60] seminal literature review, the link between Rapid Response Teams and the failure-to-

rescue event is approached, in order to improve “nurse autonomy and control to rescue patients deteriorating

in a medical surgical setting” [60]. Schmid et al [60] note that empirically derived consequences of failure-

to-rescue are responded to by the use of more effective “surveillance, recognition and response-to-rescue

events” [60]. Furthermore, Taenzer et al [35], after surveying recent literature defines the failure-to-rescue

event as “hospital deaths after adverse events” [35] and describes it as “an established measure of patient

safety and hospital quality” [35]. Taenzer et al conclude that data supports the observation that “adverse

events in general ward patients are preceded by a significant period (on the order of hours) of physiologic

deterioration. Thus, the lack of early recognition of physiologic decline plays an major role in the failure-

to-rescue problem” [35] These two reports outline the utility of the Rapid Response Team and system as

mechanisms to enhance patient safety.

The failure-to-rescue concept further illustrates the use of the Rapid Response Team as a patient safety

tool. In the Institute of Medicine’s (IOM) report entitled Crossing the Quality Chasm: A New Health Sys-

tem for the 21st Century [61], it was estimated that Rapid Response Teams addressing the failure-to-rescue

problem was predicted to save 66,000 lives in the 100,000 Lives Saved Campaign [61]. To this end, the mea-

surement of patient safety associated with failure to rescue was investigated by the Agency for Healthcare

Research and Quality (AHRQ). Building off of Schmid et al [60] and Silber et al [59], the AHRQ developed

patient safety indicators (PSI) to quantify the failure-to-rescue event, focusing on hospitalized surgical pa-

tients. These patient safety indicators are capable of being calculated through the use of administrative data

and serve as a retrospective audit of hospital performance. [62] The Patient Safety Indicator (PSI) IV [63]

was proposed as an improvement over risk adjusted mortality rate to detect patient outcome differences

across multiple hospitals. Hospitals would therefore score higher if they exhibited less adverse events among

patients who experience complications. [59, 64] In line with the changes of the National Patient Safety Goals,

the patient safety outcome was also changed. It now refers to the death rate among surgical inpatients with

serious treatable conditions, or the failure to rescue based on secondary complications post-surgery. [63]

The current definition describes it as: “the numerator selecting all deceased patients meeting inclusion rules

for the denominator, and the denominator requiring that all surgical discharges 18 years or older, or any

pregnancies identified by an operating room procedure, or any elective surgery with complications of care

listed during death reporting.” [63] Romano et al [65] validated the occurrence of failure-to-rescue events,

and concluded that the event population within the surgical, medical and obstetric patients in American

non-federal acute care hospitals for the year 2000 numbers approximately 270,000 within 95% confidence, ap-

proximately 17.5% of 1,534,520 events. [65] These numbers were consistent across all types of hospitals, from

public to for-profit, rural to urban and academic to community. The failure-to-rescue event exhibited the

highest recorded incidence, identified by ICD-9 coding followed by Decubitus Ulcers and Obstetric Trauma

- vaginal without instrumentation.


1.2.4 Contextual Summary

In the overall context of Patient Care and Safety Compromising Events (PCSCEs), patient deterioration

serves to note the clinical behaviour of patients prior to episodic outcome, under which subsequent clinical

management occurs. The adverse event serves as the episodic outcome that results from mismanagement,

and that mismanagement is documented by the failure-to-rescue event. As Rapid Response Teams operate

to mitigate the PCSCE as a whole, they attempt to disrupt this cascade of events from occurring, and

correctly manage care and thus attempting to act upon the failure-to-rescue event. For this reason clinically

significant outcomes are selected for analysis as a method of tracking adverse events, and Early Warning

Scores, algorithms described as predicting patient decompensation are investigated, all in the context of the

Rapid Response System as a means to mitigate the failure-to-rescue event.

1.3 The Rapid Response System: Putting the Pieces Together

1.3.1 The Afferent Limb: Early Warning Scores

The afferent limb of the Rapid Response System functions to detect the conditions necessary to and trigger

the efferent limb. This mechanism functions to detect patient abnormalities and activate the appropriate

response, depending on the state of the patient. This method of operation is consistent with the track-and-

trigger response described in Smith et al [18], as well as the second to fifth links in the “chain of prevention”

described by Smith [30]. It is additionally worthwhile to note that in a recent narrative review in BMJ

Quality and Safety, Mark L Graber [66] describes the most promising tool in the identification of diagnostic

error in clinical practice is by “using ‘trigger tools’ to identify from electronic health records cases at high risk

for diagnostic error” [66]. Conventional wisdom in the diagnostic literature suggests that cognitive errors due

to both representative and availability heuristics, are the most common forms of error. [67] For this reason,

the isolation and study of the trigger mechanism in the afferent limb of the Rapid Response System serves

as a means to identify and investigate errors in patient care that come about as a result of diagnostic failure.

In designing a system to effectively detect error and patient events, Nangalia etl al describes the use of

monitoring solutions to observe patients without direct interaction. Such methods are described as being

capable of producing large quantities of data, however, “[the] interpretation [of this data] to be of clinical

use and much necessary research work remains to be done.” [68]. For this reason, although there are a wide

variety of monitoring systems capable of generating data; whether they be observational charts [57], dash-

boards [69] or standard care, the triggering system that detects the event in question and its implementation

requires further examination. Alongside the manual observation, Clinical Decision Support (CDS) systems

provide automated monitoring, and computer generated response cues as to when an event is within a pre-

dictive range. Tarassenko et al, Bellomo et al and Jones et al [54, 53, 70] all describe interventions with

provide decision support, via both static and dynamic systems providing information as to the condition of

the patient, and rationale behind the triggering action. Although these are not an extensive list of decision

support tools, they do illustrate some of the conceptual mechanisms for triggering an alert: probabilistic

trending, medical equipment monitoring and Electronic Health Record (EHR) sampling. The display meth-

ods also illustrate a wide variety of mechanisms: health index display, clinical alarms, personal electronic

alerting. Generally, however, in the case of the Rapid Response System, the Early Warning Score (EWS)

serves as the trigger mechanism for clinicians.

The EWS is an aggregate thresholding of multiple clinical parameters, compared to an overall threshold

value. For the purpose of this research, Early Warning Scores that operate on a binary basis taking any

abnormality as evidence of prediction, rather than an aggregate sum of parameter operation, will not be


evaluated. The constraints illustrated above serve to identify deterioration as a complex system, rather than

simple point disparities. Therefore, the EWS concept operates to quantify the deterioration of a patient

to the point at which an patient care or safety compromising event is believed to be predictable. Such

quantification is performed in an objective manner, from a basis of both clinical selection of parameters, and

statistical modeling. The validity of this clinical outcome prediction, however, often overlaps with clinical

judgement and practice that is not boxed in to a standard model for care, as outliers are not tolerated

within such a system. When compared to physician experience and judgement, objective modeling has been

investigated by Sinuff et al [71] in the ICU, and Fullerton et al [72] in the pre-hospital environment.Clinical

judgement is noted to have moderate discrimination ability, with low sensitivity yet high specificity when

compared to objective calling criteria. Furthermore, the addition of objective criteria to the decision process

improves sensitivity at the expense of specificity.

In evaluating the impact of objective criteria to the decision process, and tracing forward from Lee at

el [73], we know that the use of Early Warnings Scores has been built off the framework of chronic health

evaluation tools such as APACHE [37], SAPS [41] and MPM [45]. Furthermore, in recent literature, Sobol et

al [74] describes the use of a scoring mechanism of the clinical decision to admit patients following high-risk

intra-abdominal surgery into the ICU. This scoring mechanism, the Surgical Apgar Score [75] is used as a

predictor of mortality after surgery, via intra-operative variables, a philosophical extension of the chronic

health evaluation tools above, and is similar to the EWS in its role in predicting patient deterioration.

Other risk methods, such as the NICE guideline on the management of acute upper GI bleeding [76] support

the use of laboratory values such as urea, hemoglobin, alkaline phosphatase, bilirubin and transaminases

as measures of risk for PCSCE associated with the disease in question. In their development, such tools

have been shown to be effective in predicting clinical outcomes, however, they lack the temporal sensitivity

required to identify the acute patient care and safety compromising event. Furthermore, in describing the

reasons under which failure-to-rescue occurs, Jones et al [13] list “Intervals between measurements can easily

be 8 hours or longer.”, implying that this time window is still under investigation. To evaluate these criteria,

the identification of the most commonly cited and evaluated Early Warning Scores in literature follows:

• Figure B.3: Modified Early Warning Score (MEWS) [29], with 444 citations according to Google

Scholar, and 234 citations according to Web of Science.

• Figure B.4: VitalPAC™Early Warning Score (ViEWS) [23], with 63 citations according to Google

Scholar and 43 citations according to Web of Science.

Alongside these two commonly used and evaluated scores, the following three significant scores serve to

broaden the scope of retrospective literature comparison capable:

• Figure B.7: Cardiac Arrest Risk Triage Score (CART) [55], the most recent and believed effective Early

Warning Score to use statistical modeling as a base.

• Figure B.6: Cuthbertson Discriminant Functions (CDF) [56], the first major Early Warning Score to

be based upon statistical modeling and mathematical prediction methods.

• Figure B.5: National Early Warning Score (NEWS) [24], a recent modification of ViEWS developed by

the Royal College of Physicians in the United Kingdom to serve as a model for standardization across

the National Health Service.

1.3.2 The Efferent Limb: Rapid Response Teams

Following the use of an Early Warning Score to classify a patient as likely to progress towards a negative

clinical outcome or patient care or safety comprising event, the Rapid Response Team will be deployed.


In 1995, the concept of the Medical Emergency Team was described by Lee et al [73], which describes the

medical emergency team (first implemented in 1990) as a resuscitation team intended for the rapid detection

and correction of vital signs. This team was activated via

“specific conditions, physiological/pathological abnormalities and “any time urgent help is re-

quired” [73]

These activation criteria were reported to include:

“Acute respiratory failure, Status epilepticus, Coma, Pulmonary oedema, Severe drug overdose,

New arrhythmia, Acute severe exacerbation of asthma, Surgical, Upper airways obstruction,

Shock, Near drowning, Acute psychiatric disturbance, Carbon monoxide poisoning, Decreased

level of consciousness, Blood pressure, Respiratory rate, Pulse rate, Sodium level, Blood glucose,

Temperature, and Potassium level” [73]

This team from Lee et al is described as a modification of the cardiac arrest team (or code blue team) by

Hillman et al [77]. The precise terminology used to identify the team varies, and in the intervening years has

broadened to include Critical Care Response Team (CCRT), Rapid Response Team (RRT), and Critical Care

Outreach Team (CCOT). For the purposes of consistency, as noted previously, the term Rapid Response

Team (RRT) will be used. In addition, Hillman et al [78] conclude that

“There is a high incidence of serious vital sign abnormalities in the period before potentially

preventable hospital deaths. These antecedents may identify patients who would benefit from

earlier intervention.” [78]

From this information, and acknowledging the binary criteria described in Appendix I, the Rapid Response

Team’s role in intervening to prevent further escalation of vital signs abnormalities is reinforced, and the

conceptual framework for the detection and mitigation of patient care and safety compromising events is

established.

In describing the use of the Rapid Response Team, Ott et al [79] identify the characteristics of patients

experiencing calls in the radiology department. The Rapid Response Team in this study used a binary trigger

mechanism, similar to the Ministry of Health and Long Term Care calling criteria in Ontario in Figure B.8.

Patients were described as having a Charlson Comorbidity index of greater than or equal to 4; respiratory

support; cardiovascular support; cardiac arrhythmia; prior sedation; abnormal respiratory rate, heart rate,

systolic blood pressure or diastolic blood pressure; tachycardia or tachypnoea. These characteristics, along

with category of admitting diagnosis, demographic information, care location and radiology modality were

noted as the patient characteristics of patients receiving RRT care.

It is worthwhile to note, however, that the rapid response team is not the only technique that operates

on the same facet of patient safety in the literature. Verceles et al [80] have demonstrated that the rapid

response team is not necessary to reduce mortality when compared to a similar hospital protocol for de-

compensating (septic) patients. In addition to the septic cohort, Joffe et al [81] demonstrated that in the

pediatric population, hospital mortality detected by before-and-after Medical Emergency Team studies may

be confounded by co-interventions brought to bear during the studied time.

When investigating the triage performance of the Rapid Response System, Schneider et al [31] describe

that once patients are adjudicated to stay on the ward or transfer to ICU following a RRT call, repeat RRT

activations in that population is approximately 12.7%. In general, however, the conclusions of Schneider et al

were that “The rate of unexpected cardiac arrest in the 24 h following RRT activation is very low for patients

triaged to stay on the ward. Major triage errors by the RRT appear uncommon.” [31] This information

supports the notion that RRTs provide effective transitional care (triage services) to the decompensation


patient, and suitably mitigate subsequent activations by initiating the specialized resources to resolve the

crisis.

Rapid Response Team activation related to failure-to-rescue, or the failure of the RRT to correctly be

activated (afferent limb failure) was found by Tirkkonen et al [82] to be related to the documentation

frequency of vital signs. The RRT associated with this study was a binary mechanism related to abnormal

heart rate, systolic blood pressure, peripheral arteriolar oxygen saturation and respiratory rate. Comparing

documentation frequency of vitals, afferent limb failure (ALF) was more consistent between monitored and

normal ward patients. An afferent limb failure was termed as a “[..] MET criterion was documented at

20-360 min before the MET activation, it was recorded as a delayed MET activation and ALF.” [82] These

monitored patients were frequently monitored (less than 6 hours before the RRT call) than their normal

ward counterparts. Drawing from this information, as well as population characteristics such as International

Classification of Disease (ICD) diagnostic categorizations, the conclusion of Tirkkonen et al was that:

“Documentation of vital signs before MET activation is suboptimal. Documentation frequency

seems to increase if automated monitors are implemented, but our results suggest that benefits of

intense monitoring are lost without appropriate and timely interventions, as afferent limb failure,

delay to call MET when predefined criteria are fulfilled, was independently associated to increased

hospital mortality.” [82]

Therefore, activation performance of the RRT, as a binary mechanism of detecting patient vital sign abnor-

mality and subsequent crisis is suboptimal for the monitoring scheme presented.

In detecting medical errors, the activation of the Rapid Response Team serves as a surveillance method,

according to Braithwaite et al [1]. This surveillance is in line with the use of the RRT to identify patient

care and safety compromising events. This study served as a qualitative assessment of RRT episodes to

determine whether they were associated with medical errors. One third of the RRT calls investigated in

the study were found to be associated with errors, which were determined to be “generally serious and life

threatening” [1]. These errors, root causes and care process improvements note that the Rapid Response

Team, if operating objectively and with particular vital signs monitoring during critical elements of care, will

reduce known medical errors. Figure 1.1 describes these errors, root causes and care process improvements

from Braithwaite et al [1]:

Error Root Cause Care Process Improvement

Cardiopulmonary arrest, variousetiologies from delays in treat-ment

Ambiguous physician in charge Objective criteria for MET acti-vation

Inadequate surveillance for med-ical deterioration in patients re-ceiving radiology tests

Patients in radiology departmentnot supervised by medical per-sonnel

Pulse oximetry monitoring dur-ing radiology tests/procedureswith MET activation if alarm

Inadequate surveillance for med-ical deterioration in patients intransport

Patients in transport not super-vised by medical personnel

Pulse oximetry monitoring dur-ing transport with MET activa-tion if alarm

Figure 1.1: Root Cause Analysis of Medical Errors Addressed by the Rapid Response Team [1]

To this end, studies have attempted to quantify the effect Rapid Response Systems have on predicting

patient decompensation and outcomes. One significant study, named Medical Early Response, Intervention

and Therapy (MERIT) [83], concluded that:

“The MET system greatly increases emergency team calling, but does not substantially affect the

incidence of cardiac arrest, unplanned ICU admissions, or unexpected death.” [83]


Critiquing Hillman et al [83], Chrysochoou and Gunn [84] describe two significant points: that the study

was underpowered, and exhibited a lack of traceability post-education and pre-implementation to determine

effectiveness of RRT implementation. Following the MERIT study, Cretikos et al [85] attempted to qualify

the implementation of RRTs within the MERIT study. This qualification concluded that improved knowledge

of activation criteria, understanding the purpose of the RRT system and attitude towards the standard of

care and the RRT itself led to improved utilization of the RRT.

Beyond the MERIT study, Bellomo et al [86] conducted a prospective before-and-after trial of the Rapid

Response Team at a tertiary referral hospital. This trial concluded that outcomes related to mortality

and/or cardiac arrest decreased in incidence following the introduction of a Medical Emergency Team.

Specifically, the measurement of patient bed-days following survival of cardiac arrest in both ICU and

standard hospital care significantly differed before-and-after the introduction of the RRT [86]. In addition

to the measurement of patient bed-days, another study by Bellomo et al [87] showed that the incidence

post-surgery of adverse events, mortality and average duration of hospital stay reduced. These changes

were significant for the outcomes of respiratory failure, stroke, sepsis, renal failure accompanied by renal

replacement therapy, intensive care unit admissions, and duration of stay after major surgery. Recent

literature by Karpman et al [88] suggests that Rapid response Teams increase ICU admission rates, however,

fail to “improve the severity-of-illness adjusted outcome of patients transferred from the ward” [88]. This

failure highlights that the impact of rapid response teams have an impact on patient outcomes related to

ICU transfer, yet fails to improve mortality and severity of illness.

Furthermore, Chan et al [89] investigated the impact Rapid Response Teams have on hospital-wide code

rates and mortality. This study investigated four months of RRT calls and determined that “rapid response

team implementation was not associated with reductions in hospital-wide code rates or mortality.” [89] Buist

et al [90] investigated the impact that Rapid Response Team interventions reduce the incidence of unexpected

cardiac arrest and subsequent mortality. In this study, two years worth of patients, one cohort pre and one

cohort post RRT implementation were investigated. Based on rates of event, this study determined that

“In unstable clinically inpatients early intervention a medical team by emergency significantly reduces the

incidence of and mortality from cardiac arrest in unexpected hospital.” [90] Besides these two contradicting

studies, Howell et al [91] investigated the sustained effectiveness of a Rapid Response System composed of the

patients’ primary caregivers. Such a system activated based on the binary criteria from Bellomo et al [87],

and concluded that the system was “independently associated with reduced unexpected mortality” [91] In

summary, the effectiveness and composition of the Rapid Response Team and Rapid Response System are

lacks a consistent definition and direction, as was described in the systematic reviews prior.

The effectiveness of the Rapid Response Team also hinges somewhat on the property of dose. Jones et

al [92] address whether the utilization rate of the RRT or dose, and by proxy its sensitivity, affects efficacy.

In the discussion, call per admission ratio optimization was investigated, and a “dose” between “25.8 and 56.4

calls per 1000 admissions” [92] was described as optimum. The stability condition, however, is a ratio between

RRT activations and patient care and safety compromising events. This stability condition is endemic to the

purpose of the RRT: reducing the number of failure-to-rescue events.

In addition to the outcomes associated with the team, Reader et al [93] and Morris et al [94] have

investigated the team dynamics associated with the RRT. Conclusions drawn from these studies are that,

while effective teamwork is a requirement of optimal care, the composition of the RRT and subsequent

care hierarchy for optimal outcome performance has not been established, and requires further prospective

research.

Beyond acute outcomes, Jones et al [95] investigated the role of the RRT in end-of-life care in Australia,

Canada and Sweden. The study concluded that end-of-life care discussions were instigated in thirty-three

percent of calls, suggesting that end-of-life care and resources at these hospitals may not be available for all


patients at all hours of contact with the RRT. Furthering this research, Downar et al [96] investigated the

impact of RRTs on end-of-life care culture in the hospital. They concluded that again thirty-three percent

of dying patients were consulted by the RRT, meaning that end-of-life care support could be instigated

through the RRT. In tracking RRTs, the Province of Ontario’s Critical Care Information System (CCIS)

tracks whether an end-of-life discussion was initiated, and when. This metric provides for long-term planning

of institutional culture and use of the Rapid Response System for patients, with the potential for impact

across multiple visits and for chronic care purposes.

Chapter 2

Analytic Purpose

2.1 Problem Definition

2.1.1 Definition

The problem defined to be solved by this thesis is that: Rapid Response Teams promulgate a model

for the detection and mitigation of Patient Care and Safety Compromising Events (PCSCE), yet, from

recent discussion [97], there are deficits in care associated with the calling algorithms. To evaluate these

clinical deficits, an analysis of the Rapid Response Team and its contextual environment will contribute to

the reduction of said deficits, through the discovery of new knowledge.

In defining the problem analyzed by this thesis, the following knowledge deficits motivate the systematic

method by which analysis is conducted:

1. Lack of knowledge of how clinical variables impact the mechanism behind PCSCE within the context

of Rapid Response System activation, and the relationship between these variables. In the context of

the system analysis of the defined problem, this consists of the input characteristics of variables to the

binary PCSCE prediction model.

2. Lack of information concerning the detection of PCSCE, and which Early Warning Scores are most

predictive outside of their originating databases in classifying PCSCE. In the context of the system

analysis of the defined problem, this consists of the system operating characteristics of known PCSCE

event prediction mechanisms.

3. Lack of knowledge of which clinical variables elicit the pathogenic mechanism behind the PCSCE within

the context of Rapid Response System activation and the hospital encounter. In the context of the

system analysis of the defined problem, this consists of output analysis; PCSCE modeling along the

time-horizon.

2.1.2 Rationale

From evidence-based guidelines in both the United Kingdom [17] and Australia [19], conclusions have

been drawn that suggest further research into the evaluation of track and trigger systems in different clinical

settings, as well as the development and validation of track and trigger systems that advance the efficacy and

precision of care. Furthermore, the evaluation, auditing and feedback of these systems within the evidence

hierarchy of the guidelines is also noted as of import, with cost-effectiveness remaining an unknown variable.

14

Chapter 2. Analytic Purpose 15

From recent literature, both Loekito et al [25] and Jarvis et al [26] present laboratory values as being

predictive of mortality, in the context of imminent deterioration. Calzavacca et al [27] also present ad-

ditional biomarkers as being predictive of Rapid Response Team activation. Although recent individual

studies assist in providing a rationale for the examination of laboratory test and biomarkers as potentially

impacting the mechanism behind deterioration prediction and Rapid Response Team activation, the most

significant rationale is provided by the guideline presented by the National Institute for Health and Clinical

Excellence [17], in which the pathway of care for monitoring patients at risk of deterioration suggests that

monitoring biochemistry, specifically “lactate, blood glucose, base deficit, [and] arterial pH” [17] based on

clinical circumstances such as relevance is recommended as a component of track and trigger systems. In

addition to clinical and laboratory parameters, administrative data has been investigated for its ability to

qualify the admission to the Intensive Care Unit by Garland et al [98], and through hospital coding coding,

hospital abstracts were judged to adequately identify the presence and timing of admission to the ICU in

Manitoba, between 1999 and 2008.

2.2 Research Questions

1. What contribution do selected clinical and laboratory parameters have to the existing early warning

score predictive model of the Patient Care and Safety Compromising Events (death, cardiac arrest,

code blue, ICU transfer)?

2. How do the predictive accuracies of existing early warning scores compare with each other for the

identification of the Patient Care and Safety Compromising Events (death, cardiac arrest, code blue,

ICU transfer)?

3. Does imputation of missing data improve the predictive accuracy of Patient Care and Safety Compro-

mising Events (death, cardiac arrest, code blue, ICU transfer) by early warning scores?

2.2.1 Summary of Input Data and Outcomes

To answer the research questions above, a retrospective study was designed.

Dataset

The dataset of clinical and laboratory data was collected from a query of Toronto East General Hospital’s

(TEGH) Clinical Information System (CIS). A detailed description of the dataset can be found in Chapter 3.

Methods

Established computational and mathematical methods were used to establish solutions for the research

questions above, and generally analyze the predictive accuracy of clinical and laboratory variables as well as

existing early warning systems in the prediction of Patient Care and Safety Compromising Events. These

methods will be described in detail in Chapters 4, 5, and 6.

Outcomes

Although PCSCE are nuanced in their definition, in order to select measurable endpoints of clinical

significance, the following outcomes serve as endpoints of system function. These outcomes can be seen in

the systematic review literature as reliable endpoints for tracking RRT performance, or extensions thereof:

• Code Blue, including Cardiac, Respiratory Arrest and Other (CodeBlue), # occurrences: 77

Chapter 2. Analytic Purpose 16

• Cardiac Arrest only (CardiacArrest), # occurrences: 52

• Transfer to the ICU (ICUTransfer), # occurrences: 432

• Death (Death), # occurrences 660

• Composite Outcome combining Code Blue, Transfer to ICU and Death (Composite), # occurrences:

930

This data will be linked to the patient level, allowing for temporal identification of each anonymized subject,

outcome, event and parameter for each encounter(s) with the hospital.

2.2.2 Bias

• Bias in the selection of a total dataset not representative of the ward patient cohort will be dealt with

by not comparing analyses to the standard population, rather only as to a sample of “known” events

by the gold standard of classification (current clinical practice). Generally this bias negatively impacts

the ability of the patient cohort to accurately predict events associated with the general ward patient,

rather than events associated with the patient decompensation chain. This bias serves to impact

the generalizable model, through the identification of variable hierarchies capable of deriving outcome

predictions and solving for data features. Therefore, as a function of the patient cohort/environment,

this analysis serves to rate the known decompensation chain related adverse events, as a function of

their clinical and laboratory parameters, rather than a function of patient environment.

• Bias in the measuring of events in discrete versus continuous time will be dealt with by noting the

information gap and identifying the entropy inherent in the discrete measurement during analysis

described below, as well as using hazard models to examine the impact the temporal features of the

data have on outcomes modeling. This bias has the potential to negatively impact the goodness of fit

of predictive models that take into account the temporal nature of events, and synchronicity between

measurement points. Time to event estimations are also impacted, as the selection of a single point

along the temporal vector of an encounter for prediction incurs predictive penalties should there be a

limited number of variables collected at that point.

• Bias due to the definition of failure-to-rescue not incorporating clinical judgement in its ability to

identify patient decompensation and following potential activation of a RRT. Given that the failure-to-

rescue event is identified solely by the accompanying activation of the RRT, and the identification of

the RRT activation is bereft of clinical context, this bias must be incorporated into the interpretation

of the results, and the framing of the adverse event detection sequence.

• Bias due to the lack of data for each identified parameter across each patient and temporal index will

be dealt with by:

1. Normalizing temporal indices to the first point of measurement per encounter.

2. Selecting patient sub-groups within the dataset to provide for the analysis of parameters that are

not found in all patients.

3. Ensuring that temporal differences between outcome and measurement are included in the anal-

ysis.

This bias negatively impacts the ability of models to incorporate the per encounter variance of param-

eters in their prediction of events without the use of imputation or exclusion.

Chapter 3

Characteristics of Data

There are two subsets of the data used for analysis in this thesis:

1. Administrative Data

Table B.1 contains the header variables present in the Critical Care Secretariat’s reporting tool to track

RRT operation.

2. Clinical Data

Figure B.2 lists the clinical parameters selected from the previous chronic health evaluation scores

(APACHE, SAPS, MPM), and early warning scores (MEWS, ViEWS, NEWS, CART, CDF). Table B.3

describes clinical and laboratory parameters present in the query from TEGH selected for analysis.

The dataset generated for this project was obtained from the Electronic Medical Record (EMR) of Toronto

East General Hospital, an enterprise software package by the Cerner Corporation. Through this package,

the variables queried from the EMR to generate information about the patient from laboratory, pathology

and point of care sources. To simplify the query, data corresponding to laboratory and pathology services

was selected in bulk, including the set of parameters in Figure B.2, resulting in the total list of parameters

in Table B.3.

Data analysis was conducted using the R language for data analysis and graphics [99], version 2.15.3

(2013-03-01) – ”Security Blanket”, compiled for the i486-pc-linux-gnu (32-bit) platform. The computer

running R used Debian Linux 6.0, kernel 2.6.32-5-686 #1 SMP Fri May 10 08:33:48 UTC 2013 i686.

When processing the data, the language itself provides several barriers. Specifically, in R, vectors of

character data are stored as factors, rather than numeric. Converting a factor to a numeric vector requires

the specification of said factor’s levels (an ordered list of coded values). As the pathology and laboratory

variables varied by their ability to contain information beyond a binary value or free text field, their ability

to be numerically predictive without selective analysis requires a great deal of human input, and potential

clinical opinion. As a result, non-numeric results are excised from the database, as there is no easy way

to batch process each parameter to render them all useful. In addition, many of these variables exhibit a

high degree of missingness, making their impact on numerical analysis subsequently be negative [100]. For

this reason, non-numerical values are excised from the dataset, and variables containing solely non-numerical

values excised from the list.

Another core element of the data is the missingness of the dataset. Table B.6 describes increments of

a threshold number of encounters, and the percentage of variables with less than or equal to that number

containing data, number of variables in the naught set, and the names of those variables. This table includes

the calculated Early Warning Scores for each encounter, which are present in all encounters by virtue of

17

Chapter 3. Characteristics of Data 18

their being calculated within each encounter and included in the summary data frame passed into the table

generating function. In order to select a dataset that exhibits a low degree of missingness, the cutoff of 80%

of encounters containing data was selected (1662.4 encounters is exactly 80%), and as a result, Table B.6

only shows data for those with data for encounters from 70% and above.

Taking both of the above points into account, in order to optimize the performance of the modeling, a

subset of the data was selected. This subset reduces the parameter list to:

• Parameters with numeric values.

• Given 2078 encounters, and that 80% of 2078 = 1662, parameters that have values in greater than

approximately 1662 encounters. From the table, the closest number is 1661, with 91.48% of parameters

below the encounter number threshold.

Although the early warning score calculations are included in the lists of variables above the threshold, they

do exist for all encounters by process of their creation and will automatically be included in the dataset.

For the purposes of logistic regression, they will be not be included. In selecting this subset, two significant

points of bias are introduced:

1. Given that laboratory values are not collected unless ordered by clinical staff, the impact on outcome

prediction of each parameter is not necessarily reflected in the frequency of parameter collection. For

this reason, this selection method may introduce bias.

2. Furthermore, the clinical state and disposition of the patient is not taken into account on selection. For

example, given that laboratory testing such as glomerular filtration rate (EstimatedGFR) is a method

of identifying kidney dysfunction, for a population of patients with kidney disease it may be a useful

predictor of outcomes, yet for the general population it is not tested enough, nor does the test provide

enough information to be consistently useful.

Therefore, using the selection method above, and consulting the table (removing the early warning scores),

the following parameters were input into the logistic regression: AnionGap, Baso#, Chloride, Creatinine,

DiastolicBP, Eos#, EstimatedGFR, Hb, HCT, INR, Lymp#, MCH, MCHC, MCV, Mono#, MPV, Neut#,

Platelets, Potassium, Pulse, RBC, RDW, RespiratoryRate, Sodium, SystolicBP, Temperature, TotalCO2,

Urea, VSFiO2, WBC. Given that they are calculated values, and would exhibit high interdependence, thus

confounding the analysis, the Early Warning Scores are not included in the set of parameters fed to further

analysis.

Bayley et al [101] recently performed a case study documenting the challenges in using electronic health

data for comparative effectiveness research, and in doing so, came up with categories of challenges [101]:

• Missing data

• Erroneous data

• Uninterpretable data

• Inconsistencies among providers and over time

• Data stored in non-coded text notes

Of these challenges, missing, erroneous and inconsistent data are all visible in the numerical data used for

analysis, and “data stored in non-coded text notes” contributes to the lack of information associated with

the non-numerical pathology and laboratory data.

In selecting the aforementioned subset to account for missingness and non-coded text values, several

significant points of bias are introduced:


1. Given that a great deal of this data is transcribed into the EMR, erroneous data may exist in the dataset,

and subsequently introduce numerical bias due to outliers passing through the analysis. Additionally,

the lack of known units for the dataset also proves a problem, as identifying outliers without context of

the normal range is impossible from a clinical standpoint. Furthermore, the potential for inconsistent

recording methods for vital signs, or laboratory processing may also result in erroneous data, as well

as a high degree of variability of parameter measurements that are not consistent with the true value.

2. Given that laboratory values are not collected unless ordered by clinical staff, the impact on outcome

prediction of each parameter is not necessarily reflected in the frequency of parameter collection. For

this reason, this selection method may introduce bias.

3. The clinical state and disposition of the patient is not taken into account on selection. Given that

laboratory testing such as glomerular filtration rate (EstimatedGFR) is a method of identifying kidney

dysfunction, for a population of patients with kidney disease it may be a useful predictor of outcomes,

yet for the general population it is not tested enough, nor does the test provide enough information to

be consistently useful.

4. Beyond clinical biases, several variables are calculated from other parameters, leading to the potential

for collinearity, and variable functional interdependence:

• MCH is calculated via Hb and RBC

• MCHC is calculated via Hb, MCV and RBC

• MCV is calculated via HCT and Hb

• EstimatedGFR is calculated via Creatinine, Age, Weight (and Height, Crystatin C in some for-

mulae)

Given their representative value clinically, however, the ability to excise them from the dataset warrants

future further consideration and impact assessment.

Right censoring occurs when the final outcome for a subset of data is unknown, and the patients have

passed the observation threshold. Left censoring occurs when observed survival time is less than the actual

survival time, for example, when the start point of the patient observation is unknown. Interval censoring

occurs when the observation failure occurs within a fixed interval, such as an event occurring post-discharge,

but the patient is subsequently readmitted. The dataset exhibits multiple types of censoring at each instance,

as the parameters in which the Early Warning Scores are calculated from are not collected in regular batches,

and the events, although tied to the encounter, are not often within the temporal window of observed clinical

data. The use of survival modeling as a motivating factor to use survival modeling, and forms the basis of

the research question associated with the output analysis in Chapter 6. Hug has demonstrated this method

as a means to tackle ICU patient mortality estimation in his masters thesis [102].

Recent publications have examined the collection of vital signs at point of care, and the recording of those

values in the EMR. Semler et al [103] conducted a study investigating the accuracy by which respiratory

rate was recorded, using a flash mob technique to ensure randomization of observation. The conclusion from

this study was that:

“Among hospitalized patients across the United States, recorded respiratory rates are higher than

directly observed measurements and are significantly more likely to be 18 or 20 breaths/min.”[103]

Fieler et al [104] examined the transcription of vital signs via the device itself. Rather than having an

intermediate recording step, the physiological monitoring equipment was configured to send the vital signs


information directly to the EMR. The study exhibited heavy bias in the evaluation of the workflow of the

equipment, in particular with regards to errors in equipment function. In summary, documentation errors

are understood and known to impact the quality of information passed into the EMR. Potential ways for

mitigating transcription error have been developed by vital signs monitoring equipment [104], however have

not been deployed significantly enough to have an impact on global workflow. Additionally, more attention

is required to prevent bias at point of collection and prevent data inconsistencies [103].

Calzavacca et al [27], Jarvis et al [26] and Loekito et al [25] both present models for the prediction of

Rapid Response Team activation, as well as Death using biomaker and laboratory information. Of the values

from these papers, most of the biomarkers are not available in the dataset, however, Table 3.1 describes the

values from the papers (separated by a double line), and whether they were passed to analysis.

Parameter Analysis Decision

Brain natriuretic peptide Not available in data query

D-Dimer Not passed to analysis (<80% encounters with data)

Myoglobin Collected as binary screening variable, not included

in numeric dataset

Creatine kinase MB isoenzyme Not available in data query

Troponin 1 Troponin T, a similar compound, was not passed to

analysis (<80% encounters with data)

Hemoglobin Passed to analysis (>80% encounters with data)

Urea Passed to analysis (>80 % encounters with data)

White Blood Cell Count Passed to analysis (>80 % encounters with data)

Albumin Not passed to analysis (<80% encounters with data)

Creatinine Passed to analysis (>80 % encounters with data)

Sodium Passed to analysis (>80 % encounters with data)

Potassium Passed to analysis (>80 % encounters with data)

Hematocrit Passed to analysis (>80% encounters with data)

Total Bicarbonate Not passed to analysis (<80% encounters with data)

Bilirubin Not passed to analysis (<80% encounters with data)

pH Not passed to analysis (<80% encounters with data)

Table 3.1: Laboratory and Biomaker Variables Present in

Query from TEGH Clinical Information System and Capable of

Passthrough for Analysis

3.1 Patient Population

The patient population for this analysis is the set of patients who have had a new Rapid Response Team

consult between 2009 and 2012 from Toronto East General Hospital (TEGH), N = 2528 patients. Toronto

East General Hospital is a community general hospital in Toronto, Ontario, Canada. Its primary academic

and research affiliations are with the University of Toronto, however, it does not serve as a primary member

of the Toronto Academic Health Sciences Network. It serves as a teaching hospital for community medicine,

and as the home for several researchers at the University of Toronto Faculty of Medicine.

Patients with these consults are identified to the Ontario Critical Care Secretariat, a component of

the Province of Ontario’s Ministry of Health and Long-Term Care. The Critical Care Secretariat was


established in June 2004, and is responsible for the “overall implementation and evolution of the initiatives

under Ontario’s Critical Care Strategy” according to the Ministry of Health and Long-Term Care’s website 1.

Ontario’s implementation of the Rapid Response Team concept has been part of the Provincial Critical

Care Strategy since 2006. This has resulted in a patient population that has been identified as potentially

decompensating, whose outcomes are associated with the quality of care provided by both the hospital in

which the RRT resides, as well as the RRT itself. [20, 21]

In order to ensure that the patient set contained those with potential failure-to-rescue events associated

with the operationalization of the early warning score paradigm, patients were selected from those those

patients identified to Ontario’s Critical Care Secretariat as requiring RRT intervention at the participating

hospitals.

3.2 Administrative Data: Critical Care Secretariat

This dataset was constructed via a query of the Ontario Critical Care Secretariat’s electronic RRT

reporting system, which collects information from each hospital (as reported at the end of the month) via

the submission of data collection forms to the Secretariat’s office. The field headers of the Critical Care

Secretariat data can be found in Table B.1. From these headers, a description of the data contained in each

field can be found in Table B.2. This data does exhibit missingness, with the field appearing blank, however,

it also does some fields denote the lack of data, rather than the expected response.

3.3 Clinical Data: Toronto East General Electronic Health Record

Query

This dataset was collected via queries against the TEGH EMR. Code for these queries can be found in

Sections A.1, A.2, A.3, A.4. The variables that were the result of the query and subsequent selection detailed

above are described in Table B.3, and their numerical characteristics are described in Table B.4. The full

set of parameters and the quantiles dividing the number of encounters in which each variable contains data

can be seen in Figures B.9, B.10, B.11, B.12.

3.4 Pre-Processing

3.4.1 R Libraries Required

The libraries used to perform both preprocessing and analysis in R follow:

• xtable [105]

• Hmisc [106]

• ggplot2 [107]

• lattice [108]

• scatterplot3d [109]

• rmeta [110]

• randomForest [111]

• rms [112]

• MASS [113]

• copula [114]

• entropy [115]

• fclust [116]

• pROC [117]

• cmprsk [118]

1http://health.gov.on.ca/en/pro/programs/criticalcare/secretariat.aspx

http://health.gov.on.ca/en/pro/programs/criticalcare/secretariat.aspx


• glmnet [119]

• ResourceSelection [120]

• base [99]

• stats [99]

3.4.2 Step I

1. Load base Critical Care Secretariat dataset

2. Load base Encounter dataset, created by TEGH to manage patient identifiers

3. Load base Clinical and Laboratory data, queried by TEGH

3.4.3 Step II

1. Change all date strings to numeric UNIX epoch time in order to facilitate comparison between values

2. Filter the Encounter and Critical Care Secretariat datasets to remove all encounters not found in the

Clinical and Laboratory dataset, for future matching

3. Create unique identifiers for each Rapid Response Team call

4. Excise all non-numerical data from the Clinical and Laboratory dataset, and ensure that all remaining

variables are of type numeric, exclude ’MEWSAVPUScale’ from excision

3.4.4 Step III

1. Create transforms to allow for conversion between identifiers:

• Encounter ID

• Patient ID

• RRT Call ID

3.4.5 Step IV

1. Process per encounter the maximum, minimum, mean, and median values per clinical and laboratory

parameter, as well as early warning scores and pass them into a master matrix

3.4.6 Step V

1. Generate for each category of outcome (Code Blue, Transfer to ICU, Death) a data frame of information

concerning each instance of an outcome, including:

• Date and time stamp of outcome

• Encounter ID for instance

• Patient ID for instance

• Qualifying information concerning the event


3.4.7 Step VI

1. Create a wrapper function for the outcome data above, allow for the selection of outcomes (Code Blue,

Transfer to ICU, Death, Cardiac Arrest, Composite of all outcomes except Cardiac Arrest)

2. Produces summary table of clinical and laboratory data per encounter, for an amended version, see

Table B.6

3.4.8 Step VII

1. Create data frame per encounter and vital sign of temporal data associated with measurement

3.4.9 Step VIII

1. For each early warning score, per encounter, generate the score within a temporal threshold of each

outcome’s date and time stamp (Death, ICU Transfer, Code Blue), along with the calculated score.

Chapter 4

Input Analysis

4.1 Introduction and Rationale

In order to evaluate parameter impact on the binary prediction of clinical outcomes, as a method of

predicting patient decompensation, regression and classification methods were selected. Furthermore, in

order to appropriately gauge the impact of these parameter’s independence, the use of a Bayesian statistical

model was selected.

Logistic regression serves as a standard method of binary outcome prediction, which was used to create

both of the Early Warning Scores developed via statistical modeling, CART and CDF. In the preface to

Applied Logistic Regression [121], Hosmer and Lemeshow describe logistic regression as:

“[...] commonly employed in many fields including but not nearly limited to biomedical research,

business and finance, criminology, ecology, engineering, health policy, linguistics and wildlife

biology” [121]

Logistic regression does not inherently use a method for marginal interpretation, and as such requires ad-

ditional methods to attempt to separate variable co-linearity, such as stepwise regression [113] and Least

Absolute Shrinkage and Selection Operator (LASSO) regularization [122].

Rather than solely using continuous values via regression, the identification of group-based parameters to

influence outcome prediction via a classification model allows for a different approach to outcome prediction.

In selecting a classification technique from the supervised learning literature to compare against logistic

regression, Random Forests were selected based on reporting from the International Conference on Machine

Learning. From the aforementioned conference, Caruana et al [123] report that Random Forests, out of a

selection of supervised learning techniques perform best when empirically compared against other supervised

learning methods, given that both before and after scaling they rank in the set of top three performing tools

(no other tools were duplicated before and after scaling).

The Empirical Copula was discussed as a method of establishing the joint distributions between variables

for the retrospective modeling of the likelihood of acute deterioration in patients at the 2012 International

Conference on Complexity in Acute Illness [124]. This accounting for multivariate analysis is a means of

identifying sources of type I error, as well as the evaluation of the precision of parameter estimates [100].

In parallel to the methods used for analysis, in order to adequately account for clinical and laboratory

parameters which have varying number of values collected in each encounter, as well as abnormalities at

different time points, the clinical and laboratory dataset is propagated for modeling using both maximum

and minimum values for each variable per encounter. In addition to modeling for variability, Table 4.1

describes the directionality of the parameters of values in the clinical and laboratory dataset, as described

24

Chapter 4. Input Analysis 25

by the Chief of the Department of Critical Care Medicine, Sunnybrook Health Sciences Centre, Toronto,

Ontario, Canada. Running this subset, selected for known directionality (with increasing using maximum

and decreasing using minimum values per encounter), allows for an estimated of the impact of directionality

on predictive modeling to be investigated.

Parameter Directionality

EstimatedGFR Unidirectional Decreasing

Potassium Primarily Increasing, Bidirectional

DiastolicBP Primarily Decreasing, Bidirectional

MPV Primarily Decreasing, Bidirectional

MCHC Primarily Decreasing, Bidirectional

Baso# Unidirectional Increasing

Lymph Primarily Increasing, Bidirectional

RDW Unknown

Chloride Primarily Decreasing, Bidirectional

Sodium Primarily Decreasing, Bidirectional

Age Unidirectional Increasing

VSFiO2 Unidirectional Increasing

Urea Unidirectional Increasing

WBC Primarily Increasing, Bidirectional

INR Unidirectional Increasing

SystolicBP Primarily Decreasing, Bidirectional

Creatinine Unidirectional Increasing

RespiratoryRate Primarily Increasing, Bidirectional

Hb Primarily Decreasing, Bidirectional

MCH Primarily Decreasing, Bidirectional

Eos# Unidirectional Increasing

RBC Primarily Decreasing, Bidirectional

HCT Primarily Decreasing, Bidirectional

TotalCO2 Primarily Decreasing, Bidirectional

Pulse Primarily Increasing, Bidirectional

AnionGap Unidirectional Increasing

Temperature Primarily Increasing, Bidirectional

MCV Primarily Decreasing, Bidirectional

Neut# Primarily Increasing, Bidirectional

Platelets Primarily Decreasing, Bidirectional

Mono# Unidirectional Increasing

O2SaturationArterial Unidirectional Decreasing

Table 4.1: Expert Opinion as to Directionality of Selected Param-

eters


4.2 Logistic Regression

4.2.1 Methods

Linear Models and Generalizations: Least Squares and Alternatives [125] discusses the use of the Gen-

eralized Linear Model (GLM) for binary response (binomial) via the logit link, which is termed the logistic

regression model. These models are described by the relationship between the expected value of a response

variable and predictor values [125]:

E(yi) =

J∑j=1

xijβj = x′iβ (4.1)

Where the linear predictor is described as ηi = x′iβ, therefore E(yi) = ηi = x′iβ [125]. Describing the link

function modeling the relationship as g(πi) = logit(πi) between E(yi) and ηi [125]. Therefore we can express

the linear predictor in terms of yi, given i ∈ [1, I] predictors as [125]:

ln

(πi

1− πi

)= x′iβ (4.2)

The logistic function can be described by the equation [125]:

π(x) =ex

1 + ex, π ∈ [0, 1] (4.3)

As discussed previously, in performing parameter selection to optimize the models and the impact of

co-variates, two standard methods implemented for GLM in R were used [113, 119]:

Stepwise Model Selection

Stepwise model selection via Akaike’s Information Criterion (AIC) [126] operates to iteratively add and

drop terms to optimize the reduction of the model’s AIC, as described in Bozdogan [127] and Venables and

Ripley [113]. The range of models examined in the stepwise search include all variables within the clinical

and laboratory dataset, and continues until the evaluation of those variables has been performed.

Model Fit via LASSO Regularization

The Least Absolute Shrinkage and Selection Operator (LASSO) regression shrinkage and selection method,

first proposed by Tibshirani [122] operates to: “minimize the residual sum of squares subject to the sum of the

absolute value of the coefficients being less than a constant.” [122] This method attempts to account for the

variance in estimated coefficients, and provide for interpretable models by potentially removing coefficients

in an iterative total fashion. This method is described in Friedman et al [119], and serves to fit the GLM

via a LASSO penalized maximum likelihood regularization path. This path is defined by the equation [119]:

min(β0,β)∈R

[1

2N

N∑i=1

(yi − β0 − xTi β)2 + λPα(β)

], αε→0 = 1 + ε (4.4)

Pα(β) =

p∑j=1

[1

2(1− α)β2

j + α | βj |]

(4.5)

In optimizing this function, and performing k-fold cross validation on the model, the parameter of minimum

mean cross validation error (binomial deviance) acts to minimize the deviance from the logistic best fit [128].


4.2.2 Results

Figures B.36 and B.34 describe the p-values for the base set of clinical and laboratory parameters with

and without accounting for directionality, prior to optimization through stepwise model selection or LASSO

regularization. Figures B.37 and B.35 describe the p-values for both base aforementioned sets following

stepwise regression. Tables B.22, B.23, B.24, B.25, B.26, B.27, B.28, B.29, B.30, B.31, B.12, B.13,

B.14, B.15, B.16, B.17, B.18, B.19, B.20, and B.21 describe the raw output from R of the model output

parameters of these figures in numerical detail. Tables B.10 and B.11 describe the raw output from R of

the selected parameters based on β coefficient computation for the minimum mean cross-validation error,

per encounter for both of the clinical and laboratory parameter sets.

Code Blue

• For the outcome of code blue, minimum INR is the most predictive parameter, whereas the maximum

basophil count is the least predictive.

• Using stepwise optimization, maximum mean corpuscular volume (a function of hematocrit and hemoglobin)

is the most predictive parameter, with maximum estimated GFR as the least predictive, and basophil

count has been removed from the subset. Final Akaike’s Information Criterion (AIC): 512.1

• Accounting for directionality, the regression of the base subset selects maximum mean corpuscular

volume as the most predictive variable, whereas maximum potassium is the least predictive.

• Performing stepwise optimization on the directional subset, maximum mean corpuscular volume is again

the most predictive parameter, while maximum platelet count is the least predictive, and potassium

has been removed from the subset. Final Akaike’s Information Criterion (AIC): 377.13

• For the LASSO regularization, comparing the selection of the parameters to the base set results in a far

more selective subset, where values represent primarily laboratory values including minimum INR and

maximum mean corpuscular volume. Basophil count has been excised, although maximum estimated

GFR has not.

• Again for the LASSO regularization, accounting for directionality shows primarily laboratory values,

however, minimum systolic blood pressure along with respiratory rate are included in the subset as

is maximum mean corpuscular volume. maximum platelet count has been removed from the dataset

(with minimum platelet count remaining), although maximum potassium has not been removed from

the dataset.

Cardiac Arrest

• For the outcome of cardiac arrest, again minimum INR is the most predictive parameter, and again

the minimum diastolic blood pressure is the least predictive.

• Using stepwise optimization, maximum mean corpuscular volume is again the most predictive param-

eter, yet maximum potassium is the least predictive and only minimum diastolic blood pressure has

been removed from the subset. Final Akaike’s Information Criterion (AIC): 406.06

• Accounting for directionality, the regression of the base subset selects maximum mean corpuscular

volume as the most predictive variable, whereas maximum creatinine is the least predictive.


• Performing stepwise optimization on the directional subset, maximum mean corpuscular volume is

again the most predictive, while maximum platelet count is the last predictive, and creatinine has been

removed. The p-value magnitudes, although not shifted significantly are improved by the optimization

(the only outcome case in which values range solely from 0.9 to 1 in both cases). Final Akaike’s

Information Criterion (AIC): 294.5

• For the LASSO regularization, comparing the selection of the parameters to the base set results in a

subset with no values, as the minimum cross-validated error results in β coefficients of 0 for all parame-

ters, given a maximum λ, or highest variability within the dataset. Taking the next closest approximate

value, we note that it does contain both minimum INR and maximum mean corpuscular volume given

an approximate λ of 0.01 and mean cross-validated error of 0.27 for the set (AnionGap.Max, INR.Min,

MCV.Max, TotalCO2.Min). Both maximum potassium and minimum diastolic blood pressure have

been removed from the subset.

• Again for the LASSO regularization, accounting for directionality comparing the selection of the pa-

rameters to the base set results in a subset with no values, as the minimum cross-validated error

results in β coefficients of 0 for all parameters, given a maximum λ, or highest variability within the

dataset. Taking the next closest approximate value, we note that it does contain maximum mean

corpuscular volume given an approximate λ of 0.01 and mean cross-validated error of 0.34 for the set

(SystolicBP.Min, AnionGap.Max, MCV.Max). Both maximum platelet count and creatinine have been

removed from the subset.

Transfer to the ICU

• For the outcome of transfer to the ICU, minimum respiratory rate is the most predictive parameter,

and the maximum mean platelet volume is the least predictive.

• Using stepwise optimization, minimum respiratory rate is the most predictive parameter, while min-

imum lymphocyte count is the least predictive, and mean platelet volume has been removed. Final

Akaike’s Information Criterion (AIC): 1442.83

• Accounting for directionality, the regression of the base subset selects minimum respiratory rate as the

most predictive variable, whereas maximum hematocrit is the least predictive.

• Performing stepwise optimization on the directional subset results in the selection of minimum res-

piratory rate as the most predictive parameter, yet maximum diastolic blood pressure as the least

predictive, and hematocrit has been removed. Final Akaike’s Information Criterion (AIC): 1063.86

• For the LASSO regularization, comparing the selection of the parameters to the base set results in a

far more selective subset, where values represent primarily laboratory values, including minimum respi-

ratory rate. Mean platelet volume has been removed from the subset, although minimum lymphocyte

count has not.


however, minimum respiratory rate is included in the subset. Maximum hematocrit has been removed

(although minimum hematocrit remains), and maximum diastolic blood pressure is still contained

within the subset.


Death

• For the outcome of death, minimum red cell distribution width is most predictive parameter, yet the

minimum sodium is the least predictive.

• Using stepwise optimization, minimum red cell distribution width is the most predictive parameter,

while minimum white blood cell count is the least predictive parameter, and only minimum sodium is

removed from the subset. Final Akaike’s Information Criterion (AIC): 1508.44

• Accounting for directionality, the most predictive parameter is maximum age, while the least predictive

parameter is minimum red blood cell count.

• Performing stepwise optimization on the directional subset results in maximum age being selected as

the most predictive parameter, and mean corpuscular hemoglobin is the least predictive, and only

minimum red blood cell count is removed from the subset. Final Akaike’s Information Criterion (AIC):

952.6

• For the LASSO regularization, comparing the selection of the parameters to the base set results in a far

more selective subset, where values represent primarily laboratory values, including minimum red cell

distribution width. Sodium is not removed from the dataset, while minimum white blood cell count is

excised.


however, maximum age is included. Minimum red blood cell count is removed from the dataset (while

maximum red blood cell count is included), while mean corpuscular hemoglobin is removed entirely.

Composite Outcome of Code Blue, Transfer to ICU and Death

• For the composite outcome of code blue, transfer to the ICU and death, minimum respiratory rate is

the most predictive parameter, while minimum systolic blood pressure is the least predictive parameter.

• Using stepwise optimization, minimum respiratory rate is the most predictive parameter, while mini-

mum diastolic blood pressure is the least predictive parameter, and systolic blood pressure is removed

from the subset. Final Akaike’s Information Criterion (AIC): 1849.81

• Accounting for directionality, minimum respiratory rate is the most predictive parameter, and minimum

hematocrit is the least predictive parameter.

• Performing stepwise optimization on the directional subset, results in minimum neutrophil count as

the most predictive parameter, while minimum red blood cell distribution width is the least predictive

parameter and hematocrit is removed from the subset. Final Akaike’s Information Criterion (AIC):

1111.36

• For the LASSO regularization, comparing the selection of the parameters to the base set results in

a far more selective subset, where values represent primarily laboratory values, including minimum

respiratory rate. Diastolic blood pressure is removed from the subset, as is systolic blood pressure.


although rather than minimum, maximum respiratory rate is included in the dataset. Both red blood

cell distribution width and hematocrit are removed from the subset.


In summary, the laboratory parameters that are shown to be predictive through regression modeling

are: hematology results (MCV, MCH, RDW, INR), as well as age and respiratory rate are proven to be the

most predictive parameters accounting for variability and directionality. As minimizing the AIC produces a

preferred model, we can also note that accounting for directionality produces more preferable models using

stepwise optimization.

4.3 Random Forest Classification

4.3.1 Methods

Random Forests are “a combination of tree predictors such that each tree depends on the values of a

random vector sampled independently and with the same distribution for all trees in the forest” [129]. The

classification algorithm itself is described below [130]:

1. For t ∈ [1, T ]:

(a) Draw a bootstrap sample Z from the training data, of size N . This bagging averages

noise across models, to reduce variance. Given that trees generated in bagging are

identically distributed, the expectation of a number of such trees is therefore the same

as each individual. This is confirmed by the variance equation for B trees:

var(mean(B)) =σ2

B= ρσ2 +

1− ρB

σ2 (4.6)

(b) Grow a Random Forest tree Trf to the bootstrapped data, by recursively repeating at

the terminal node the following until a defined minimum node size is reached.

i. Select a set number of variables at random m of N , where m ≤ N ∈ R,√Nor1

ii. Pick the best variable/split point from the aforementioned set

iii. Split the node into two daughter nodes

2. Output the ensemble of trees: {Trf}T1 .

3. To make a prediction at each new point x:

Let Ct(x) be the class prediction of the tth Random Forest tree. Therefore CTrf (x) =

majority vote {Ct(x)}T1 .

An important component of the Random Forest algorithm is the evaluation of variable importance. For

this purpose, two significant components of the algorithm require addressing: majority voting, and out of

bag sampling. Majority voting occurs following class prediction, and is mathematically expressed via the

following. For the K-class response, at observation pair (x, y) the bagging estimate is defined as [130]:

fbag(x) =1

N

N∑n=1

(f)∗n(x) (4.7)

Where (f)∗n(x) is the prediction for bootstrap sample n ∈ 1, 2, ..., N . In the K-class response, (f)bag =

{p1(x), p2(x), ..., pK(x)} with each pk(x) = # trees predicting class k# trees , atx. [130]. For B trees, the classifier

selects the class with the most “votes” (i.e. predicting trees) from the B trees, so that the classifier function

Gbag(x) = arg maxk(fbag(x)) [130].

Out of bag (OOB) sampling occurs at each observation (x, y), as the set of trees corresponding too

bootstrap samples in which (x, y) does not appear. This set is used to create two variable importance

measures:


• Decrease in Accuracy, calculated for variable x when after growing the bth tree, the OOB samples are

passed down the tree and prediction accuracy is recorded. Once this prediction accuracy is recorded,

the values for variable x are randomly permuted in the OOB samples, and accuracy is computed again.

This decrease in accuracy serves as a measure of variable x in the Random Forest. Prediction accuracy

is related to Mean Squared Error (MSE) given that, for input x, the response is modeled by y = f(x)+ε,

and therefore the expected prediction error of an estimate f(x) = xT β is represented by [130]:

E(y − f(x))2 = σ2 + E(xT β − f(x))2 = σ2 +MSE(f(x)) (4.8)

The accuracy being measured by relative difference.

• Gini Index, calculated for k ∈ [1,K] input values to the classification, for N observations at node b

as [130]:

pbk =1

Nb

∑X

I(y = k) (4.9)

Ginim =∑k 6=k′

pbkp′bk =

K∑k=1

pbk(1− pbk) (4.10)

Where pbk is the proportion of k-class observations in node b.

For classification, the Gini index serves as a more accurate measure of variable importance than Decrease in

Accuracy, given the implementation of the Random Forest algorithm in the randomForest package in R [111].

Breiman [131] describes the measure of variable importance used in usual tree construction as:

“At every split one of them [tree] variables is used to form the split and there is a resulting decrease

in the gini [index]. The sum of all decreases in the forest due to a given variable, normalized by

the number of trees, forms the measure, [Mean Decrease in Gini Coefficient ]” [131]

4.3.2 Results

Figures B.32 and B.33 describe the Mean Decrease in Gini Coefficient across each set of clinical and

laboratory parameters, for all outcomes.

Code Blue

• For the outcome of code blue, the most predictive parameter is maximum estimated GFR, while the

least predictive is minimum eosinophil count.

• Accounting for directionality, for the outcome of code blue, the most predictive parameter is maximum

respiratory rate, while the least predictive is maximum basophil count.

Cardiac Arrest

• For the outcome of cardiac arrest, the most predictive parameter is maximum total carbon dioxide,

while the least predictive parameter is minimum eosinophil count.

• Accounting for directionality, for the outcome of cardiac arrest, the most predictive parameter is mini-

mum sodium (while maximum total carbon dioxide is four below), while the least predictive parameter

is minimum lymphocyte count.


Transfer to ICU

• For the outcome of transfer to the ICU, minimum respiratory rate (followed immediately by maximum

respiratory rate) is the most predictive parameter, while minimum basophil count is the least predictive

parameter.

• Accounting for directionality, for the outcome of transfer to the ICU, maximum respiratory rate (rather

than minimum respiratory rate) is the most predictive parameter, while maximum basophil count

(rather than minimum) is the least predictive parameter.

Death

• For the outcome of death, the parameter of urea is most predictive (maximum followed by minimum),

while the least predictive parameter is minimum basophil count.

• Accounting for directionality, for the outcome of death (followed immediately by maximum urea),

maximum age is the most predictive parameter, while the least predictive parameter is maximum

basophil count (rather than minimum).


• For the composite outcome of death, code blue, and transfer to the ICU, maximum respiratory rate

is the most predictive parameter, while basophil count (minimum followed by maximum) is the least

predictive parameter.

• Accounting for directionality, for the composite outcome, maximum respiratory rate is still the most

predictive parameter, while maximum basophil count (rather than minimum) is the least predictive.

In summary, a semblance of consistency between predictive parameters in both models are observed. From

this, we can generalize that those laboratory parameters exhibiting effective prediction for the classification

model (in particular age, estimated GFR and urea) are generally consistent with measures of metabolic

deterioration such as septic shock [132] leading to multi-organ-system failure, or are matched to the consensus

parameters for literature [5].

4.4 Empirical Copula

4.4.1 Methods

From Roger Nelsen’s An Introduction to Copulas [133], the copula is a “function that joins or couples

multivariate distribution functions to their one-dimensional marginal distribution functions”. Both Frees

and Kolev at al [134] and Beaudoin et al [135] describe the use of families of copulas for the analysis

of multivariate outcomes, in both actuarial and medical science. To this end, the selection of a Normal

Archimedian Copula, serves to illustrate the use of base model for Gaussian distribution-based analysis,

accounting for left censoring [135]. Although this does not cover the non-Gaussian space, nor deal with both

interval and right censoring, it serves as a starting point for potential future work, and an initial estimate of

marginal distribution assessment for our analysis purposes. The mathematically definition and description

of the use of copulas as a method of evaluating variable independence follows.


Where X = (X1, ..., Xd) is a vector with a continuous marginal cumulative distributions F1, ..., Fd, of

which the cumulative distribution of X can be expressed via [136]:

F ′X = C {F1(x1), ..., Fd(xd)} ,x ∈ R (4.11)

Where C is the copula, a d dimension cumulative distribution function with margins on the uniform space

[0, 1]d.

The empirical copula is an estimator of the unknown copula C, defined as the “empirical cumulative

distribution function computed from pseudo-observations Uij = nFj(Xij)/(n + 1)” [136], where Fj is the

empirical cumulative distribution function computed from X1j , ..., Xnj , and n/(n+ 1) ensures the boundary

conditions of [0, 1]d remain intact. The empirical copula can be represented by the equation [136]:

Cn(u) =1

n

n∑i=1

1(Ui ≤ u),u ∈ [0, 1]d (4.12)

Deheuvels [137] describes the mathematical method of decomposing the empirical copula process into a

“a finite number of a symptotically mutually independent sub-processes whose joint limiting distribution is

tractable under the hypothesis that a multivariate distribution is equal to the product of its margins.” [138]

As such, Genest et al [138] built upon this work to prove that linear rank statistics have the same distribution

in both the serial and parallel sets of of observations, therefore the multivariate distribution is equal to the

product of its margins for both serial and parallel observation sets. The test statistic of mutual independence

of the components of the set of observations X, can be represented by the equation [136]:

In =

∫[0,1]d

n

{Cn(u)−

d∏i=1

ui

}2

du (4.13)

In using a copula process to evaluate variable dependence, the correlogram, a standard graphical display

of variable correlation is represented via the dependogram [138]. This dependogram is generated through

the empirical copula process, and as such allows for an evaluation of variable dependence outside of the

statistical complexity of contour mapping and the unit space joint distribution. The dependogram displays

the magnitude of the test statistic interaction between the described variables and whether or not it passes

the threshold of dependence indicated by line and its length past the dot respectively.

4.4.2 Results

Figures B.28, B.29, B.30, and B.31 all illustrate variable interdependence for the five most predic-

tive variables for both logistic regression and random forest classification, as well as the modified set for

directionality.

Code Blue

• For logistic regression, overpowering dependence between maximum and minimum MCV, otherwise

ranked dependence between: minimum INR and minimum creatinine, minimum INR and minimum

MCV, minimum creatinine and minimum monocyte count and maximum mcv and monocyte count.

• For logistic regression accounting for directionality, overpowering dependence is observed between max-

imum and minimum MCV, ranked relative dependence between: maximum MCV and minimum RBC,

minimum systolic blood pressure and minimum RBC, maximum anion gap and minimum RBC and

maximum MCV and minimum systolic blood pressure.


• For random forest classification, ranked dependence is observed between: maximum estimated GFR

and minimum creatinine, minimum Hb and minimum RBC, both overpowering maximum estimated

GFR and maximum total carbon dioxide, and minimum creatinine and maximum total carbon dioxide.

• For random forest classification accounting for directionality, overpowering dependence is observed

between: minimum Hb and maximum RBC, further ranked dependence between: maximum respiratory

rate and minimum Hb, maximum RBC and maximum diastolic blood pressure, minimum systolic

blood pressure and minimum Hb, maximum respiratory rate and maximum diastolic blood pressure

and maximum respiratory rate and minimum systolic blood pressure.

Cardiac Arrest

• For logistic regression, overpowering dependence between maximum estimated GFR and minimum

potassium, further ranked dependence between: minimum INR and maximum estimated GFR, mini-

mum INR and minimum potassium, minimum MCH and minimum platelet count, minimum potassium

and minimum platelet count, minimum INR and minimum platelet count, and minimum INR and min-

imum MCH.

• For logistic regression accounting for directionality, overpowering dependence between maximum MCV

and minimum MCH, further ranked dependence between: maximum MCV and minimum RBC, mini-

mum RBC and minimum potassium, minimum potassium and maximum anion gap.

• For random forest classification, ranked dependence is observed between: minimum Hb and minimum

HCT, minimum RBC and minimum HCT, minimum Hb and minimum RBC, minimum Hb and RBC

and HCT, maximum total carbon dioxide and maximum estimated GFR.


between: minimum total carbon dioxide and maximum anion gap, further ranked dependence between:

minimum sodium and minimum RBC, minimum sodium and minimum Hb, and maximum diastolic

blood pressure and maximum total carbon dioxide.

Transfer to ICU

• For logistic regression, overpowering dependence between minimum and maximum respiratory rate,

further ranked dependence between: minimum MCH and maximum respiratory rate, minimum respi-

ratory rate and minimum diastolic blood pressure, minimum diastolic blood pressure and maximum

respiratory rate, minimum MCH and maximum respiratory rate, minimum respiratory rate and mini-

mum eosnophil count, and minimum respiratory rate and minimum MCH.

• For logistic regression accounting for directionality, overpowering dependence between minimum dias-

tolic blood pressure and minimum systolic blood pressure, further ranked dependence between: min-

imum respiratory rate and maximum total carbon dioxide, maximum sodium and maximum total

carbon dioxide, minimum diastolic blood pressure and maximum total carbon dioxide, minimum di-

astolic blood pressure and maximum sodium, maximum total carbon dioxide and minimum systolic

blood pressure, minimum diastolic blood pressure and minimum respiratory rate, minimum respiratory

rate and maximum sodium, and minimum respiratory rate and minimum systolic blood pressure.

• For random forest classification, overpowering dependence is observed between minimum diastolic

blood pressure and minimum systolic blood pressure. Further ranked dependence is observed between:


minimum and maximum respiratory rate, maximum respiratory rate and minimum total carbon diox-

ide, maximum respiratory rate and minimum diastolic blood pressure, minimum respiratory rate and

minimum diastolic blood pressure, minimum respiratory rate and minimum systolic blood pressure,

maximum respiratory rate and minimum systolic blood pressure, minimum respiratory rate and mini-

mum total carbon dioxide, minimum diastolic blood pressure and minimum total carbon dioxide and

minimum systolic blood pressure and minimum total carbon dioxide.


between: minimum systolic blood pressure and minimum diastolic blood pressure, further ranked de-

pendence between: minimum respiratory rate versus maximum respiratory rate, minimum respiratory

rate and minimum diastolic blood pressure, minimum respiratory rate and minimum diastolic blood

pressure, minimum diastolic blood pressure and maximum age, minimum respiratory rate and mini-

mum systolic blood pressure, minimum respiratory rate and maximum age, and maximum respiratory

rate and minimum systolic blood pressure.

Death

• For logistic regression, ranked order of strength of dependence between variables results in the following

list: minimum neutrophil count and minimum urea, minimum RDW and minimum urea, maximum

potassium and minimum urea, maximum potassium and minimum pulse, minimum neutrophil count

and minimum pulse, and minimum RDW and maximum potassium.

• For logistic regression accounting for directionality, the ranked order of strength of dependence between

variables results in the following list: minimum RDW and maximum platelet count, maximum age and

minimum pulse, and minimum neutrophil count and minimum pulse.

• For random forest classification, ranked dependence is observed between: maximum creatinine and

minimum estimated GFR, maximum urea and minimum estimated GFR, maximum urea and maximum

creatinine, maximum and minimum urea, minimum urea and minimum estimated GFR, maximum

creatinine and maximum INR, maximum INR and minimum estimated GFR, maximum urea and

maximum INR, and maximum urea, maximum creatinine and minimum estimated GFR.


between: maximum urea and maximum creatinine, further ranked dependence is observed between:

maximum creatinine and minimum total carbon dioxide, maximum urea and minimum total carbon

dioxide, maximum age and maximum urea, maximum age and maximum creatinine, and maximum

age and minimum neutrophil count.


• For logistic regression, overpowering dependence between minimum MCV and minimum MCH (calcu-

lated values both depending on hemoglobin), further ranked dependence between: minimum respiratory

rate and maximum dependence rate, minimum respiratory rate and minimum neutrophil count, and

maximum respiratory rate and minimum MCH.

• For logistic regression accounting for directionality, overpowering dependence between minimum neu-

trophil count and minimum white blood cell count, and further ranked dependence between minimum

respiratory rate and maximum platelet count and minimum respiratory rate and minimum neutrophil

count.


• For random forest classification, ranked dependence is observed between: maximum respiratory rate

and maximum fraction of inspired oxygen, maximum respiratory rate and minimum respiratory rate,

minimum total carbon dioxide and maximum urea, maximum fraction of inspired oxygen and minimum

respiratory rate, maximum respiratory rate and maximum urea, maximum fraction of inspired oxygen

and maximum urea, maximum respiratory rate and minimum total carbon dioxide, minimum total

carbon dioxide and minimum respiratory rate, and maximum urea and minimum respiratory rate.

• For random forest classification accounting for directionality, overpowering dependence is observed be-

tween: minimum total carbon dioxide and maximum anion gap, further ranked dependence is observed

between: minimum total carbon dioxide and maximum INR, maximum anion gap and maximum INR,

maximum respiratory rate and maximum anion gap, maximum respiratory rate and maximum platelets,

maximum respiratory rate and maximum INR, maximum respiratory rate and minimum total carbon

dioxide, maximum anion gap and maximum platelet count, and minimum total carbon dioxide and

maximum platelet count.

In summary, dependence between variables, confirms elements of expected bias:

• Calculated variables such as EstimatedGFR, which is derived from Creatinine, display their appropriate

relationship in the dependogram, as well as exhibit a strong dependence statistic. For example, the

dependence between EstimatedGFR.Max and Creatinine.Min [139].

• Dependence between the same variable represented at both ends of the variability spectrum, such as

EstimatedGFR.Max and EstimatedGFR.Min exhibit a strong dependence statistic.

Compared to other variable interdependence, the two points of bias above exhibit greater dependence statis-

tics. This therefore points to the impact of calculated results potentially confounding the predictive model,

yet their ability to exhibit clinical or physiological relevance is of import, and therefore further impact

assessment is necessary.

4.5 Summary of Results and Future Work

From both classification and regression modeling, respiratory rate, the consensus predictor from litera-

ture [5] is verified as being predictive of ICU transfer, and is consistent with known directionality. Those

laboratory, and extra-EWS parameters such as age, urea, estimated GFR, INR, MCV, RDW and MCH are

consistent with the prediction model of multi-organ system failure as is described by the frailty decompen-

sation models of Rockwood et al [140, 141, 142, 143], and the pathogenic model of septic shock presented

by Parrillo [132] leading to multi-organ-system failure. Both of these decompensation models fit within

the framework of the patient decompensation event, as they both are components of the Jones et al [50]

descriptive model. Furthermore, the dependogram analysis confirms that hematology and urea are linked,

and although estimated GFR does link to the hematological parameters, it exhibits a higher dependence

with such variables as potassium and creatinine.

Clinically, the information provided by this modeling is not novel, however, this modeling does show

that, for binary outcomes, the predictive impact of both hematology and metabolic function tests is superior

to conventional parameters. Furthermore, the regression and classification analyses that do not account for

directionality fail to account for “clinically perceived known behaviour” of both maximum and minimum

values of variables within physiological norms observed in the clinical environment. This lack of perceived

clinical relevance does not impact the interpretation of the general results presented in summary, however,


attention should be paid to the clinical narrative in iterating the deployment of this information, and use of

the signed value of coefficients for further model construction.

Physiologically, the prediction of systemic failure for clinical outcomes makes sense within the realm of

a decompensation event. For code blue/cardiac arrest, measures of oxygen activity (total CO2) and ion

channel activity related to the electrical event in question (sodium) or circulatory system (INR, MCV),

however, the acute timeliness of these laboratory values in predicting events depend on physiological circuits

that have been compromised leading up to the formal event in question. As such, HCT, RBC, sodium, and

lymphocyte count should be taken as relatively strong predictive parameters, rather than finite. Generally,

more consistent laboratory testing appears to be beneficial based on the predictive ability of laboratory

tests, however, further analysis as to its cost effectiveness would be required to render a final implementation

decision.

Chapter 5

System Analysis


As a method of evaluating the existing characteristics of the RRT via the EWS, Receiver Operating

Curves (ROC) curves allow for the identification of test performance. ROC curves are a standard method

of predicting a diagnostic test accuracy [144, 145, 146]. In this case, the test being the ability to detect

potential Patient Care and Safety Compromising Events, isolated to the failure-to-rescue event, represented

via the pair of: clinically significant outcome within a time-to-event threshold, and the Early Warning Score

meeting a defined test threshold. This threshold has been expressed by multiple sources [13, 49], the size of

which analysis has not conclusively determined [82].

Supporting the ROC analysis are entropy estimators, which quantitatively evaluate the calculated EWSs

across the dataset to gauge their variability, and impact as on serving as predictive parameters for the ROC

curves. Alongside entropy estimators, the Hosmer-Lemeshow Goodness of Fit characteristic serves to identify

the performance of the base data supporting the ROC curves, and its ability to predict outcomes as function

of variability and statistical coherence.

5.2 Receiver Operating Characteristic Curves

5.2.1 Methods

For the time-to-event threshold (dT Threshold) of 4, 6, 8, 12, 16 and 24 hours, ROC curves were generated

at EWS thresholds according to the pROC package in R [117]. These graphs describe the predictive strength

of each EWS against outcomes within a time-to-event threshold. The time-to-event threshold selected allows

for the identification of a potential PCSE, as EWSs outside the threshold, yet correctly meeting the time to

event threshold will contribute negatively to the predictive value.

Sensitivity and specificity are calculated via the following equations:

Sensitivity =# of True Positives

# True Positives+ # False Negatives(5.1)

Specificity =# of True Negatives

# True Negatives+ # False Positives(5.2)

(5.3)

Given the above qualifications, the following table describes the positive and negative test characteristics

for the ROC curves:

38

Chapter 5. System Analysis 39

Positive (+) Negative (–)

True (T) Outcome within the time-to-

event threshold, EWS above the

threshold value

No outcome within the time-to-

event threshold, EWS below the

threshold value

False (F) No outcome within time-to-event

threshold, EWS above the thresh-

old value

Outcome within the time-to-

event threshold, EWS below

threshold

Table 5.1: Positive and Negative Test Characteristic Descriptions

for Receiver Operating Characteristic Curves

Using the R package pROC [117], Curves are generated via the automatic selection of a threshold, and

the generated of both case and control data (corresponding to believed truths and falsehoods Table 5.1, and

for that threshold the sensitivity and specificity are calculated. For the generated list of thresholds, a curve

is constructed on the sensitivity and specificity axes.

5.2.2 Results

Figures B.18, B.20, B.22, B.24, and B.26 display ROC curves for each Early Warning Score across

time-to-event thresholds. Figures B.19, B.21, B.23, B.25, B.27 display ROC curves for each time-to-event

threshold across Early Warning Scores. Table B.9 displays the summary Area Under the Curve (AUC) for

each ROC displayed in the aforementioned Figures. The tables following describe AUC ordered EWS list of

the ROC curves generated per time-to-event threshold in each header.

Time-to-Event Threshold: 4 Hours

Outcome 1st Rank EWS

(AUC)

2nd Rank EWS

(AUC)

3rd Rank EWS

(AUC)

4th Rank EWS

(AUC)

5th Rank EWS

(AUC)

Code

Blue

MEWS (0.806) CDF (0.792) ViEWS (0.792) NEWS (0.790) CART (0.636)

Death CDF (0.759) MEWS (0.744) NEWS (0.743) ViEWS (0.734) CART (0.672)

ICU

Trans-

fer

MEWS (0.889) ViEWS (0.878) NEWS (0.869) CDF (0.794) CART (0.633)

Composite MEWS (0.806) NEWS (0.799) ViEWS (0.796) CDF (0.773) CART (0.657)

Cardiac

Arrest




(AUC)

2nd Rank EWS

(AUC)

3rd Rank EWS

(AUC)

4th Rank EWS

(AUC)

5th Rank EWS

(AUC)

Code

Blue




ICU

Trans-

fer


Composite MEWS (0.825) NEWS (0.815) ViEWS (0.813) CDF (0.794) CART (0.657)

Cardiac

Arrest

MEWS (0.809) CDF (0.794) VIEWS (0.790) NEWS (0.786) CART (0.645)



(AUC)

2nd Rank EWS

(AUC)

3rd Rank EWS

(AUC)

4th Rank EWS

(AUC)

5th Rank EWS

(AUC)

Code

Blue

MEWS (0.825) ViEWS (0.805) NEWS (0.802) CDF (0.800) CDF (0.630)


ICU

Trans-

fer


Composite MEWS (0.837) ViEWS (0.825) NEWS (0.825) CDF (0.811) CART (0.663)

Cardiac

Arrest




(AUC)

2nd Rank EWS

(AUC)

3rd Rank EWS

(AUC)

4th Rank EWS

(AUC)

5th Rank EWS

(AUC)

Code

Blue


Death MEWS (0.837) CDF (0.829) NEWS (0.827) ViEWS (0.824) CART (0.693)

ICU

Trans-

fer



Cardiac

Arrest

MEWS (0.854) ViEWS (0.835) CDF (0.829) NEWS (0.826) CART (0.646)



(AUC)

2nd Rank EWS

(AUC)

3rd Rank EWS

(AUC)

4th Rank EWS

(AUC)

5th Rank EWS

(AUC)

Code

Blue

MEWS (0.871) ViEWS (0.854) NEWS (0.845) CDF (0.838) CDF (0.618)

Death MEWS (0.855) CDF (0.843) NEWS (0.843) ViEWS (0.842) CART (0.699)

ICU

Trans-

fer




Cardiac

Arrest

MEWS (0.870) ViEWS (0.855) CDF (0.850) NEWS (0.844) CDF (0.630)



(AUC)

2nd Rank EWS

(AUC)

3rd Rank EWS

(AUC)

4th Rank EWS

(AUC)

5th Rank EWS

(AUC)

Code

Blue


Death MEWS (0.875) ViEWS (0.862) NEWS (0.861) CDF (0.859) CART (0.703)

ICU

Trans-

fer



Cardiac

Arrest


In summary, from the tables and figures mentioned above, the MEWS score consistently exhibits the best

Area Under the Curve (AUC) than the other scores. The ViEWS and NEWS scores, given their derivation

base perform similarly to each other, both graphically and from their AUC results. Additionally, between

ViEWS, NEWS, CDF and MEWS, the overlap of the AUC confidence intervals reveals that although the

scores may exhibit a hierarchy, there is no highly significant difference between the scores’ performance.

Furthermore, where NEWS, ViEWS and CDF vacillate between second to fourth place position in the

score hierarchy, the CART score exhibits the worst AUC than the other scores. The cause of this reduced

effectiveness cannot be attributed to the construction of the ROC curves, as the same steps were taken to

generate the CART score as with the others, and the thresholds were generated automatically from the range

of values for each individual score, as not to create user selection bias. A possible explanation is that the

CART score was designed for solely the prediction of Cardiac Arrest, and validation has not been proven

in a follow-up study. Examining the time-to-event threshold stratified ROC curves, we see that the larger

the threshold, the more likely the curve will be able to predict a true positive event (as can be logically

expected), and therefore exhibits a greater AUC.

5.3 Entropy Estimators

5.3.1 Methods

In a review of variability analysis techniques for clinical applications, Bravi et al [147] describe the use of

the information domain for time series analysis. This domain serves to catalogue the degree of complexity

and chaos associated with the elements in an ordered time series.

Approximate entropy serves as a measure of entropy to discern the changing complexity across the

dataset [148]. It is calculated via the:

“negative natural logarithm of the conditional probability that a dataset of length N , having

repeated itself for m samples within a tolerance r, will repeat itself again for one extra sample.

A window of length m is run along the signal to generate a set of data vectors of length m. One

then computes the number of times that the Euclidean distance between all pairs of these vectors


is less than a threshold r. This is repeated for windows of length m+ 1, and the logarithm of the

ratio of these two numbers is taken.” [149]

Sample Entropy also serves the same purpose as approximate entropy, yet is a more accurate calcu-

lator [150]. This accuracy is due to sample entropy’s exclusion of the approximate entropy vector self-

comparison in its calculation. According to Bravi et al [147], this both avoids the bias these self-matches

introduce, but also means that sample entropy should always be preferred to approximate entropy.

Fuzzy entropy seeks to remove the sensitivity introduced by the manual selection of the tolerance r [147].

Rather than identifying the euclidean distance between windows via a binary tolerance, the fuzzy K-means

algorithm [151] is used to evaluate the distance. This serves to avoid the tolerance causing sensitivity

discontinuities.

Alongside the approximate, sample and fuzzy estimators, the classic measure of entropy also serves as a

useful tool. Shannon entropy, is described as:

“the sum of the relative frequencies weighted by the logarithm of the inverse of the relative

frequencies (i.e. when the frequency is low, the weight is high, and vice versa).” [147]

Again, as a measure of system complexity, it serves to reinforce the variability measures of approximate,

sample and fuzzy entropy. Two estimators are used: empirical and Chao-Shen [152].

5.3.2 Results

Table B.7 details the numerical values of entropy calculated for Early Warning Score values. The tables

following describe the table sorted by sample entropy selected by subset of outcome type identified in the

header. Figures B.14 and B.13 group the values presented in the table for convenience. To provide more

context for this analysis, Table 5.8 describes the distribution of Early Warning Scores across encounters.

Early

Warning

Score

Mean Standard

Deviation

Mean Absolute

Deviation

Absolute

Maximum

Value

Absolute

Minimum

Value

CART 12.436 9.986 6.252 57 0

MEWS 1.902 2.088 1.563 12 0

NEWS 2.437 2.647 2.352 13 0

ViEWS 2.099 2.387 1.604 13 0

CDF 11.757 6.841 2.597 48.568 -4.887

Table 5.8: Distribution of Early Warning Scores Across Encounters

Cardiac Arrest Risk Triage Score

Score Variability Approx. Sample Fuzzy Shannon ChaoShen

CART Min 0.76 0.75 0.37 7.42 7.44

CART Mean 1.87 0.80 0.61 7.49 7.55

CART Max 1.88 0.81 0.58 7.54 7.54

CART Median 1.01 0.90 0.49 7.40 7.42

VitalPAC™Early Warning Score



ViEWS Min 0.06 0.06 0.11 3.96 4.72

ViEWS Mean 0.50 0.70 0.60 7.52 9.90

ViEWS Max 1.76 0.86 0.55 7.57 7.64

ViEWS Median 1.03 1.03 0.55 7.31 8.07

National Early Warning Score


NEWS Min 0.09 0.09 0.14 4.36 5.01

NEWS Mean 0.63 0.71 0.61 7.52 9.22

NEWS Max 1.81 0.92 0.56 7.58 7.62

NEWS Median 1.17 1.20 0.51 7.37 8.01

Modified Early Warning Score


MEWS Min 0.04 0.05 0.10 3.66 4.41

MEWS Mean 0.37 0.68 0.61 7.54 10.19

MEWS Max 1.68 0.74 0.60 7.58 7.68

MEWS Median 0.83 0.83 0.42 7.41 8.32

Cuthbertson Descriminant Functions


CDF Max 1.19 0.24 0.48 7.61 7.61

CDF Mean 0.79 0.63 0.62 7.63 7.63

CDF Min 1.01 0.70 0.35 10.58 6.97

CDF Median 0.81 0.76 0.59 7.64 7.64

In summary, when examining Figure B.13, the pattern of entropy between Max, Min, Median, and Mean

values is consistent and replicated across Early Warning Scores. Due to the CDF score values ranging

between positive and negative, and the minimum value breaking the trend due to its higher potential of

exhibiting a change in sign, this behaviour is not untoward.

The calculation of entropy, however, is not without fault. In particular, the selection of function param-

eters associated with the calculation of entropy are described by the statement:

“There exist neither objective criteria nor literature guidelines for the choice of these parame-

ters.” [149]

Bravi et al [149] subsequently describe that a more accurate measure of the selection of parameters is to

optimizing clinically significant features of the analysis. This potential bias in parameter selection results

in the resulting entropy not representing the underlying qualities of the data, rather a manufactured and

conditioned response. For this reason the parameters used to calculate entropy above have been selected to

optimize speed of analysis, rather than clinical significance. Fuzzy entropy, in particular, suffers from the

lack of optimization of the K-means parameters. As a result, sample entropy is the preferred selector for


function in analysis, as it bests approximate entropy, and is more particular to the observed variance than

standard Shannon’s entropy.

For sample entropy, MEWS exhibits the absolute minimum entropy, suggesting good sensitivity perfor-

mance. CART exhibits the maximum entropy for the minimum values set, despite the presence of CDF,

which exhibits both positive and negative values and takes the second place position. This display further

describes the potential results of the ROC curves. When automatically thresholding values to provide sensi-

tivity and specificity estimates, the variability of the parameter being thresholded as well as its range impacts

the sensitivity, as is noted above in the positive case of MEWS.

5.4 Hosmer Lemeshow Goodness of Fit

5.4.1 Methods

The Hosmer Lemeshow C-statistic [121] describes the goodness of fit for logistic models of Early Warning

Score-based predictions of clinical outcomes based on the ROC curve case and control data, as a means of

describing the efficacy of the ROC curves. Hosmer et al [153] suggest that the Hosmer Lemeshow test is,

compared against other known goodness-of-fit tests, comparably effective, yet as the sample size increases,

the power of the test to detect departures from linearity decreases. Compared to Pearson’s χ2, a variety of

other named tests, the Hosmer Lemeshow Test demonstrates better power than others to detect asymmetric

link functions rendered by logistic regression, and subsequently deal with variability. [153]

The χ2 test statistic for the Hosmer-Lemeshow Test can be defined as the sum of squares of the Pearson

residual [121]:

χ2 =

J∑j=1

(yj −mjπj)2

mjπj(1− πj)(5.4)

Where yj , the fitted value for the jth group can be described by [121]:

yj = mjπj = mjeg(xj)

1 + eg(xj)(5.5)

Where g(xj) is the estimated logit of the group j. The C-statistic is calculated from the Pearson chi-squared

statistic [153], and can be defined as [121]:

C =

J∑j=1

(Oj −Njπj)2

Njπj(1− πj)(5.6)

Where O is the number of observed events, Nπ number of expected events, N number of total observations,

π predicted risk probability for the jth risk group. Therefore, a reliable evaluation statistic, given that

πj(1− πj) = 1 can be written grammatically as:

test statistic =∑ (observed value− expected value)2

expectedvalue≈ χ2 ≈ C (5.7)

5.4.2 Results

Table 6.11 displays the Hosmer-Lemeshow χ2 test statistic for each ROC curve. The tables following

describe the subset for each time-to-event threshold identified in the header.



Outcome dT Threshold Early Warning Score Chi-Squared Test Statistic

CardiacArrest (0,4] NEWS 0

CodeBlue (0,4] ViEWS 0.013

CCRTActivation (0,4] NEWS 0.027

Death (0,4] CART 0.037

CodeBlue (0,4] NEWS 0.078

CardiacArrest (0,4] ViEWS 0.085

CodeBlue (0,4] CART 0.089

CardiacArrest (0,4] CART 0.125

CCRTActivation (0,4] CART 0.302

CCRTActivation (0,4] CDF 0.371

CCRTActivation (0,4] ViEWS 0.455

CCRTActivation (0,4] MEWS 0.484

CardiacArrest (0,4] MEWS 3.418

Composite (0,4] NEWS 5.688

ICUTransfer (0,4] MEWS 6.942

CodeBlue (0,4] MEWS 7.427

CodeBlue (0,4] CDF 10.308

Composite (0,4] ViEWS 11.797

ICUTransfer (0,4] ViEWS 12.852

ICUTransfer (0,4] NEWS 13.782

CardiacArrest (0,4] CDF 14.178

ICUTransfer (0,4] CART 23.806

ICUTransfer (0,4] CDF 24.218

Composite (0,4] MEWS 27.309

Death (0,4] ViEWS 34.408

Death (0,4] MEWS 47.554

Composite (0,4] CART 49.256

Death (0,4] CDF 52.146

Death (0,4] NEWS 56.369

Composite (0,4] CDF 274.543



CardiacArrest (0,6] NEWS 0.003








Death (0,6] CART 0.378








Death (0,6] NEWS 5.35










Death (0,6] MEWS 52.386



Death (0,6] CDF 247.197




CardiacArrest (0,8] NEWS 0
















Death (0,8] NEWS 4.581








Death (0,8] CART 25.598


Death (0,8] MEWS 50.719




Death (0,8] CDF 383.904
















Death (0,12] NEWS 3.347









Death (0,12] MEWS 13.941


Death (0,12] CART 55.728





Death (0,12] CDF 389.679












Death (0,16] NEWS 2.539













Death (0,16] MEWS 10.285



Death (0,16] CART 49.171





Death (0,16] CDF 357.05










Death (0,24] MEWS 5.243









Death (0,24] NEWS 11.799








Death (0,24] CART 57.263






Death (0,24] CDF 311.222

In summary, significance in this case means that the model prediction is not statistically dissimilar than

the observation, as we are comparing the model against the observed values, using the null hypothesis. For the

test, significance describes that the models accurately exhibit predicted behaviour. CDF offers significantly

higher χ2 test statistics than the other scores, and for the statistics above 50, the majority are generated

via either CDF or CART. This suggests that there are larger differences between observed and expected

values within the model, again highlighting the range of values in the dataset, and supporting the entropy

estimator conclusions.

Examining the χ2 test statistic values associated with time-to-event threshold shows that the smaller

the threshold, the smaller the average χ2 test statistic. This produces the conclusion that the lower the

time-to-event threshold, the smaller the differences between the observed and expected values, therefore the

tighter the range of the fit of the data, therefore confirming the effect found in the trend that the greater

the time-to-event threshold will exhibit a greater AUC.


In summary, the Receiver Operating Characteristic curves show that the MEWS score is the most pre-

dictive based its Area Under the Curve. Additionally, the ViEWS, NEWS and CDF scores all exhibit strong

predictive strength, although the confidence intervals of these four scores overlap, casting into doubt the

significance of their difference in predictive strength. The CART score exhibits the lowest predictive ability,

however, when compared alongside the CDF score, both in goodness of fit and via entropy estimators, it

exhibits a high degree of variability, a potential confounder of the result, however, further analysis may prove

beneficial. Clinically, this suggests that the standard of care provided for by the MEWS score as adopted

by the NHS in the United Kingdom, and via automated systems such as Cerner’s Lighthouse module, is not

significantly better than that promoted by the Royal College of Physicians [24] and Prytherch et al [23].

Chapter 6

Output Analysis


In order to establish a measurement of the impact of co-variates on outcome modeling, a time-to-event

approach over the horizon of patient stay allows for the quantity of data and its temporal sequence to be

accounted for. Furthermore, the use of hazard modeling also allows for an examination of imputation and

censoring, significant components of the temporal features of the data. This approach has been used in

literature, with Boniatti et al [154] investigating the temporal features associated with response to RRT

calls and the subsequent associated patient outcomes.

Kaplan-Meier curves serve to establish a baseline measure of the outcome function, isolated by cause-

specific parameters. Cox-Proportional Hazards expand this baseline model, accounting for censoring and the

impact of co-variates in a parametric manner. The discriminative ability of these models will be evaluated via

the use of Harrel’s c or Somer’s D, in order to describe a measure of their predictive efficacy. In addition to

Cox-Proportional Hazards, Gray’s Method of Comparative Risk will be used to evolve Cumulative Incidence

functions, and describe the temporal characteristics of parameters, along with their sub-distributions of risk.

The core component of hazard modeling, the survival function, S(t) represents the probability of failure

occurring at or after time t, via the following relationship between cumulative distribution function F (t) and

probability density function f(t):

S(t) = P (T ≥ t) = 1− F (t) = 1−∫ t

0

f(u)du

The hazard function, a related function is defined by:

h(t) = lim∆t→0

p(t ≤ T < t+ ∆t | T ≥ t)∆t

(6.1)

The general relationship between the hazard and survival functions is defined by:

h(t) =f(t)

S(t)(6.2)

h(t) = − ∂

∂t{ln(S(t))} (6.3)

S(t) = e−∫ t0h(u)du (6.4)

50

Chapter 6. Output Analysis 51

Where the cumulative hazard function H(t) is defined as:

H(t) = −ln(S(t)) (6.5)

Building on these base definitions, the standard non-parametric method of estimating survival and hazard

functions from observed and censored data is the Kaplan-Meier estimator [155], mathematically termed as a

right-censored maximum-likelihood method product-limit estimator. To identify and response to co-variates

which influence the survival curve of interest, proportional hazards and cumulative risk have been employed

to answer the research question. The mechanics of the aforementioned analysis are described in Survival

Analysis: A Self-Learning Text [156], and reproduced in summary here.

6.2 Kaplan-Meir Plots

6.2.1 Methods

Building the Kaplan-Meir estimators via the computation at a discrete failure time of the survival function

uses the following equation [102]:

S(t− 1) =

t−1∏i=1

p(T > i | T ≥ i) (6.6)

This results in a piecewise constant function.The solutions to the Kaplan-Meier curves for each variable

associated with each outcome serve as the basis for the Cox-Proportional Hazard (CPH) models.

6.3 Cox-Proportional Hazards

6.3.1 Methods

In order to take into account censoring and imputation, transitioning from the standard survival model

to the proportional hazard model, the baseline hazard function is described as [102]:

hx(y) = h0(y)p(x) (6.7)

Where h0 is the baseline hazard at p(x) = 1, solving for p creates the relationship between the two hazards.

For the purpose of this analysis, the baseline hazard is calculated via the Kaplan-Meier estimator. Defining

p as [102]:

p(x) = eβT x (6.8)

Solving for the survival function using the base definition, we can arrive at the baseline hazard raised to

p(x) [102]:

Sx(y) = (S0(y))p(x) (6.9)

Cox [157] notes that the weights for the explanatory variables β can be estimated independently from the

baseline hazard function, and therefore the model can be described as semi-parametric. The likelihood of β

can be expressed as [102]:

L(β) =

k∏j=1

eβT xj∑

l∈Rj eβT xl

(6.10)

Where the product terms represent the probability of outcome based on observations xj at time yj . Using

maximum likelihood estimation, optimal β values can be obtained.


6.3.2 Results

Solving the Cox-Proportional Hazard model, the predictive impact of each variable to the model is

quantified by the p-Value, which describes for the model the fit of the variable to the hypothesis test statistic.

Therefore, as is consistent with frequentist statistics, the lower the p-Value, the better the fit of the variable

to the hypothesis.

Other than the p-Value, the Hazard Ratio (HR) serves as the standard measure of effect of the Cox-

Proportional Hazards model. In this case, it is calculated via the regression coefficients associated with

each variable (β), via the formula eβ [156]. As with the odds ratio, the HR of 1 describes a null value for

the relationship between the instantaneous event rate of outcome versus null-outcome for the variable in

question. The higher the ratio, the higher the hazard of the outcome, and therefore the subsequent degree

of the variable on that outcome’s occurrence. A HR below 1 notes that the event is less likely to occur due

to the variable.

Figures B.16 and B.15 graphically display the results of the Cox-Proportional Hazards modeling for

each outcome in question, via forest plots for hazard ratios and dotcharts for p-values. The tables following

summarize the raw output from R of the HRs per variable per outcome.

Code Blue

Parameter Hazard Ratio 95% CI Parameter Range p-Value

INR 1.38 1.3-1.47 [0.9,7.3] <0.0001

Temperature 0.87 0.82-0.93 [3.4,97] <0.0001

DiastolicBP 0.90 0.84-0.95 [0,850] 0.0005

SystolicBP 0.90 0.84-0.95 [0,209] 0.0005

VSFiO2 0.90 0.84-0.95 [0.21,99] 0.0006

RespiratoryRate 0.90 0.85-0.96 [0,288] 0.0009

Urea 1.10 1.04-1.17 [0.8,56.1] 0.0021

Pulse 0.92 0.86-0.97 [1,888] 0.0053

Baso# 1.07 1-1.13 [0,3.6] 0.0401

Eos# 1.07 1-1.13 [0,4.2] 0.0401

Lymp# 1.07 1-1.13 [0.2,6.9] 0.0401

Mono# 1.07 1-1.13 [0.1,3.2] 0.0401

Neut# 1.07 1-1.13 [0.6,68.3] 0.0401

Platelets 1.05 0.99-1.12 [15,880] 0.0987

MPV 1.05 0.99-1.12 [6,16.5] 0.0992

WBC 1.05 0.99-1.12 [0.9,72.2] 0.0992

Hb 1.05 0.99-1.12 [36,196] 0.1053

MCH 1.05 0.99-1.12 [19.2,38] 0.1053

MCHC 1.05 0.99-1.12 [273,369] 0.1053

MCV 1.05 0.99-1.12 [65.6,117.3] 0.1053

RBC 1.05 0.99-1.12 [1.07,6.47] 0.1053

RDW 1.05 0.99-1.12 [12.5,25.4] 0.1053

HCT 1.05 0.99-1.11 [0.118,0.586] 0.1278

EstimatedGFR 1.05 0.99-1.11 [5,470] 0.1279

Creatinine 1.02 0.96-1.09 [18,857] 0.4396

Potassium 1.00 0.94-1.07 [2.2,8.4] 0.8908

Chloride 1.00 0.94-1.06 [78,128] 0.9967


Sodium 1.00 0.94-1.06 [115,161] 0.9967

TotalCO2 1.00 0.94-1.06 [5,50] 0.9982

Cardiac Arrest


INR 1.38 1.3-1.47 [0.9,7.3] <0.0001

Temperature 0.87 0.82-0.93 [3.4,97] <0.0001

DiastolicBP 0.90 0.84-0.95 [0,850] 0.0005

SystolicBP 0.90 0.84-0.95 [0,209] 0.0005

VSFiO2 0.90 0.84-0.95 [0.21,99] 0.0006


Urea 1.10 1.04-1.17 [0.8,56.1] 0.0021

Pulse 0.92 0.86-0.97 [1,888] 0.0053

Baso# 1.07 1-1.13 [0,3.6] 0.0401

Eos# 1.07 1-1.13 [0,4.2] 0.0401

Lymp# 1.07 1-1.13 [0.2,6.9] 0.0401

Mono# 1.07 1-1.13 [0.1,3.2] 0.0401

Neut# 1.07 1-1.13 [0.6,68.3] 0.0401

Platelets 1.05 0.99-1.12 [15,880] 0.0987

MPV 1.05 0.99-1.12 [6,16.5] 0.0992

WBC 1.05 0.99-1.12 [0.9,72.2] 0.0992

Hb 1.05 0.99-1.12 [36,196] 0.1053

MCH 1.05 0.99-1.12 [19.2,38] 0.1053

MCHC 1.05 0.99-1.12 [273,369] 0.1053

MCV 1.05 0.99-1.12 [65.6,117.3] 0.1053

RBC 1.05 0.99-1.12 [1.07,6.47] 0.1053

RDW 1.05 0.99-1.12 [12.5,25.4] 0.1053

HCT 1.05 0.99-1.11 [0.118,0.586] 0.1278

EstimatedGFR 1.05 0.99-1.11 [5,470] 0.1279

Creatinine 1.02 0.96-1.09 [18,857] 0.4396

Potassium 1.00 0.94-1.07 [2.2,8.4] 0.8908

Chloride 1.00 0.94-1.06 [78,128] 0.9967

Sodium 1.00 0.94-1.06 [115,161] 0.9967

TotalCO2 1.00 0.94-1.06 [5,50] 0.9982

Death


INR 1.38 1.3-1.47 [0.9,7.3] <0.0001

Temperature 0.87 0.82-0.93 [3.4,97] <0.0001

DiastolicBP 0.90 0.84-0.95 [0,850] 0.0005

SystolicBP 0.90 0.84-0.95 [0,209] 0.0005

VSFiO2 0.90 0.84-0.95 [0.21,99] 0.0006



Urea 1.10 1.04-1.17 [0.8,56.1] 0.0021

Pulse 0.92 0.86-0.97 [1,888] 0.0053

Baso# 1.07 1-1.13 [0,3.6] 0.0401

Eos# 1.07 1-1.13 [0,4.2] 0.0401

Lymp# 1.07 1-1.13 [0.2,6.9] 0.0401

Mono# 1.07 1-1.13 [0.1,3.2] 0.0401

Neut# 1.07 1-1.13 [0.6,68.3] 0.0401

Platelets 1.05 0.99-1.12 [15,880] 0.0987

MPV 1.05 0.99-1.12 [6,16.5] 0.0992

WBC 1.05 0.99-1.12 [0.9,72.2] 0.0992

Hb 1.05 0.99-1.12 [36,196] 0.1053

MCH 1.05 0.99-1.12 [19.2,38] 0.1053

MCHC 1.05 0.99-1.12 [273,369] 0.1053

MCV 1.05 0.99-1.12 [65.6,117.3] 0.1053

RBC 1.05 0.99-1.12 [1.07,6.47] 0.1053

RDW 1.05 0.99-1.12 [12.5,25.4] 0.1053

HCT 1.05 0.99-1.11 [0.118,0.586] 0.1278

EstimatedGFR 1.05 0.99-1.11 [5,470] 0.1279

Creatinine 1.02 0.96-1.09 [18,857] 0.4396

Potassium 1.00 0.94-1.07 [2.2,8.4] 0.8908

Chloride 1.00 0.94-1.06 [78,128] 0.9967

Sodium 1.00 0.94-1.06 [115,161] 0.9967

TotalCO2 1.00 0.94-1.06 [5,50] 0.9982

Transfer to ICU


INR 1.38 1.3-1.47 [0.9,7.3] <0.0001

Temperature 0.87 0.82-0.93 [3.4,97] <0.0001

DiastolicBP 0.90 0.84-0.95 [0,850] 0.0005

SystolicBP 0.90 0.84-0.95 [0,209] 0.0005

VSFiO2 0.90 0.84-0.95 [0.21,99] 0.0006


Urea 1.10 1.04-1.17 [0.8,56.1] 0.0021

Pulse 0.92 0.86-0.97 [1,888] 0.0053

Baso# 1.07 1-1.13 [0,3.6] 0.0401

Eos# 1.07 1-1.13 [0,4.2] 0.0401

Lymp# 1.07 1-1.13 [0.2,6.9] 0.0401

Mono# 1.07 1-1.13 [0.1,3.2] 0.0401

Neut# 1.07 1-1.13 [0.6,68.3] 0.0401

Platelets 1.05 0.99-1.12 [15,880] 0.0987

MPV 1.05 0.99-1.12 [6,16.5] 0.0992

WBC 1.05 0.99-1.12 [0.9,72.2] 0.0992

Hb 1.05 0.99-1.12 [36,196] 0.1053

MCH 1.05 0.99-1.12 [19.2,38] 0.1053

MCHC 1.05 0.99-1.12 [273,369] 0.1053


MCV 1.05 0.99-1.12 [65.6,117.3] 0.1053

RBC 1.05 0.99-1.12 [1.07,6.47] 0.1053

RDW 1.05 0.99-1.12 [12.5,25.4] 0.1053

HCT 1.05 0.99-1.11 [0.118,0.586] 0.1278

EstimatedGFR 1.05 0.99-1.11 [5,470] 0.1279

Creatinine 1.02 0.96-1.09 [18,857] 0.4396

Potassium 1.00 0.94-1.07 [2.2,8.4] 0.8908

Chloride 1.00 0.94-1.06 [78,128] 0.9967

Sodium 1.00 0.94-1.06 [115,161] 0.9967

TotalCO2 1.00 0.94-1.06 [5,50] 0.9982



INR 1.38 1.3-1.47 [0.9,7.3] <0.0001

Temperature 0.87 0.82-0.93 [3.4,97] <0.0001

DiastolicBP 0.90 0.84-0.95 [0,850] 0.0005

SystolicBP 0.90 0.84-0.95 [0,209] 0.0005

VSFiO2 0.90 0.84-0.95 [0.21,99] 0.0006


Urea 1.10 1.04-1.17 [0.8,56.1] 0.0021

Pulse 0.92 0.86-0.97 [1,888] 0.0053

Baso# 1.07 1-1.13 [0,3.6] 0.0401

Eos# 1.07 1-1.13 [0,4.2] 0.0401

Lymp# 1.07 1-1.13 [0.2,6.9] 0.0401

Mono# 1.07 1-1.13 [0.1,3.2] 0.0401

Neut# 1.07 1-1.13 [0.6,68.3] 0.0401

Platelets 1.05 0.99-1.12 [15,880] 0.0987

MPV 1.05 0.99-1.12 [6,16.5] 0.0992

WBC 1.05 0.99-1.12 [0.9,72.2] 0.0992

Hb 1.05 0.99-1.12 [36,196] 0.1053

MCH 1.05 0.99-1.12 [19.2,38] 0.1053

MCHC 1.05 0.99-1.12 [273,369] 0.1053

MCV 1.05 0.99-1.12 [65.6,117.3] 0.1053

RBC 1.05 0.99-1.12 [1.07,6.47] 0.1053

RDW 1.05 0.99-1.12 [12.5,25.4] 0.1053

HCT 1.05 0.99-1.11 [0.118,0.586] 0.1278

EstimatedGFR 1.05 0.99-1.11 [5,470] 0.1279

Creatinine 1.02 0.96-1.09 [18,857] 0.4396

Potassium 1.00 0.94-1.07 [2.2,8.4] 0.8908

Chloride 1.00 0.94-1.06 [78,128] 0.9967

Sodium 1.00 0.94-1.06 [115,161] 0.9967

TotalCO2 1.00 0.94-1.06 [5,50] 0.9982


The p-values of the Cox-Proportional Hazards identify that INR is the most predictive parameter in all

models, with the outcomes of cardiac arrest and code blue being the least predictive overall respectively. The

quadruple of total carbon dioxide, sodium, potassium and chloride interchangeably take up the four least

predictive parameters, with an event split between chloride and total carbon dioxide for least predictive.

Estimated GFR and creatinine are more predictive than blood pressure for code blue and cardiac arrest

The hazard ratios of the Cox-Proportional Hazards shows that INR has the highest hazard ratio, while

temperature has the lowest. With the exception of the highest range of the INR, and the lowest of tem-

perature, fraction of inspired oxygen and respiratory rate, all of the other hazard ratios display overlapping

confidence ratios.

In summary, as can be expected given the amount of temporal data available in the dataset in Figure B.9,

the parameters VSFIO2, SystolicBP, DiastolicBP, RespiratoryRate, and Temperature exhibit relatively high

predictive ability in the model. Furthermore, all of these parameters were collected via the query in Sec-

tion A.3, and exhibit the most data in Figure B.12. Not consistent with this pattern is the result that INR

is the most predictive parameter for all outcomes in the CPH model. INR also exhibits a positive HR, as

well as the largest difference from the baseline HR of 1 than all other parameters across all outcomes.

6.4 Gray’s Method of Cumulative Incidence of Competing Risk

6.4.1 Methods

In a similar manner to Cox-Proportional Hazards, competing risk serves to model the cause-specific

proportional hazards of co-variates associated with the outcome in question. In using the competing risk

model to evolve a cumulative incidence function, the marginal cause probabilities can be estimated, and a

more complete accounting of the co-variates can be taken, as not to exhibit bias in the model. To model

sub-distribution hazard λk, of Z vector of co-variates, based off Gray [158], Gray [159] further defines the

function as:

λk(t,Z) =∂Fk(t;Z)

∂t

1− Fk(t,Z)

=−∂log(1− Fk(t,Z))

∂t(6.11)

Where Z is a vector of co-variates affecting said failure (input parameters). Given the inclusion of “cured”

patients in the potential dataset, and that ε = 1 is not always a possible circumstance for patient k, the

cumulative distribution function can be simplified to [159]:

Fk(t; Z) = 1− e−∫ t0λk0e

ZT (s)β0ds (6.12)

Therefore, as with the Cox-Proportional Hazards model, the partial likelihood can be expressed by the

following equation for Fk(t; Z) [159]:

L(β) =

n∏i=1

[λk0(Ti)e

ZTi (Ti)β∆Ti∑j∈Ri λk0(Ti)e

ZTj (Ti)β∆Ti

](6.13)

These cumulative incidence models are analogous to the Cox-Proportional Hazards model, yet for any failure

type they serve to model the sub-distribution hazard function.


From Coviello and Boggess [160], the cumulative incidence can be simplified to:

Ik(t) =∑j|tj≤t

S(tj − 1)∂kjnj

(6.14)

For the Kaplan-Meier estimate S(tj − 1) and a hazard of type k.

6.4.2 Results

In solving this model, the test statistic output by this model is described in Gray [158]. The p-values

of the parameter set passed to the cumulative risk function per outcome are presented in Figure B.17. The

tables following describe the raw output from R of the model output parameters of these figures in numerical

detail.

For all of the outcomes, the order of diastolic blood pressure, pulse, systolic blood pressure, temperature,

respiratory rate, fraction of inspired oxygen, INR, urea, potassium is the set of most predictive parameters,

while RDW or HCT are the least predictive.

Code Blue

stat pv df

DiastolicBP 2.19E+02 0.00E+00 2.00E+00

Pulse 1.10E+02 0.00E+00 2.00E+00

SystolicBP 2.19E+02 0.00E+00 2.00E+00

Temperature 7.01E+01 6.66E-16 2.00E+00

RespiratoryRate 5.73E+01 3.55E-13 2.00E+00

VSFiO2 4.93E+01 1.97E-11 2.00E+00

INR 9.69E+00 7.88E-03 2.00E+00

Urea 9.63E+00 8.11E-03 2.00E+00

Potassium 8.03E+00 1.81E-02 2.00E+00

AnionGap 8.03E+00 1.81E-02 2.00E+00

TotalCO2 8.03E+00 1.81E-02 2.00E+00

Chloride 8.03E+00 1.81E-02 2.00E+00

Sodium 8.03E+00 1.81E-02 2.00E+00

EstimatedGFR 8.01E+00 1.82E-02 2.00E+00

Baso# 6.61E+00 3.68E-02 2.00E+00

Eos# 6.61E+00 3.68E-02 2.00E+00

Lymp# 6.61E+00 3.68E-02 2.00E+00

Mono# 6.61E+00 3.68E-02 2.00E+00

Neut# 6.61E+00 3.68E-02 2.00E+00

Creatinine 6.57E+00 3.75E-02 2.00E+00

MPV 5.31E+00 7.02E-02 2.00E+00

Platelets 5.31E+00 7.02E-02 2.00E+00

WBC 5.31E+00 7.02E-02 2.00E+00

Hb 5.31E+00 7.02E-02 2.00E+00

MCH 5.31E+00 7.02E-02 2.00E+00

MCHC 5.31E+00 7.02E-02 2.00E+00

MCV 5.31E+00 7.02E-02 2.00E+00


RBC 5.31E+00 7.02E-02 2.00E+00

RDW 5.31E+00 7.02E-02 2.00E+00

HCT 5.31E+00 7.02E-02 2.00E+00

Cardiac Arrest

stat pv df

DiastolicBP 2.19E+02 0.00E+00 2.00E+00

Pulse 1.10E+02 0.00E+00 2.00E+00

SystolicBP 2.19E+02 0.00E+00 2.00E+00

Temperature 7.01E+01 6.66E-16 2.00E+00


VSFiO2 4.93E+01 1.97E-11 2.00E+00

INR 9.69E+00 7.88E-03 2.00E+00

Urea 9.63E+00 8.11E-03 2.00E+00

Potassium 8.03E+00 1.80E-02 2.00E+00

AnionGap 8.03E+00 1.80E-02 2.00E+00

TotalCO2 8.03E+00 1.80E-02 2.00E+00

Chloride 8.03E+00 1.80E-02 2.00E+00

Sodium 8.03E+00 1.80E-02 2.00E+00


Baso# 6.61E+00 3.68E-02 2.00E+00

Eos# 6.61E+00 3.68E-02 2.00E+00

Lymp# 6.61E+00 3.68E-02 2.00E+00

Mono# 6.61E+00 3.68E-02 2.00E+00

Neut# 6.61E+00 3.68E-02 2.00E+00

Creatinine 6.57E+00 3.74E-02 2.00E+00

MPV 5.31E+00 7.02E-02 2.00E+00

Platelets 5.31E+00 7.02E-02 2.00E+00

WBC 5.31E+00 7.02E-02 2.00E+00

Hb 5.31E+00 7.02E-02 2.00E+00

MCH 5.31E+00 7.02E-02 2.00E+00

MCHC 5.31E+00 7.02E-02 2.00E+00

MCV 5.31E+00 7.02E-02 2.00E+00

RBC 5.31E+00 7.02E-02 2.00E+00

RDW 5.31E+00 7.02E-02 2.00E+00

HCT 5.31E+00 7.02E-02 2.00E+00

Transfer to ICU

stat pv df

DiastolicBP 2.19E+02 0.00E+00 2.00E+00

Pulse 1.10E+02 0.00E+00 2.00E+00

SystolicBP 2.19E+02 0.00E+00 2.00E+00

Temperature 7.01E+01 5.55E-16 2.00E+00



VSFiO2 4.93E+01 1.94E-11 2.00E+00

INR 9.69E+00 7.85E-03 2.00E+00

Urea 9.63E+00 8.11E-03 2.00E+00

Potassium 8.03E+00 1.81E-02 2.00E+00

Chloride 8.03E+00 1.81E-02 2.00E+00

Sodium 8.03E+00 1.81E-02 2.00E+00

TotalCO2 8.03E+00 1.81E-02 2.00E+00

AnionGap 8.03E+00 1.81E-02 2.00E+00


Baso# 6.60E+00 3.69E-02 2.00E+00

Eos# 6.60E+00 3.69E-02 2.00E+00

Lymp# 6.60E+00 3.69E-02 2.00E+00

Mono# 6.60E+00 3.69E-02 2.00E+00

Neut# 6.60E+00 3.69E-02 2.00E+00

Creatinine 6.57E+00 3.75E-02 2.00E+00

HCT 5.31E+00 7.04E-02 2.00E+00

MPV 5.31E+00 7.04E-02 2.00E+00

Platelets 5.31E+00 7.04E-02 2.00E+00

WBC 5.31E+00 7.04E-02 2.00E+00

Hb 5.31E+00 7.04E-02 2.00E+00

MCH 5.31E+00 7.04E-02 2.00E+00

MCHC 5.31E+00 7.04E-02 2.00E+00

MCV 5.31E+00 7.04E-02 2.00E+00

RBC 5.31E+00 7.04E-02 2.00E+00

RDW 5.31E+00 7.04E-02 2.00E+00

Death

stat pv df

DiastolicBP 2.19E+02 0.00E+00 2.00E+00

Pulse 1.10E+02 0.00E+00 2.00E+00

SystolicBP 2.19E+02 0.00E+00 2.00E+00

Temperature 7.00E+01 6.66E-16 2.00E+00


VSFiO2 4.93E+01 2.00E-11 2.00E+00

INR 9.70E+00 7.82E-03 2.00E+00

Urea 9.65E+00 8.03E-03 2.00E+00

Potassium 8.04E+00 1.79E-02 2.00E+00

Chloride 8.04E+00 1.79E-02 2.00E+00

Sodium 8.04E+00 1.79E-02 2.00E+00

TotalCO2 8.04E+00 1.79E-02 2.00E+00

AnionGap 8.04E+00 1.79E-02 2.00E+00


Baso# 6.61E+00 3.68E-02 2.00E+00

Eos# 6.61E+00 3.68E-02 2.00E+00


Lymp# 6.61E+00 3.68E-02 2.00E+00

Mono# 6.61E+00 3.68E-02 2.00E+00

Neut# 6.61E+00 3.68E-02 2.00E+00

Creatinine 6.58E+00 3.72E-02 2.00E+00

HCT 5.31E+00 7.03E-02 2.00E+00

MPV 5.31E+00 7.03E-02 2.00E+00

Platelets 5.31E+00 7.03E-02 2.00E+00

WBC 5.31E+00 7.03E-02 2.00E+00

Hb 5.31E+00 7.03E-02 2.00E+00

MCH 5.31E+00 7.03E-02 2.00E+00

MCHC 5.31E+00 7.03E-02 2.00E+00

MCV 5.31E+00 7.03E-02 2.00E+00

RBC 5.31E+00 7.03E-02 2.00E+00

RDW 5.31E+00 7.03E-02 2.00E+00


stat pv df

DiastolicBP 2.19E+02 0.00E+00 2.00E+00

Pulse 1.10E+02 0.00E+00 2.00E+00

SystolicBP 2.19E+02 0.00E+00 2.00E+00

Temperature 7.00E+01 6.66E-16 2.00E+00


VSFiO2 4.93E+01 2.01E-11 2.00E+00

INR 9.70E+00 7.85E-03 2.00E+00

Urea 9.65E+00 8.01E-03 2.00E+00

Potassium 8.05E+00 1.79E-02 2.00E+00

Chloride 8.05E+00 1.79E-02 2.00E+00

Sodium 8.05E+00 1.79E-02 2.00E+00

TotalCO2 8.05E+00 1.79E-02 2.00E+00

AnionGap 8.05E+00 1.79E-02 2.00E+00


Baso# 6.60E+00 3.69E-02 2.00E+00

Eos# 6.60E+00 3.69E-02 2.00E+00

Lymp# 6.60E+00 3.69E-02 2.00E+00

Mono# 6.60E+00 3.69E-02 2.00E+00

Neut# 6.60E+00 3.69E-02 2.00E+00

Creatinine 6.58E+00 3.72E-02 2.00E+00

HCT 5.31E+00 7.03E-02 2.00E+00

MPV 5.31E+00 7.03E-02 2.00E+00

Platelets 5.31E+00 7.03E-02 2.00E+00

WBC 5.31E+00 7.03E-02 2.00E+00

Hb 5.31E+00 7.03E-02 2.00E+00

MCH 5.31E+00 7.03E-02 2.00E+00

MCHC 5.31E+00 7.03E-02 2.00E+00

MCV 5.31E+00 7.03E-02 2.00E+00


RBC 5.31E+00 7.03E-02 2.00E+00

RDW 5.31E+00 7.03E-02 2.00E+00

In summary, reinforcing the results from the CPH modeling, from the cumulative incidence model, INR

is again the most predictive parameter, slightly ahead of Urea albeit behind the vital signs set above. This

is behaviour consistent with the dataset, given the amount of data provided by the vital signs set displayed

in Figures B.9, B.10, B.11, and B.12.

6.5 General Discrimination Indices for Censored Data

6.5.1 Methods

Gauging the predictive power of the Cox-Proportional Hazards model serves to identify the relative

strength of the modeling of the clinical outcomes in question. For the purpose of this analysis, two measures

describe the predictive power of the Cox-Proportional Hazards model:

• Harrell’s C correlation index [161] (c)

• Somer’s D rank correlation index [162] (D)

Both of these serve as a general discrimination index, or measure of correlation similar to mean squared

error. c is defined as “the proportion of all usable patient pairs in which the predictions and outcomes are

concordant” [161] c is calculated by the following method, as described in Harrell at el [161]:

“In predicting the time until death, c is calculated by considering all possible pairs of patients,

at least one of whom has died. If the predicted survival time is larger for the patient who lived

longer, the predictions for that pair are said to be concordant with the outcomes. If one patient

died and the other is known to have survived at least to the survival time of the first, the second

patient is assumed to outlive the first. When predicted survivals are identical for a patient pair,

rather than 1 is added to the count of concordant pairs in the numerator of c.” [161]

Where the c is the set [0.5,1.0], with the limits of [no predictive discrimination,perfect separation of patients],

D is derived by the equation [161] and exists within set [-1,1] with 0 indicating no correlation:

2(c− 0.5) (6.15)

6.5.2 Results

Table B.8 describes the indices above for the base survival model on which the Cox-Proportional Hazard

model is based for each clinically significant outcome.

Outcome dT Threshold Early Warning Score Chi-Squared Test Statistic p-Value

CardiacArrest (0,4] NEWS 0 0

CardiacArrest (0,8] NEWS 0 0

Composite (0,16] ViEWS 0.001 0

CardiacArrest (0,6] NEWS 0.003 0

CodeBlue (0,6] NEWS 0.003 0

CardiacArrest (0,6] CART 0.004 0

ICUTransfer (0,6] ViEWS 0.004 0


CardiacArrest (0,6] ViEWS 0.006 0

CodeBlue (0,8] ViEWS 0.006 0




Death (0,4] CART 0.037 0

CodeBlue (0,6] CART 0.039 0







Death (0,16] ViEWS 0.129 0



Death (0,6] CART 0.378 0


CodeBlue (0,8] MEWS 0.455 0


Death (0,24] ViEWS 0.573 0





CodeBlue (0,24] CDF 1.21 0




Death (0,16] NEWS 2.539 0



CardiacArrest (0,8] MEWS 2.701 0




Death (0,12] NEWS 3.347 0





Composite (0,6] NEWS 4.377 0


Death (0,8] NEWS 4.581 0





Death (0,24] MEWS 5.243 0

Death (0,6] NEWS 5.35 0


Death (0,12] ViEWS 5.992 0

Death (0,8] ViEWS 6.499 0



ICUTransfer (0,4] MEWS 6.942 0



ICUTransfer (0,12] NEWS 7.378 0



Death (0,6] ViEWS 7.812 0











CardiacArrest (0,24] CDF 9.184 0



Death (0,16] MEWS 10.285 0




Composite (0,16] MEWS 10.644 0




Death (0,24] NEWS 11.799 0









Death (0,12] MEWS 13.941 0












ICUTransfer (0,4] CART 23.806 0

ICUTransfer (0,4] CDF 24.218 0

Death (0,8] CART 25.598 0




Death (0,4] ViEWS 34.408 0

Death (0,4] MEWS 47.554 0

Death (0,16] CART 49.171 0

Composite (0,4] CART 49.256 0

Death (0,8] MEWS 50.719 0

Death (0,4] CDF 52.146 0

Death (0,6] MEWS 52.386 0


Death (0,12] CART 55.728 0


Death (0,4] NEWS 56.369 0

Death (0,24] CART 57.263 0













Death (0,6] CDF 247.197 0

Composite (0,24] CDF 261.396 0



Death (0,24] CDF 311.222 0



Death (0,16] CDF 357.05 0


Death (0,8] CDF 383.904 0


Death (0,12] CDF 389.679 0

Table 6.11: Table of Hosmer Lemeshow Goodness of Fit Tests for

Receiver Operating Curve Parameters Sorted by Chi-Squared Test

Statistic

In summary, for all outcomes modeled, the c-index for all values approaches 0.5, with a difference of

O(10−3). Furthermore, the D-index approaches 0, with a difference of O(10−2). Therefore, we can conclude

that the survival models, although descriptive, have no statistical discriminative ability, and therefore should

not be trusted as accurate.


Although both Cox-Proportional Hazards and Gray’s Method of Cumulative Incidence of Competing

Risk both describe INR as being a consistently predictive value under censoring regimes, as well as when

accounting for co-variates, the discriminative ability of the underlying proportional hazard model is shown to

be quantitatively poor. Clinically, the data suggests that, although the frequently measured vital signs exhibit

the ability to predict long term outcome (as was described in the literature developing mortality prediction

scores such as APACHE), other laboratory values such as INR and Urea also weigh in as potentially of impact

via hazard modeling. Therefore, the consideration of INR and Urea, respectively measures of the coagulation

pathway and metabolic/kidney function may assist in the prediction of outcomes. This is again consistent

with the frailty and sepsis models as described in Chapter 4, as each are measures of multi-organ affecting

systemic disorders, circulation and metabolism, the disruption of either of which can lead to physiological

deterioration and subsequent death. The results, therefore, suggest that future work to refine the models

via parameter selection or weighting may prove beneficial, particularly as the model discrimination ability is

improved.

Chapter 7

Conclusions

7.1 Summary of Findings

7.1.1 Input Analysis

To answer the question “What contribution do selected clinical and laboratory parameters have to the

existing early warning score predictive model of the Patient Care and Safety Compromising Events (such as

death, cardiac arrest, code blue, ICU transfer)?”, logistic regression, random forest classification and cop-

ula dependence models provide insight into the parameters associated with clinical event modeling. The

contribution of clinical and laboratory parameters to predictive models can be summarized as that gener-

ally two classes of parameters are predictive of outcome: hematology, and metabolic or kidney function.

These measures of red blood cell activity and quantity, as well as waste clearance, serve as components

of the measures of the endogenous mediators in the chain of response leading to shock and subsequent

multi-organ-system failure [132], a potential pathogenic response related to models for frailty and patient

decompensation [141, 142, 143]. This is consistent with the bias attributed to the dataset. Dependence mod-

eling provides us with both confirmation and a ranking of bias between variables, such that the dependence

brought about via known linkages for calculated variables, as well as both ends of the variability spectrum

both overpower linkages between other sets of variables, further confirming data biases.

7.1.2 System Analysis

To answer the question “How do the predictive accuracies of existing early warning scores compare with

each other for the identification of the Patient Care and Safety Compromising Events (such as death, cardiac

arrest, code blue, ICU transfer)?”, receiver operating curves, entropy estimators and goodness-of-fit methods

provide for a picture of the accuracy of existing early warning scores operating within the dataset. Comparing

predictive methods, the MEWS score exhibits the most effective performance according to Receiver Operating

Characteristic (ROC) curve analysis statistic of Area Under the Curve (AUC). Accounting for the confidence

interval on the analysis, however, we see that between MEWS, ViEWS, NEWS and CDF they overlap and

therefore statistically the performance of one is not significantly different than the other. From the evaluation

of the entropy estimators, we know that both the CART score exhibits the most troubling variability, despite

the known variability of the CDF score, a potential explanation of its poor performance. From the goodness-

of-fit analysis, the early warning scores on outcome modeling built upon the positive and negative framework

of predictive values associated with the ROC curves confirm the conclusions of the entropy estimators.

Additionally, this analysis identifies that with the lower time-to-event threshold comes a tighter range of fit

66

Chapter 7. Conclusions 67

data, therefore confirming the pattern of higher AUC with greater time-to-event thresholds exhibited by the

ROC curves.

7.1.3 Output Analysis

To answer the question “Does imputation of missing data improve the predictive accuracy of Patient

Care and Safety Compromising Events (such as death, cardiac arrest, code blue, ICU transfer) by early

warning scores?”, proportional hazard and cumulative risk modeling was employed to make use of the

temporal features of the data and account for them in modeling of clinical events. Using hazard modeling, a

technique which uses imputation and uses the temporal information of the dataset (rather than the values)

to predict outcome, serves to qualify the impact of imputation on the predictive accuracy of models. Cox-

Proportional Hazards, as well as Cumulative Incidence of Comparative Risk show that INR is the most

predictive parameter, beyond those of the parameters collected at high frequency intervals. Additionally,

urea also exhibits predictive power (albeit behind the high frequency of collection data), further qualifying

the ability of laboratory tests of hematology and kidney function to predict outcomes. However, when using

Harrell’s C or Somer’s D on the survival models shows that the discriminative ability of these models is poor,

and therefore their statistical effectiveness is null. In summary, the imputation of data and use of temporal

data as a means to model outcomes fails to statistically improve the predictive accuracy of PCSCE, however

it does display potential features of the dataset tied to temporal characteristics of data.

7.1.4 Clinical Impact

Clinically, this work demonstrates that laboratory values are predictive for clinically significant outcomes

related to PCSCE. In particular, hematology results such as INR and kidney function results such as esti-

mated GFR and urea bear monitoring (as they already are in routine practice) as a means to predict PCSCE.

These mechanisms are consistent with other means of predicting risk, such as the surgical Apgar score [75]

or the Rockall and Blatchford methods in upper gastrointestinal bleeding [76]. Furthermore, in concordance

with this work, clinical guidance on acute decompensation prepared by NICE in the United Kingdom [17], as

well as policy guidance from Australia [19] suggest there is an impact to more specific analysis of laboratory

values, particularly biochemistry and hematology. Therefore, in line with Soar and Subbe [163], should there

be no creation of additional early warning scores, attention through the use of policy dictates may provide

for more scrutiny of appropriate laboratory parameters as necessary.

7.2 Limitations and Future Work

This analysis serves as an alpha iteration of the solution design task, and the results from this thesis

require further expansion and validation through scrutiny and replication. The lack of clinical input and

direct clinical validity through prospective analysis serves as an identifiable limitation, which curtails the

analysis’s impact on potential applications within the hospital. Furthermore, the limitation of a lack of a

validation cohort for the predictive models outside of the known RRT associated patients hinders the ability

of the model to display predictive generalizability among the patient cohort. Therefore, the construction of

a complementary analysis framework for a new set of data without known RRT interventions is a potential

future point of analysis, as a means of furthering the generalizability of the models. The gap between

discrete measurement and continuous function remains a hallmark of clinical analysis outside of a continuous

monitoring environment such as the ICU, and using more sophisticated methods such as temporal clustering

may improve the results in the future. The absence of clinical and laboratory parameter values in each

Chapter 7. Conclusions 68

encounter, as well as in significant number in the temporal series per encounter, serve as a source of bias.

Selecting a subset of variables results in the potential for missing key factors that would otherwise be

significant if not for measurement frequency, and in building for the future, tools such as principle component

analysis or a more computationally efficient form of copula might prove useful in performing initial variable

selection, and clustering tools to allow variables to be fed into regression and classification modeling.

Appendix A

Cerner Command Language Queries

A.1 Blood Pressure

drop program tegh_qry_ccrt_b_p go

create program tegh_qry_ccrt_b_p

prompt "mine" = mine

select into $1

ce.encntr_id,

event_disp = uar_get_code_display(ce.event_cd),

result_disp = ce.event_tag,

result_dt_tm = ce.event_end_dt_tm "dd-mmm-yyyy hh:mm;;d"

from

clinical_event ce

plan ce where ce.encntr_id in (

encounter_ids

)

and ce.view_level = 1

and ce.valid_until_dt_tm > cnvtdatetime(curdate, curtime)

and ce.event_cd in (


125164, ;Systolic

125167, ;Diastolic

862451) ;Diastolic

end

go

69

Appendix A. Cerner Command Language Queries 70

A.2 Laboratory Values

drop program tegh_qry_ccrt_enc_lab go

create program tegh_qry_ccrt_enc_lab


select into $1

ce.encntr_id,




from

clinical_event ce,

orders o


encounter_ids

)



and ce.order_id > 0

join o where o.order_id = ce.order_id

and o.catalog_type_cd = 4496 ;lab

end

go

A.3 Vital Signs

drop program tegh_qry_ccrt_vs go

create program tegh_qry_ccrt_vs


select into $1

ce.encntr_id,




from


clinical_event ce


encounter_ids

)




125290, ;weight

125293, ;height

125146, ;temperature

125158, ;pulse

125284, ;respiratory rate

186769, ;fio2

63910982) ;avpu

end

go

A.4 Age on Admission

drop program tegh_qry_non_ccrt_enc go

create program tegh_qry_non_ccrt_enc


select into $1

tnc.*,

mrn = tnc.mrn,

e.encntr_id,

p.person_id,

pa.alias,

pa.person_alias_id,

e.encntr_id,

record_date = tnc.record_date "dd-mmm-yyyy hh:mm;;d",

e.reg_dt_tm "dd-mmm-yyyy hh:mm;;d",

e.disch_dt_tm "dd-mmm-yyyy hh:mm;;d",

disp_disposition= uar_get_code_display(e.disch_disposition_cd),

ADM_AGE=substring(1, 5,cnvtage(p.birth_dt_tm, e.reg_dt_tm, 0))

from tegh_non_ccrt tnc,

person_alias pa,

person p,

encounter e,

dummyt d


plan tnc

join pa where pa.alias = tnc.mrn

AND PA.PERSON_ALIAS_TYPE_CD= 3088

;AND PA.END_EFFECTIVE_DT_TM > CNVTDATETIME(CURDATE,235959)

AND PA.ACTIVE_IND=1

;and pa.alias = "1007020"

;and pa.CONTRIBUTOR_SYSTEM_CD = 0

join p where p.person_id = pa.person_id

and p.person_type_cd = 912

join e where e.person_id = pa.person_id

AND E.CONTRIBUTOR_SYSTEM_CD != 14724

and e.active_ind = 1

and e.encntr_type_cd in (15426,597920, 597924,10041404)

join d where cnvtdate(tnc.record_date,0) BETWEEN cnvtdate(e.reg_dt_tm,0) and cnvtdate(e.disch_dt_tm,235959)

end

go

Appendix B

Figures

Event Detectionvia Early Warning Score Parameters

RapidResponse Team

Specialized ResourcesCode, Trauma, or Stroke Teams

Patient Physiological Decompensation Signal Crisis Resolved

Early Warning Score Threshold

Figure B.1: Unified Modeling Language Representation of Rapid Response System (Activity Diagram)

Parameter Description

PatientDeIdentityID Patient identification within the dataset, in order to pre-

serve uniqueness while discarding personal health infor-

mation

Age Patient’s age in years

Gender Gender of the patient (male or female)

ICUAdmissionTime Time and date of patient’s admission to the ICU

ICUDischargeTime Time and date of patient’s discharge from the ICU

ICUDischargeLocation Location to which the patient was discharged from the

ICU

SeenByCCRT Time and date when the patient was seen by the RRT

SeenByMD Time and date when the patient was seen by the MD on

the RRT

SeenByRN Time and date when the patient was seen by the RN on

the RRT

SeenByRT Time and date when the patient was seen by the RT on

the RRT

NotifiedBy Which clinician notified the RRT (MD, RN, RT, PT)

PrimaryReason Primary reason why the RRT was notified

AdmittingService Admitting service to the hospital of the patient

ABCTriage If triage was required, what level

CCRTCallingCriteriaMetHistorical Whether the RRT calling criteria were met historically

CCRTCallingCriteriaMetTimelinessHistorical The timeliness of the RRT calling criteria date

DateTimePatientMetCCRTCallingCriteria Time and date the patient met the callingr criteria

73

Appendix B. Figures 74

CCRTNotifiedDateTime Time and date the RRT was notified on

CodeBlueYesNo Whether a code blue occurred for this patient

PrimaryEvent.Historical The primary event for this patient

PatientReassessed Whether the patient was reassessed

CCRTMDNotified Whether the RRT MD was notified

CCRTMDNotifiedTime If the RRT MD was notified, time and date of notification

TimeLastTeamMemberLeft Time the last RRT member left the patient

CCRTCallOutcomes Transfer outcomes post RRT visit

EndOfLifeDiscussionInitiatedYesNo Whether an end of life discussion was intitated

EndOfLifeDiscussionInitiated What type of end of life discussion took place

ICURequestDateTime Time and date of the request for ICU transfer

TimeWithPatientAfterICUAdmissionHours Number of hours RRT spent with patient post ICU ad-

mission

TimeWithPatientAfterICUAdmissionMinutes Number of minutes RRT spent with patient post ICU

admission

Table B.1: Selected Critical Care Secretariat Reporting Database Field

Headers

Parameter Type Description of Data

PatientDeIdentityID char Values representing unique identifier for the

dataset

Age num Values [18,102.422]

Gender str Binary gender {F, M}ICUAdmissionTime str Date and timestamp of action

ICUDischargeTime str Date and timestamp of action

ICUDischargeLocation str Discharge location {Coronary Care, ICU}SeenByCCRT str Date and timestamp of action

SeenByMD str Date and timestamp of action

SeenByRN str Date and timestamp of action

SeenByRT str Date and timestamp of action

NotifiedBy str Clinician which notified the RRT {Other, MD,

Physio, RN, RT}PrimaryReason str Description of reason why RRT was activated

{Airway threatened, Altered Mental Status, De-

saturation, HR > 130, HR < 40, Oliguria, Other,

Patient / Family Counseling, Prolonged Seizure,

RR > 30, RR < 8, SBP > 200, SBP < 90, SOB,

Worried About Patient}AdmittingService str Hospital service which admitted the pa-

tient{Ambulatory Care, Cardiac Surgery,

Cardiology, Dermatology, Emergency Medicine,

Endocrinology, ENT, Family Practice, Gastroen-

terology, Gen. Internal Medical, Gen. Surgery,

Infectious Disease, Nephrology, Neurology,

OBS/GYN, Oncology/Haematology, Ophthal-

mology, Orthopaedic, Plastic, Psychiatry, Rehab,

Respirology, Thoracic, Urology}ABCTriage str Indication of triage status{A - Admit to Level 3

ICU, B - Borderline, C - Consults}


CCRTCallingCriteriaMetHistorical str Binary action confirmed {Yes,No}CCRTCallingCriteriaMetTimelinessHistorical str Timeliness at which the above action took place

at threshold of 24 or 8 hours {>= 24 hrs, < 8 hrs,

>= 8 hrs}DateTimePatientMetCCRTCallingCriteria str Date and timestamp of action

CCRTNotifiedDateTime str Date and timestamp of action

CodeBlueYesNo str Binary action confirmed {Yes,No}PrimaryEvent.Historical str Event description, classified by code blue type

{Cardiac Arrest (Chest Compression and/or Elec-

trical Defibrillation Performed), Non-Code Blue

event, Other Code Blue, Respiratory Arrest (Pos-

itive Pressure Ventilation Provided), Respiratory

arrest without cardiac arrest (Positive pressure

ventilation provided)}PatientReassessed str Binary action confirmed {Yes,No}CCRTMDNotified str Binary action confirmed {Yes,No}CCRTMDNotifiedTime str Date and timestamp of action

TimeLastTeamMemberLeft str Date and timestamp of action

CCRTCallOutcomes str Description of response to RRT

{Other,Transferred to ICU, Stay on Unit,

Transferred to Step Down Unit}EndOfLifeDiscussionInitiatedYesNo str Binary action confirmed {Yes,No}EndOfLifeDiscussionInitiated str Description of end of life discussion type {Code

status discussion initiated, Code status previously

established, Code status revisited, Discussion re-

fused by family / patient / MRP, Not appropriate

at this time or for this patient}ICURequestDateTime str Date and timestamp of action

TimeWithPatientAfterICUAdmissionHours num No values for dataset

TimeWithPatientAfterICUAdmissionMinutes num No values for dataset

Table B.2: Data Description of Critical Care Secretariat Database

Abbreviation Description

SystolicBP Systolic Blood Pressure

DiastolicBP Diastolic Blood Pressure

WBC White Blood Cell Count

RBC Red Blood Cell Count

Hb Hemoglobin Level

HCT Hematocrit

MCV Mean Corpuscular Volume (HCT/Hgb)

MCH Mean Corpuscular Hemoglobin (Hb/RBC)

MCHC Mean Corpuscular Hemoglobin Concentration (Hb/MCV

x RBC)

RDW Red Blood Cell Distribution Width

Platelets Platelet Count

MPV Mean Platelet Volume

Neut# Neutrophil Count

Lymp# Lymphocyte Count

Mono# Monocyte Count


Eos# Eosinophil Count

Baso# Basophil Count

Urea Urea Level

Sodium Sodium Level

Potassium Potassium Level

Chloride Chloride Level

TotalCO2 Carbon Dioxide Level

Creatinine Creatinine Level

EstimatedGFR Estimated Glomerular Filtration Rate

AnionGap Ion Gap Measurement

INR International Normalized Ratio

PTT Partial Thromboplastin Time

Calcium Calcium Level

Phosphorus Phosphorus Level Measurement

TroponinT(HighSens.) Troponin T Level Measurement

Magnesium Magnesium Level Measurement

Glucose(Random) Random Glucose Level Measurement

SpecificGravity(Urine) Specific Gravity of Urine

UrinepH pH of Urine

UrineProtein Protein Level in Urine

Urobilinogen Urobilinogen Level Measurement

L-LacticAcid-Plasma Lactate Level

TotalProtein Total Protein Count

Albumin Albumin Level

pHArterial Arterial Blood pH

pCO2Arterial Arterial pCO2

pO2Arterial Arterial pO2

HCO3Arterial Arterial HCO3 Level

FiO2 Fraction of Inspired Oxygen Level

O2SaturationArterial Arterial O2 Saturation

pHVenous Venous pH

pCO2Venous Venous pCO2

pO2Venous Venous pO2

BaseExcessVenous Venous Blood Gas Base Excess Level

HCO3Venous Venous HCO3

O2SaturationVenous Venous O2 Saturation

Carboxy-Hb Carboxyhemoglobin Levels

ALT Alanine Amino Transferase Level

AST Aspartate Amino Transferase Level

ALP Alkaline Phosphatase Level

TotalBilirubin Total Bilirubin Level

Amylase(Serum) Amylase Level in Blood

D-DimerFEU D-Dimer Levels (Fibrinogen Equivalent Units)

Glucose(Fasting) Fasting Glucose Level

BaseExcessVenous Arterial Blood Gas Base Excess Level

Retic# Reticulocyte Count

Ferritin Ferritin Level

ProteinElectrophoresis Measurement of Types of Protein in Blood

Iron Iron Levels

Transferrin Transferrin Levels


TransferrinSaturation Transferrin Saturation Level

Vit.B12 Vitamin B12 Level

RBCFolate Red Blood Cell Folate Level

24HrUrineCreat. Urine Creatinine level at 24 Hour Increment

24HrUrineProtein Urine Protein Level at 24 Hours Increment

AlbuminElectrophoresis Measurement of Types of Albumin in Blood

Alpha1Globulin Alpha 1 Globulin Level in Blood

Alpha2Globulin Alpha 2 Globulin Level in Blood

BetaGlobulin Beta Globulin Level in Blood

GammaGlobulin Gamma Globulin Level in Blood

M-Spike Monoclonal Protein Levels

A\GRatio Albumin\Globulin Ratio

HbA1C Hemoglobin A1C Level

IgG Immunoglobulin G Level

IgA Immunoglobulin A Level

IgM Immunoglobulin M Level

UricAcid Uric Acid Level

HbA2 Hemoglobin A2 Level

Albumin(Urine) Albumin Level in Urine

Alpha-1Globulin(Urine) Alpha 1 Globulin Level in Urine

Alpha-2Globulin(Urine) Alpha 2 Globulin Level in Urine

BetaGlobulin(Urine) Beta Globulin Level in Urine

GammaGlobulin(Urine) Gamma Globulin Level in Urine

M-Spike(Urine) Monoclonal Protein Levels in Urine

HbA Hemoglobin A Level

Cholesterol Cholesterol Level

Triglycerides Triglycerides Level

HDL High-Density Lipoprotein Level

LDL Low-Density Lipoprotein Level

CHOL/HDLRatio Cholesterol to High-Density Lipoprotein Level

CPK Creatinine Phosphokinase Level

TSH(2ndGeneration) Thryroid Stimulating Hormone 2nd Generation Assay

Neutrophils(wbc) Neutrophils Levels in White Blood Cells

Lymphocytes(wbc) Lymphocytes Levels in White Blood Cells

AtypicalLymphocytes Atypical Lymphocyte Levels

Monocytes(wbc) Monocyte Levels in White Blood Cells

Eosinophilswbc) Eosinophil Levels in White Blood Cells

Basophils(wbc) Basophil Levels in White Blood Cells

Bands Band Levels (Immature Cells)

Metamyelocytes Metamyelocyte Levels

Myelocytes Myelocyte Levels

Promyelocytes Promyelocyte Levels

Blasts(wbc) Blast Levels (Immature Cells)

LDH Lactate Dehydrogenase Test

PhenytoinLevel Phenytoin Levels

ValproicAcidLevel Valprioc Acid Levels

PhenobarbitalLevel Phenobarbital Levels

NRBC’s/100WBC’s Number of Red Blood Cells per 100 White Blood Cells

FibrinogenLevelAssay Fibrinogen Level Assay

UrineOsmolality Osmolality of Urine


RandomUrineSodium Random Urine Sodium Level

RandomUrinePotassium Random Urine Potassium Level

RandomUrineChloride Random Urine Chloride Level

AlphaFetoprotein Alpha Fetoprotein Level

FreeT3 Triiodothyronine Levels

FreeT4 Thyroxine Levels

Cortisol(AM) Cortisol Levels (am)

PlasmaOsmolality Osmolality of Plasma

Fluid-RBC RBC in Fluid

NucleatedCell(BodyFluid) Nucleated Cells in Body Fluid

Neutrophils(BodyFluid) Neutrophils in Body Fluid

Lymphocytes(BodyFluid) Lymphocytes in Body Fluid

Monocytes(BodyFluid) Monocytes in Body Fluid

Eosinophils(BodyFluid) Eosinophils in Body Fluid

NonHematopoieticCells(BodyFluids) Non Hematopoetic Cells in Body Fluid

OtherCells(BodyFluid) Other Cells (than above) in Body Fluid

DirectBilirubin Direct Measurement of Bilirubin

Anti-cardiolipinIgG Anticardiolipin Antibody Type Immunoglobulin G Lev-

els

Anti-cardiolipinIgM Anticardiolipin Antibody Type Immunoglobulin M Lev-

els

Creatine(Fluid) Creatine in Fluid

PlasmaFolate Plasma Folate Levels

AcetaminophenLevel Acetaminophen Levels

Salicylate Salicylate Levels

EthanolLevel Ethanol Levels

ParathyroidH.-Intact Parathyroid Hormone Intact Levels

DigoxinLevel Digoxin Levels

anti-HBs Hepatitis B Surface Antigen Antibody Levels

SedimentationRate Sedimentation Rate of Red Blood Cells

CA125 Cancer Antigen 125 Level

Lipase(Serum) Lipase Levels

Cortisol(Random) Random Cortisol Levels

Vancomycin(Pre) Vancomycin Levels Pre-Dialysis

Vancomycin(Random) Random Vancomycin Levels

Temperature Temperature in degrees Celsius

Pulse Pulse Rate

VSFiO2 Fraction of Inspired Oxygen Regular Measurement

MEWSAVPUScale Modified Early Warnign Score Scale of Consciousness:

Alert, Voice, Pain, Unresponsive Consciousness Scale

RespiratoryRate Respiratory Rate

Weight Weight

Height Height

TotalProtein(Fluid) Total Protein Present in Fluid

Albumin(Fluid) Albumin Level in Fluid

LDH(Fluid) Lactate Dehydrogenase Test in Fluid

pH(Fluid) pH of Fluid

Haptoglobin Haptoglobin Level

GammaGT Gamma-Glutamyl Transpeptidase Level

Anti-thrombinIIIAct Antithrombin III Activity Level


ProteinCActivity Protein C Activity Levels

APCResistance Activated Protein C Resistance Present

ProteinS(Free) Free Protein S Level

Homocysteine(Plasma) Homocysteine Level in Plasma

TotalTestosterone Total Testosterone Levels

DNAAntibodies DNA Antibody Levels

C3 Complement C3 Test

C4 Complement C4 Test

Ceruloplasmin Ceruloplasmin Levels

Alpha-1-Antitrypsin Alpha-1-Antitrypsin Levels

c ANCA(PR3) c Antineurtophil Cytoplasmic Antibody Protinase 3 Test

p ANCA(MP0) p Antineurtophil Cytoplasmic Antibody Myelperoxidase

Test

TotalUrineVolume Total Volume of Urine

UrineProtein/CreatinineRatio(24hrs) Protein to Creatinine Ratio in Urine

RandomUrineMicroalbumin Microalbumin Level in Urine, Random Sample

RandomUrineCreat. Creatinine in Urine, Random Sample

Microalbumin/CreatinineRatio Microalbumin to Creatinine Ratio

Ammonia(Paeds) Paediatric Ammonia

CyclosporinLevel Cyclosporin Levels

24HrUrineVolume Volume of Urine, 24 Hour Test

24HrUrineMagnesium Magnesium in Urine, 24 Hour Test

24HrUrine-Delta-AminolevulinicAcid Delta-Aminolevulinic Acid in Urine, 24 Hour Test

CreatinineClearance Creatinine Clearance Levels

Anti-GbmAntibodies Anti-Glomerular Basement Membrane Antibodies Test

GabapentineLevel Gabapentine Levels

Calcium(Ionized) Ionized Calcium Concentration

TheophyllineLevel Theophylline Levels

CRP-AcutePhaseReactant C-Reactive Protein Acute-Phase Reactant Levels

Microalbumin24HRUrine Microalbumin Levels in Urine, 24 Hour Test

24HrRatio(Microalbumin/Creatinine) Microalbumin to Creatinine Ratio in Urine, 24 Hour Test

RheumatoidFactor Measurement of Rheumatoid factor antibody

Glucose(Fluid) Glucose Levels in Fluid

PSA Prostate-Specific Antigen Test

CEA Carcinoembryonic Antigen Test

RandomUrinePhosphorus Random Sampling of Phosphorus Level in Urine

Anti-Microsomal Antithyroid microsomal antibody test

CSFGlucose Glucose Amount in Cerebrospinal Fluid (CSF)

CSFProtein Protein Amount in Cerebrospinal Fluid

CSFRBC Red Blood Cell Count in Cerebrospinal Fluid

NucleatedCell(CSF) Nucleated Cell Count in CSF

Neutrophils(CSF) Neutrophil Count in CSF

Lymphocytes(CSF) Lymphocyte Count in CSF

Monocytes(CSF) Monocyte Count in CSF

Eosinophils(CSF) Eosinophil Count in CSF

Basophils(CSF) Basophil Count in CSF

NonHematopoieticCells(CSF) NonHematopoietic Cell Count in CSF

OtherCells(CSF) Cell Count for Categories not Listed in CSF

Anti-Thyroglobulin Anti-Thyroglobulin Antibody Test

CarbamazepineLevel Carbamazepine Drug Level Test


TopiramateLevel Topiramate Drug Level Test

FreeKappaChains Kappa Free Light Chain Level

FreeLambdaChains Lambda Free Light Chain Level

Kappa:LambdaRatio Ratio of Kappa Free Light Chains to Lambda Free Light

Chains

NumberofStones Number of Kidney Stones

Gentamicin-Random Random Sample Gentamicin Level

Gentamicin-Trough Gentamicin Trough Level

Anti-XALevel(LMWH) Factor Xa Level, Low Molecular Weight Heparin

CA19-9 Carbohydrate Antigen 19-9 Level

ACTH Adrenocorticotropic Hormone Level

24HrUrineFreeCortisol Free Cortisol Levels after 24 Hour Fasting Urine Sample

Aldosterone Aldosterone Levels

ReninActivity Renin Secretion Level

Insulin-LikeGrowthFactor1 Insulin-like growth factor 1 Level

Prolactin Prolactin Level

FSH Follicle-Stimulating Hormone Levels

RandomUrineCalcium Random Sample of Urine, Calcium Level

1.25-diOHVitaminD 1,25-Dihydroxyvitamin D Level

25-Hydroxy-VitaminD 25-Hydroxyvitamin D2 and D3 Level

VitaminB1PlasmaLevel Thiamin Level in Plasma

HistoneAntibodies Histone Antibody Level

AngiotensinConv.Enz. Angiotensin Converting Enzyme Levels

Cortisol(PM) Cortisol Levels, Afternoon Testing

BileSalts Bile Acid Levels

ChromograninA Chromogranin A Levels

IGGSubclass1 Immunoglobulin G Subclass 1 Level




FactorXAssay Coagulation Factor X Acitivty Assay Results

LithiumLevel Lithium Level

CRP-Cardiovascularrisk C-Reactive Protein

Amylase(Fluid) Amylase Level in Fluid

24HrUrineSodium Sodium, 24 Hour Fasting Urine

24HrUrinePotassium Potassium, 24 Hour Fasting Urine

24HrUrineChloride Chloride, 24 Hour Fasting Urine

24hrUrineAldosterone Aldosterone, 24 Hour Fasting Urine

24HrUrineMetanephrine:Total Total Metanephrines, 24 Hour Fasting Urine

MetanephrinesTotal/CreatRatio Total Metanephrines to Creatinine Ratio

Normetanephrine Normetanephrine Level

Normetanephrine/CreatRatio Normetanephrine to Creatinine Ratio

Metanephrine Metanephrine Level

24hrUrineMet/CreatRatio Metanephrine to Creatinine Ratio, 24 Hour Fasting Urine

24HrUrineEpinephrine:Total Total Epinephrine Level, 24 Hour Fasting Urine

24HrUrineEPT/CreatRatio Epinephrine to Creatinine Ratio, 24 Hour Fasting Urine

24HrUrineNorepinephrine:Total Total Norepinephrine Level, 24 Hour Fasting Urine

24HrUrineNor/CreatRatio Norepinephrine to Creatinine Ratio, 24 Hour Fasting

Urine

Beta2Microglobin β2 Microglobulin Level


B-hCG Beta Human Chorionic Gonadotropin Level

Gentamicin-Peak Peak Gentamicin Level

Tobramycin(Pre) Pre-Dose Tobramycin Level

CD4/CD8Ratio CD4 to CD8 T-Cell Ratio

CD3+TCELLS(SI) CD3+ T-Cell Levels (syncitia-inducing)

CD3+CD4+HELPERTCELLS(SI) CD3+, CD4+ Helper T-Cell Levels (syncitia-inducing)

CD3+CD8+CYTOTOXICCELLS(SI) CD3+, CD8+ Cytotoxic T-Cell Levels (syncitia-

inducing)

CD19+BCELLS(SI) CD19+ B-Cell Levels (syncitia-inducing)

CD3-CD56+NKCELLS(SI) CD3-, CD56+ NK T-Cel Levels (syncitia-inducing)

CD3+TCELLS(ABS) CD3+ T-Cell Levels (absolute)

CD3+CD4+HELPERTCELLS(ABS) CD3+, CD4+ Helper T-Cell Levels (absolute)

CD3+CD8+CYTOTOXICCELLS(ABS) CD3+, CD8+ Cytotoxic T-Cell Levels (absolute)

CD19+BCELLS(ABS) CD19+ B-Cell Levels (absolute)

CD3-CD56+NKCELLS(ABS) CD3-, CD56+ NK T-Cel Levels (absolute)

FactorVAssay Coagulation Factor V Activity Assay

FactorVIIIAssay Coagulation Factor VIII Activity Assay

Estradiol(Serum) Estradiol Levels

GliadinAntibody-IgA Gliadin (Deamidated) Antibody Immunoglobuin A

GliadinAntibody-IgG Gliadin (Deamidated) Antibody Immunoglobuin G

24HrUrineLead Lead Level, 24 Hour Fasting Urine

OralGTT(Fasting) Oral Glucose Tolerance Test, Post Fasting

OralGTT(2Hour) Oral Glucose Tolerance Test, Post 2 Hour Interval

Gastrin Gastrin Level

TBII Thyroid Stimulating Hormone-Binding Inhibiting Im-

munoglobulin Level

HbF Fetal Hemoglobin Level

ErythropoietinAssay Erythropoietin Assay Test

TroponinT Troponin-T Level

CK-MBQuantitative Creatine Kinase MB Isoenzyme Level

CKRI Creatine Kinase Isoenzyme Reflex Level

EDTAPlateletCount Ethylenediaminetetraacetic acid produced Platelet

Count

HeparinPlateletCount Heparin produced Platelet Count

CitratedPlateletCount Sodium Citrate produced Platelet Count

TissueTransglutaminaseIgG Tissue Transglutaminase (tTG) Antibody, Immunoglob-

ulin A

Ammonia Ammonia Level

Tobramycin(Post) Post-dose Tobramycin Level

AntiAcetylcholineAB Anti-Acetylcholine Antibody B Level

LamotrigineLevel Lamotrigine Level

HIVViralLoadTest Viral Load Test, HIV

24HrUrine5Hydroxyindoleacetic 5-Hydroxyindoleacetic Acid Level, 24 Hour Fasting Urine

24HrUrineCalcium Calcium Level, 24 Hour Fasting Urine

VitaminB6PlasmaLevel Vitamin B6 Level, Plasma

Insulin Insulin Level

Amikacin(Random) Random Sample Amikacin Level

Copper Copper Level

RandomUrineUrea Random Sample Urine Urea

TotalPSA Total Prostate-Specific Antigen Level


FreePSA/TotalPSA Free to Total Prostate-Specific Antigen Ratio

PTT(paeds) Partial Thromboplastin Time, Paediatric

PTT1:1Mix(paeds) Partial Thromboplastin Time 1:1 Plasma Mix Test, Pae-

diatric

GrowthHormone Human Growth Hormone Level

Fluid-WBC White Blood Cell Count, Fluid

CSFWBC White Blood Cell Count, CSF

Amikacin(Pre) Pre-Dose Amikacin Levels

Amikacin(Post) Post-Dose Amikacin Levels

StoolChloride Chloride Level, Stool Sample

StoolPotassium Potassium Level, Stool Sample

StoolSodium Sodium Level, Stool Level

24HrUrinePhos. Phosphate Level, 24 Hour Fasting Urine

CD3 CD3 Antigen Levels



AbsoluteCD4Count Absolute CD4 Antigen Count

AbsoluteCD8Count Absolute CD8 Antigen Count

AlphaGalactosidaseLevel Alpha Galactosidase Level

TacrolimusLevel(FK506) Tacrolimus Level

FreeTestosterone Free Testosterone Level

CD19 CD19 Antigen Level






FactorVIIIvWFAntigen Coagulation Factor VIII with Willebrandt Factor Anti-

gen LEvel

Norepinephrine(plasma) Norepinephrine Level, Plasma

Epinephrine(plasma) Epinephrine Level, Plasma

FecalWeight Weight, Fecal Sample

FecalFat-Screen Fat Type Screen, Fecal Sample

FecalFat-Quant. Fat Quantity, Fecal Sample

C-Peptide C-Peptide Levels

SerumIgE Immunoglobulin E Levels

Tobramycin(Random) Random Sample, Tobramycin

Beta-OH-butyrate Beta Hydroxy-butyrate Level

Gentamicin(Peak) Peak Gentamicin Level

Gentamicin(Trough) Gentamicin Trough Level




VMA:Total Total Vanillylmandelic Acid Level

VMA/CreatRatio Vanillylmandelic Acid to Creatinine Ratio








Carnitine-Free Free Carnitine Level

Carnitine-Total Total Carnitine Level

D-Dimers(Quantitative) Quantity of D-Dimer Assay

Neut% Neutrophil Percentage

Lymp% Lymphocyte Percentage

Mono% Monocyte Percentage

Eos% Eosinophil Percentage

Baso% Basophil Percentage

Gentamicin(Random) Random Sample Gentamicin Level

Seg.Neutrophils Segmented Neutrophil Count

Lymphocytes Lymphocytes

Monocytes Monocytes

Eosinophils Eosinophils

Basophils Basophils

Retic% Reticulocyte Percentage

Atyp/ReactLymphs Atypical and Reactive Lymphocyte Level

Vancomycin(2hrpost) Vancomycin, 2 Hour Post-dose Level

Creatinine(Urine) Creatinine Level in Urine

C1EsteraseInhibitors C1 Esterase Inhibitor Levels

NucleatedRBC-% Nucleated Red Blood Cell Percentage

NucleatedRBC-# Nucleated Red Blood Cell Count

CorrectedWBC Corrected White Blood Cell Count

Neutrophils(cwbc) Neutrophils in Corrected White Blood Cell Count

Lymphocytes(cwbc) Lymphocytes in Corrected White Blood Cell Count

Monocytes(cwbc) Monocytes in Corrected White Blood Cell Count

Eosinophils(cwbc) Eosinophils in Corrected White Blood Cell Count

Basophils(cwbc) Basophils in Corrected White Blood Cell Count

Blasts(cwbc) Blast cells in Corrected White Blood Cell Count

TotalHemoComp-CH50 Total Hemolytic Complement: CH50 Levels

Anti-cardiolipinIgA Anti-Cardiolipin Immunoglobulin A Level

Gentamicin(HighDose) High Dose Level Gentamicin

HLADR Human Leukocyte Antigen - DR Bound Level


Glucose(2HrPC) 2 Hour post-continuous Glucose Levels

CKTotal Total Creatinine Kinase Level


Lead-Blood Lead Level, Blood

ZincProtoporphyrin Zinc Protoporphyrin Level

24HrUrineCopper Copper Level, 24 Hour Fasting Urine

CarcinoembryonicAg. Carcinoembryonic Antigen Level

Table B.3: Description of Clinical and Laboratory Variables Present in

Query from TEGH Clinical Information System

Parameter Mean Standard Deviation Median Absolute Deviation

SystolicBP 120.833434350089 22.7956453693174 20.7564

DiastolicBP 67.5842185128983 20.9933812654242 10.3782

WBC 11.7412983577648 9.04253663671684 4.59606

RBC 3.40329049787047 0.697526630489211 0.696822


Hb 99.1090835658687 19.3853532068544 19.2738

HCT 0.306935957524716 0.0587713002142202 0.0578214

MCV 90.7621346746953 7.66109610907191 6.37518

MCH 29.3123476281392 2.72935882167948 2.2239

MCHC 322.967359377295 13.7140992610458 13.3434

RDW 16.5001174872416 2.47575643748148 2.37216

Platelets 287.520161734975 160.240924490162 139.3644

MPV 8.84752369755309 1.21638212878819 1.03782

Neut# 8.97054248340339 5.28340588468389 4.15128

Lymp# 1.34422752091763 1.47840114633656 0.59304

Mono# 0.570797815924893 0.393475379942953 0.29652

Eos# 0.161876890442707 0.225507626453929 0.14826

Baso# 0.0159203393825124 0.0681570383541951 0

Urea 10.5729164121982 8.51472475315813 5.63388

Sodium 138.162408704725 6.11980223629976 4.4478

Potassium 3.8597046541185 0.641187398971561 0.59304

Chloride 103.072714805319 7.22644927210536 5.9304

TotalCO2 26.089701888932 6.25338429678608 5.9304

Creatinine 117.707180433149 110.536430780344 47.4432

EstimatedGFR 85.486439038372 63.6927174633872 51.891

AnionGap 9.01033576293464 3.7307451350787 2.9652

INR 1.56507211273808 0.857546550447261 0.29652

PTT 36.5303602556653 16.0446399804079 7.413

Calcium 2.02957237056865 0.229244546486296 0.207564

Phosphorus 1.08744539708367 0.411268528516118 0.311346

TroponinT(HighSens.) 194.490864799026 483.377069001693 65.2344

Magnesium 0.847593248198255 0.162226922681222 0.118608

Glucose(Random) 7.98356643356643 3.39200669895832 2.2239

SpecificGravity(Urine) 1.01741391941392 0.00556691733109209 0.00741299999999984

UrinepH 6.00913242009132 0.862921697299477 0.7413

UrineProtein 1.285 2.77941596253305 0.459606

Urobilinogen 6.73178733031674 10.1223185065456 0

L-LacticAcid-Plasma 2.36510926902788 2.51832598130336 0.88956

TotalProtein 56.0068617173354 10.2354734474902 10.3782

Albumin 25.4174298589911 6.18699955229053 5.9304

pHArterial 7.37119301557338 0.109911028722189 0.0889560000000007

pCO2Arterial 44.5184066452709 15.6601332228606 11.8608

pO2Arterial 93.6388705260176 39.7951439185091 23.7216

HCO3Arterial 25.0996884735202 7.53981581998637 7.413

FiO2 45.737536117678 21.524283558003 14.826

O2SaturationArterial 95.9610525315855 5.21617093750605 2.9652

pHVenous 7.33100686498856 0.120463566040509 0.0963689999999999

pCO2Venous 46.6468571428571 16.1785616423696 10.3782

pO2Venous 48.6251428571429 27.7460089870722 10.3782

BaseExcessVenous 6.01932773109244 5.1282530477801 4.4478

HCO3Venous 24.1245714285714 7.84076521874172 7.413

O2SaturationVenous 73.7752293577982 14.6051728310929 13.3434

Carboxy-Hb 1.51500200160128 0.562979014555377 0.59304

ALT 101.334580438268 365.289301167569 25.2042

AST 129.235505713077 636.416101638168 25.2042


ALP 138.340361131406 169.431158411747 53.3736

TotalBilirubin 21.5604151223128 47.8436966126988 5.9304

Amylase(Serum) 103.652567975831 214.713857598555 37.065

D-DimerFEU 1976.57831325301 1478.68218456117 1442.5698

Glucose(Fasting) 7.56519337016575 3.50107158498118 2.2239

BaseExcessArterial 5.93439878234399 4.67773275934711 4.4478

Retic# 90.6409214092141 61.9621126367522 48.77754

Ferritin 519.881091617934 917.004913413121 238.6986

ProteinElectrophoresis 57.0833333333333 10.0685261160751 10.3782

Iron 7.5679012345679 5.89835157288458 2.9652

Transferrin 1.58612903225806 0.555956294231441 0.555975

TransferrinSaturation 0.22432664756447 0.202782134677068 0.103782

Vit.B12 573.144050104384 320.667834307238 296.52

RBCFolate 3039.69565217391 4555.6383595544 849.5298

24HrUrineCreat. 36.5240384615385 307.5389376232 2.89107

24HrUrineProtein 3.18287356321839 16.0839070883916 0.726474

AlbuminElectrophoresis 24.1315789473684 5.60975648884198 5.9304

Alpha1Globulin 3.22105263157895 0.988601142745386 1.4826

Alpha2Globulin 9.04736842105263 2.25206981130167 2.9652

BetaGlobulin 8.33684210526316 4.34153199088149 1.4826

GammaGlobulin 12.3368421052632 6.81441131831276 4.4478

M-Spike 6.60606060606061 8.82092467495399 4.4478

A\GRatio 0.767578947368421 0.238190781164365 0.237216

HbA1C 0.0687803468208092 0.0173667436943727 0.0118608

IgG 11.8478409090909 9.62567274243801 3.936303

IgA 4.34494252873563 10.7909783457965 1.497426

IgM 0.827011494252874 0.76313020459727 0.489258

UricAcid 464.613722998729 194.545966401239 204.5988

HbA2 0.0321379310344828 0.00985510795016268 0.0088956

Albumin(Urine) 53.1521739130435 23.8895303812308 19.2738

Alpha-1Globulin(Urine) 7.51282051282051 5.67947270936398 2.9652

Alpha-2Globulin(Urine) 10.7179487179487 4.6336794402716 2.9652

BetaGlobulin(Urine) 15.9487179487179 11.6617880680738 5.9304

GammaGlobulin(Urine) 21.9473684210526 13.6737020598944 10.3782

M-Spike(Urine) 33.3 23.1902757398201 32.6172

HbA 0.967620689655172 0.00972263115880915 0.00889560000000001

Cholesterol 2.7778637510513 1.21344064277649 1.008168

Triglycerides 1.41924559932942 1.21897708555172 0.504084

HDL 0.961797520661157 0.486609804437168 0.459606

LDL 1.62325630252101 0.853709987403084 0.726474

CHOL/HDLRatio 4.13801652892562 3.04751892972881 1.4826

CPK 962.997509782995 4334.53803130549 148.26

TSH(2ndGeneration) 5.12034280936455 17.8510009320399 1.897728

Neutrophils(wbc) 13.7800085984523 13.5728717478578 10.82298

Lymphocytes(wbc) 1.09423903697334 2.88550122763181 0.59304

AtypicalLymphocytes 0.00421810699588477 0.0448329174858177 0

Monocytes(wbc) 0.934780739466896 2.71400796650023 0.44478

Eosinophilswbc) 0.127162510748065 0.406790465505535 0

Basophils(wbc) 0.0269647463456578 0.18659204089239 0

Bands 2.8690715835141 6.33286064618965 1.275036


Metamyelocytes 0.214238005644403 0.544362991165073 0

Myelocytes 0.165118371212121 0.439583137876394 0

Promyelocytes 0.0153027989821883 0.148824218737385 0

Blasts(wbc) 0.117007738607051 1.42117455192417 0

LDH 463.040229885057 1154.0624417311 128.9862

PhenytoinLevel 32.9028925619835 23.3963259113461 22.239

ValproicAcidLevel 311.478571428571 185.291544254036 174.2055

PhenobarbitalLevel 46 23.659036328642 28.1694

Others 1.64626865671642 2.87956901016554 0.44478

NRBC’s/100WBC’s 5.62307692307692 9.1191784401948 2.9652

FibrinogenLevelAssay 2.77512987012987 1.51853423970303 1.55673

UrineOsmolality 425.272727272727 180.135949513967 164.5686

RandomUrineSodium 63.2585812356979 43.5250581712211 40.0302

RandomUrinePotassium 38.9232600732601 20.6531324499366 22.31313

RandomUrineChloride 65.8991228070175 45.0642852093167 42.9954

AlphaFetoprotein 4337.2 25641.4731140554 1.4826

FreeT3 3.01618625277162 1.88019611291232 0.88956

FreeT4 16.2248603351955 8.5197259785569 4.4478

Cortisol(AM) 869.714912280702 1009.06408084324 296.52

PlasmaOsmolality 295 42.8812313256044 21.4977

Fluid-RBC 8391.70289855072 20378.2481036128 2882.9157

NucleatedCell(BodyFluid) 27564.6 128218.328262134 378.063

Neutrophils(BodyFluid) 0.299130434782609 0.373531395366987 0.103782

Lymphocytes(BodyFluid) 0.205217391304348 0.286303051573247 0.103782

Monocytes(BodyFluid) 0.0820289855072464 0.172008718231039 0.014826

Eosinophils(BodyFluid) 0.000869565217391304 0.00612111351720723 0

NonHematopoieticCells(BodyFluids) 0.00985507246376812 0.0570987986281155 0

OtherCells(BodyFluid) 0.0405797101449275 0.101255124169115 0

DirectBilirubin 28.8582677165354 48.6290990698004 4.4478

Anti-cardiolipinIgG 1.83333333333333 1.99274044791428 1.4826

Anti-cardiolipinIgM 1.30434782608696 1.29456144236697 1.4826

Creatine(Fluid) 937.888888888889 2522.35876883725 93.4038

PlasmaFolate 24.5526627218935 9.46361494084906 10.3782

AcetaminophenLevel 54.3717948717949 115.892819827829 16.3086

Salicylate 1.03857142857143 1.23524433673204 0.526323

EthanolLevel 45 0

ParathyroidH.-Intact 16.539837398374 27.9326084189805 8.00604

DigoxinLevel 1.20347826086957 0.69483437819365 0.44478

anti-HBs 1068.53333333333 3599.45560025149 17.7912

SedimentationRate 59.6216216216216 35.7396452881425 48.1845

CA125 252.166666666667 361.08569452123 127.5036

Lipase(Serum) 608.441176470588 897.535115922338 137.1405

Cortisol(Random) 1108.45238095238 911.828161024924 312.8286

Vancomycin(Pre) 16.7002375296912 8.05831277038501 8.30256

Vancomycin(Random) 19.8189054726368 9.79659332605064 8.59908

Temperature 36.2511644086643 1.51254936726002 0.7413

Pulse 86.9617400947676 25.4591715175526 17.7912

VSFiO2 0.894406901138723 5.4065482774366 0.103782

MEWSAVPUScale Alert <1 Level 0 Levels

RespiratoryRate 21.4299987719514 8.07394396932507 5.9304


Weight 76.0532100575536 23.0934738202971 16.01208

Height 162.675073823011 166.076853659531 11.26776

TotalProtein(Fluid) 25.8076923076923 11.9545217140038 11.8608

Albumin(Fluid) 13.5263157894737 6.42519046770058 5.9304

LDH(Fluid) 889.521276595745 2928.34991052421 290.5896

pH(Fluid) 7.71447368421053 0.442558271432234 0.274280999999999

Haptoglobin 2.27076923076923 1.30416676906692 1.33434

GammaGT 90.8913342503439 129.322969051327 47.4432

Anti-thrombinIIIAct 0.73 0.305973855092228 0.163086

ProteinCActivity 0.806666666666667 0.601477347869394 0.696822

APCResistance 2.98333333333333 0.306050104830347 0.22239

ProteinS(Free) 0.925555555555556 0.375436782664909 0.133434

Homocysteine(Plasma) 11.1 9.1098237572902 3.7065

TotalTestosterone 2.81428571428571 0.983918308649269 1.03782

DNAAntibodies 10.7543859649123 14.1929065554321 4.4478

C3 0.996229508196721 0.333107409776618 0.281694

C4 0.239508196721311 0.103801512986857 0.103782

Ceruloplasmin 0.259 0.0956913899223078 0.066717

Alpha-1-Antitrypsin 1.72866666666667 0.288341728476737 0.326172

c ANCA(PR3) 1.4 2.74444894286631 0.14826

p ANCA(MP0) 0.733333333333333 0.923760430703401 0

TotalUrineVolume 1.60988372093023 1.03525557422656 1.11195

UrineProtein/CreatinineRatio(24hrs) 86 0

RandomUrineMicroalbumin 345.57619047619 511.38063016188 124.68666

RandomUrineCreat. 4.84590909090909 3.98606369812335 2.14977

Microalbumin/CreatinineRatio 67.0428571428571 90.3175208441229 36.17544

Ammonia(Paeds) 51.1 28.9807558653379 32.6172

CyclosporinLevel 184.333333333333 122.696101540894 4.4478

24HrUrineVolume 1.756 1.12512221558371 0.96369

24HrUrineMagnesium 2.2 2.03141986469235 2.44629

24HrUrine-Delta-AminolevulinicAcid 16 0

CreatinineClearance 3.05384615384615 16.1038708350025 0.14826

Anti-GbmAntibodies 4.55 3.46482322781408 3.63237

GabapentineLevel 120 0

Calcium(Ionized) 1.17373333333333 0.208007103135818 0.133434

TheophyllineLevel 50.6666666666667 30.1081384346492 37.065

CRP-AcutePhaseReactant 103.088888888889 107.387949907187 57.0801

Microalbumin24HRUrine 563.333333333333 529.849349658309 502.6014

24HrRatio(Microalbumin/Creatinine) 72.5666666666667 71.5058272683656 54.1149

RheumatoidFactor 14.5483870967742 6.17974492287732 4.4478

Glucose(Fluid) 6.45671641791045 3.03535849216176 2.07564

PSA 38.2893617021277 146.5458708926 2.81694

CEA 27.2 58.6694021616038 6.96822

RandomUrinePhosphorus 9.45 0

Anti-Microsomal 812.25 1496.72049828951 91.1799

CSFGlucose 3.66521739130435 1.89990638421337 1.92738

CSFProtein 1.58041666666667 2.21705342871955 0.555975

CSFRBC 146.727272727273 384.442216960908 41.5128

NucleatedCell(CSF) 54 129.82603745012 1.4826

Neutrophils(CSF) 0.175 0.292318319644869 0


Lymphocytes(CSF) 0.32 0.372886637524656 0.192738

Monocytes(CSF) 0.123 0.20902152999153 0.059304

Eosinophils(CSF) 0 0 0

Basophils(CSF) 0 0 0

NonHematopoieticCells(CSF) 0 0 0

OtherCells(CSF) 0.011 0.0347850542618522 0

Anti-Thyroglobulin 12497.5 16691.2555699085 17498.3865

CarbamazepineLevel 22.8490566037736 13.2045974313365 10.3782

TopiramateLevel 9 4.35889894354067 1.4826

FreeKappaChains 388.10625 1277.15466172204 60.7866

FreeLambdaChains 612.073333333333 1981.93650196204 37.36152

Kappa:LambdaRatio 41.436 151.17655661038 0.845082

Result(1) 0.34 0

NumberofStones 1 0 0

Gentamicin-Random 6.33333333333333 3.95516539898565 5.33736

Gentamicin-Trough 1.69375 1.39879412352211 1.40847

Anti-XALevel(LMWH) 0.404444444444444 0.499177100614379 0.207564

CA19-9 2227.33333333333 2828.90435799209 1240.9362

ACTH 22.4615384615385 31.2932777249241 8.8956

24HrUrineFreeCortisol 3024.375 4287.38431539058 756.126

Aldosterone 191.8 147.278873796165 42.9954

ReninActivity 11.1428571428571 8.94797130506074 13.3434

Insulin-LikeGrowthFactor1 120 0

Prolactin 12.8571428571429 11.2164844674935 7.413

FSH 13 8.8090862182181 11.1195

LH2 8.16666666666667 4.75043857624395 4.4478

RandomUrineCalcium 3.44 0.466690475583121 0.489258

1.25-diOHVitaminD 45.8571428571429 35.741798926134 14.826

25-Hydroxy-VitaminD 45.6666666666667 26.5900984077407 16.3086

VitaminB1PlasmaLevel 47.7142857142857 60.7775021032023 25.2042

HistoneAntibodies 0.407692307692308 0.502302391301985 0.14826

AngiotensinConv.Enz. 34.25 13.2507861402006 12.6021

Cortisol(PM) 830.973684210526 692.036457672731 532.2534

BileSalts 23.1 0

ChromograninA 224.5 212.839141137151 223.1313

IGGSubclass1 6.646 3.42482554300215 3.40998

IGGSubclass2 3.324 1.71693913695273 1.63086

IGGSubclass3 0.602 0.558990160915199 0.0741300000000001

IGGSubclass4 0.37 0.261511950013761 0.2564898

FactorXAssay 0.675 0.21920310216783 0.229803

LithiumLevel 0.9034375 0.474012449645951 0.467019

CRP-Cardiovascularrisk 148.55 143.87565117142 65.30853

Amylase(Fluid) 1321.90625 4407.803266491 18.5325

24HrUrineSodium 198.666666666667 239.889557921974 186.8076

24HrUrinePotassium 50 18.3439363278441 28.1694

24HrUrineChloride 197.333333333333 224.347944051199 152.7078

24hrUrineAldosterone 13 7.07106781186548 7.413

24HrUrineMetanephrine:Total 3.26923076923077 1.66502483233965 1.18608

MetanephrinesTotal/CreatRatio 0.75 0

Normetanephrine 2.23 0


Normetanephrine/CreatRatio 0.62 0

Metanephrine 0.5 0

24hrUrineMet/CreatRatio 0.5725 0.688979812344442 0.289107

24HrUrineEpinephrine:Total 69.8888888888889 47.1737332751088 31.1346

24HrUrineEPT/CreatRatio 12.7222222222222 13.9436166199607 4.00302

24HrUrineNorepinephrine:Total 437.1 301.862901183817 418.8345

24HrUrineNor/CreatRatio 81.21 96.864572700469 23.20269

Beta2Microglobin 10.52 11.1600627238381 4.15128

B-hCG 19 22.9673681557117 4.4478

Gentamicin-Peak 3.97692307692308 2.66962976401655 1.77912

Tobramycin(Pre) 1.19090909090909 0.740883992882213 1.03782

CD4/CD8Ratio 0.906666666666667 0.786008602885187 1.18608

CD3+TCELLS(SI) 0.774 0.0971081870904817 0.14826

CD3+CD4+HELPERTCELLS(SI) 0.32 0.246272207120495 0.355824

CD3+CD8+CYTOTOXICCELLS(SI) 0.412 0.21844907873461 0.266868

CD19+BCELLS(SI) 0.115 0.0834665601703261 0.00741300000000001

CD3-CD56+NKCELLS(SI) 0.0975 0.10719919153924 0.081543

CD3+TCELLS(ABS) 0.546 0.287715136897592 0.415128

CD3+CD4+HELPERTCELLS(ABS) 0.272 0.272800293254974 0.22239

CD3+CD8+CYTOTOXICCELLS(ABS) 0.256 0.124418648119966 0.118608

CD19+BCELLS(ABS) 0.065 0.0191485421551268 0.014826

CD3-CD56+NKCELLS(ABS) 0.0675 0.067515430335097 0.07413

FactorVAssay 0.335 0.190918830920368 0.200151

FactorVIIIAssay 2.03 0.915678254992804 0.867321

Estradiol(Serum) 730 0

GliadinAntibody-IgA 25 24.0416305603426 25.2042

GliadinAntibody-IgG 14 0 0

24HrUrineLead 5.8 0

OralGTT(Fasting) 5.7 0

OralGTT(2Hour) 6.1 0

Gastrin 151.5 175.442013212343 49.6671

TBII 8.1 0

HbF 0.0035 0.000707106781186548 0.0007413

ErythropoietinAssay 76.7666666666667 104.318566580132 18.82902

TroponinT 0.27465312667023 0.766661612971922 0.0652344

CK-MBQuantitative 12.8650220264317 60.1478000390418 2.52042

CKRI 0.0561862917398946 0.0624045339933707 0.044478

EDTAPlateletCount 151 106.066017177982 111.195

HeparinPlateletCount 103 114.551298552221 120.0906

CitratedPlateletCount 133.5 106.773123959169 111.9363

TissueTransglutaminaseIgG 12 0

Ammonia 65.3333333333333 46.5763726643245 34.0998

Tobramycin(Post) 5.55714285714286 5.08129154935276 1.33434

AntiAcetylcholineAB 5.76 0

LamotrigineLevel 9 0

HIVViralLoadTest 13781.36375 34785.7651434643 50.497356

Viscosity 1.25 0.0707106781186548 0.0741300000000001

CellsExamined 122.333333333333 250.664450344812 0

BandLevel 360 22.3606797749979 0

24HrUrine5Hydroxyindoleacetic 23.5 11.6189500386223 6.6717


24HrUrineCalcium 3.36666666666667 2.73922130054023 2.81694

VitaminB6PlasmaLevel 98.5 64.3467170879758 67.4583

Insulin 223.333333333333 303.582498397607 63.7518

Amikacin(Random) 15.5 7.49132832013122 3.40998

Copper 24.25 9.97020561473032 10.45233

RandomUrineUrea 137.475 31.6165963801714 13.93644

TotalPSA 4.696 4.47317895908491 4.269888

FreePSA/TotalPSA 0.168 0.0679705818718657 0.014826

PTT(paeds) 104.5 68.5893577750951 71.9061

PTT1:1Mix(paeds) 51.5 17.6776695296637 18.5325

GrowthHormone 1.2 0

Fluid-WBC 3014.30107526882 11006.5492912026 410.6802

Fluid-DiffPolymorphs 52.6781609195402 32.1289170095471 44.478

Fluid-DiffMononuclear 46.9285714285714 32.4329839969535 45.9606

CSFWBC 311.666666666667 632.687811378303 78.5778

CSFDiffPolymorphs 40.5 36.3964283962039 47.4432

CSFDiffMononuclear 59.5 36.3964283962039 47.4432

Amikacin(Pre) 14.8090909090909 19.7094116880974 5.63388

Amikacin(Post) 17.5818181818182 5.7613918772842 5.63388

StoolChloride 60 36.3455636907725 32.6172

StoolPotassium 73.3333333333333 45.6216322958017 2.9652

StoolSodium 60.5 54.4472221513642 57.0801

24HrUrinePhos. 27.8 0

CD3 70.7142857142857 20.7602748293227 8.8956

CD4 32.7142857142857 25.339019990243 21.4977

CD8 42.3846153846154 21.473000339723 23.7216

AbsoluteCD4Count 0.244 0.191090438158364 0.281694

AbsoluteCD8Count 0.443 0.3990558301235 0.155673

AlphaGalactosidaseLevel 22 0

TacrolimusLevel(FK506) 5.1 0

FreeTestosterone 4.45 0.353553390593274 0.37065

CD19 12.25 8.30160627027485 5.9304

CD20 16.6666666666667 4.16333199893227 2.9652

CD10 27.3333333333333 45.6106712659804 0

CD5 57.25 30.2365672654817 3.7065

CD7 74.6666666666667 4.61880215351701 0

CD2 60.75 39.4831187555728 8.8956

FactorVIIIvWFAntigen 3.70333333333333 2.11776611865742 0.622692

Norepinephrine(plasma) 2.8 0

Epinephrine(plasma) 2.1 0

FecalWeight 868 513.359523141434 538.1838

FecalFat-Screen 3.8 2.82842712474619 2.9652

FecalFat-Quant. 36.9 49.6388960392956 52.03926

C-Peptide 412 53.7401153701776 56.3388

SerumIgE 2014.66666666667 2903.33606965045 81.543

Tobramycin(Random) 3 0

Beta-OH-butyrate 0.07 0

Gentamicin(Peak) 6.375 4.30055493247938 2.14977

Gentamicin(Trough) 1.66666666666667 1.20629661159746 0.51891

CD45 80 14.142135623731 14.826


CD38 31.5 10.6066017177982 11.1195

CD56 10.6666666666667 1.52752523165195 1.4826

VMA:Total 17.95 14.0714249456123 14.75187

VMA/CreatRatio 3 1.13137084989848 1.18608

CD34 49 63.6396103067893 66.717

CD13 60 52.3259018078045 54.8562

CD33 97 0

CD15 78.5 21.920310216783 22.9803

CD61 7 0

CD41 4 4.24264068711928 4.4478

Carnitine-Free 62 0

Carnitine-Total 83 0

D-Dimers(Quantitative) 518.925373134328 535.916838681911 271.3158

Neut% 0.785582304526749 0.124310034111559 0.103782

Lymp% 0.141141975308642 0.101742952218846 0.07413

Mono% 0.0535720164609053 0.0292099080270319 0.014826

Eos% 0.0166666666666667 0.0237561530032861 0.014826

Baso% 0.00115843621399177 0.00525101552551825 0

Gentamicin(Random) 11.9 0

Seg.Neutrophils 0.611179245283019 0.243815273141211 0.266868

Lymphocytes 0.0997641509433962 0.135928437011549 0.044478

Monocytes 0.0610141509433962 0.108124609785921 0.029652

Eosinophils 0.015377358490566 0.0487027851651263 0

Basophils 0.00212264150943396 0.00773216282433982 0

Retic% 2.801 2.06832575995063 1.912554

Atyp/ReactLymphs 0.0333333333333333 0.0215251790546173 0.014826

Vancomycin(2hrpost) 42.2333333333333 21.8220836157626 8.74734

Creatinine(Urine) 12 0

C1EsteraseInhibitors 0.39 0

NucleatedRBC-% 0.14625 0.0351899606895679 0.022239

NucleatedRBC-# 1.8125 1.18988794990677 0.66717

CorrectedWBC 17.4414634146341 11.7903345077601 10.08168

Neutrophils(cwbc) 14.7731707317073 11.442945957887 8.45082

Lymphocytes(cwbc) 1.74146341463415 3.41467243009898 0.7413

Monocytes(cwbc) 0.624390243902439 0.800868431081185 0.44478

Eosinophils(cwbc) 0.133658536585366 0.362744786989151 0

Basophils(cwbc) 0.0419512195121951 0.181152139266903 0

Blasts(cwbc) 0.065609756097561 0.393789593441014 0

Other 35.0909090909091 35.1637726800028 23.7216

Blast 0.264 0.213143144388929 0.22239

TotalHemoComp-CH50 114 0

Anti-cardiolipinIgA 2 0

Gentamicin(HighDose) 3.4 0

HLADR 58 0

CD14 1 0

Glucose(2HrPC) 5.2 0

CKTotal 71 0

CD23 15 0

Lead-Blood 0.11 0

ZincProtoporphyrin 65 0


24HrUrineCopper 0.81 0

CarcinoembryonicAg. 1.2 0

Table B.4: Description of Numeric Clinical and Laboratory Variable

Values Present in Query from TEGH Clinical Information System

Threshold Num-

ber of Encounters

Max(2078)

Percentage of Variables

with <= Threshold

Number of Encounters

Containing Data

Number of Variables with

> Threshold Number of

Encounters Max(147)

Variables with > Threshold Number

of Encounters

161 70.15 123 A\GRatio, Albumin, AlbuminElectrophoresis, ALP, Alpha1Globulin, Alpha2Globulin,

ALT, Amylase(Serum), AnionGap, AST, AtypicalLymphocytes, Bands, BaseExcessAr-

terial, BaseExcessVenous, Baso#, Baso%, Basophils(wbc), BetaGlobulin, Blasts(wbc),

Calcium, Carboxy-Hb, Chloride, Cholesterol, CHOL/HDLRatio, CK-MBQuantitative,

CKRI, Cortisol(AM), CPK, Creatinine, DiastolicBP, Eos#, Eos%, Eosinophilswbc), Es-

timatedGFR, Ferritin, FiO2, FreeT3, FreeT4, GammaGlobulin, Glucose(Fasting), Glu-

cose(Random), Hb, HbA1C, HCO3Arterial, HCO3Venous, HCT, HDL, Height, INR,

Iron, LDH, LDL, L-LacticAcid-Plasma, Lymp#, Lymp%, Lymphocytes(wbc), Magne-

sium, MCH, MCHC, MCV, Metamyelocytes, MEWSAVPUScale, Mono#, Mono%, Mono-

cytes(wbc), MPV, Myelocytes, Neut#, Neut%, Neutrophils(wbc), NRBC’s/100WBC’s,

O2SaturationArterial, O2SaturationVenous, pCO2Arterial, pCO2Venous, pHArterial,

Phosphorus, pHVenous, PlasmaFolate, Platelets, pO2Arterial, pO2Venous, Potassium,

Promyelocytes, ProteinElectrophoresis, PTT, Pulse, RandomUrineChloride, Rando-

mUrinePotassium, RandomUrineSodium, RBC, RDW, RespiratoryRate, Retic#, Sodium,

SpecificGravity(Urine), SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalPro-

tein, Transferrin, TransferrinSaturation, Triglycerides, TroponinT, TroponinT(HighSens.),

TSH(2ndGeneration), Urea, UricAcid, UrinepH, UrineProtein, Urobilinogen, Van-

comycin(Pre), Vit.B12, VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF,

Age

181 70.87 120 A\GRatio, Albumin, AlbuminElectrophoresis, ALP, Alpha1Globulin, Alpha2Globulin,

ALT, Amylase(Serum), AnionGap, AST, AtypicalLymphocytes, Bands, BaseExcessAr-

terial, Baso#, Baso%, Basophils(wbc), BetaGlobulin, Blasts(wbc), Calcium, Carboxy-

Hb, Chloride, Cholesterol, CHOL/HDLRatio, CK-MBQuantitative, CKRI, CPK, Cre-

atinine, DiastolicBP, Eos#, Eos%, Eosinophilswbc), EstimatedGFR, Ferritin, FiO2,

FreeT3, FreeT4, GammaGlobulin, Glucose(Fasting), Glucose(Random), Hb, HbA1C,

HCO3Arterial, HCO3Venous, HCT, HDL, Height, INR, Iron, LDH, LDL, L-LacticAcid-

Plasma, Lymp#, Lymp%, Lymphocytes(wbc), Magnesium, MCH, MCHC, MCV,

Metamyelocytes, MEWSAVPUScale, Mono#, Mono%, Monocytes(wbc), MPV, Mye-

locytes, Neut#, Neut%, Neutrophils(wbc), NRBC’s/100WBC’s, O2SaturationArterial,

O2SaturationVenous, pCO2Arterial, pCO2Venous, pHArterial, Phosphorus, pHVenous,

PlasmaFolate, Platelets, pO2Arterial, pO2Venous, Potassium, Promyelocytes, Protein-

Electrophoresis, PTT, Pulse, RandomUrineChloride, RandomUrinePotassium, Rando-

mUrineSodium, RBC, RDW, RespiratoryRate, Retic#, Sodium, SpecificGravity(Urine),

SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein, Transferrin, Trans-

ferrinSaturation, Triglycerides, TroponinT, TroponinT(HighSens.), TSH(2ndGeneration),

Urea, UricAcid, UrinepH, UrineProtein, Urobilinogen, Vit.B12, VSFiO2, WBC, Weight,

CART, MEWS, NEWS, ViEWS, CDF, Age


201 72.82 112 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, AtypicalLymphocytes, Bands,

BaseExcessArterial, Baso#, Baso%, Basophils(wbc), Blasts(wbc), Calcium, Carboxy-

Hb, Chloride, Cholesterol, CHOL/HDLRatio, CK-MBQuantitative, CKRI, CPK, Creati-

nine, DiastolicBP, Eos#, Eos%, Eosinophilswbc), EstimatedGFR, Ferritin, FiO2, FreeT3,

FreeT4, Glucose(Fasting), Glucose(Random), Hb, HbA1C, HCO3Arterial, HCO3Venous,

HCT, HDL, Height, INR, Iron, LDH, LDL, L-LacticAcid-Plasma, Lymp#, Lymp%,

Lymphocytes(wbc), Magnesium, MCH, MCHC, MCV, Metamyelocytes, MEWSAV-

PUScale, Mono#, Mono%, Monocytes(wbc), MPV, Myelocytes, Neut#, Neut%,

Neutrophils(wbc), NRBC’s/100WBC’s, O2SaturationArterial, O2SaturationVenous,

pCO2Arterial, pCO2Venous, pHArterial, Phosphorus, pHVenous, Platelets, pO2Arterial,

pO2Venous, Potassium, Promyelocytes, PTT, Pulse, RandomUrineChloride, Rando-

mUrinePotassium, RandomUrineSodium, RBC, RDW, RespiratoryRate, Retic#, Sodium,

SpecificGravity(Urine), SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalPro-

tein, Transferrin, TransferrinSaturation, Triglycerides, TroponinT, TroponinT(HighSens.),

TSH(2ndGeneration), Urea, UricAcid, UrinepH, UrineProtein, Urobilinogen, Vit.B12, VS-

FiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF, Age

221 73.3 110 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, AtypicalLymphocytes,

Bands, BaseExcessArterial, Baso#, Baso%, Basophils(wbc), Blasts(wbc), Calcium,

Carboxy-Hb, Chloride, Cholesterol, CHOL/HDLRatio, CK-MBQuantitative, CKRI,

CPK, Creatinine, DiastolicBP, Eos#, Eos%, Eosinophilswbc), EstimatedGFR, Fer-

ritin, FiO2, FreeT3, FreeT4, Glucose(Fasting), Glucose(Random), Hb, HbA1C,

HCO3Arterial, HCO3Venous, HCT, HDL, Height, INR, Iron, LDL, L-LacticAcid-





Platelets, pO2Arterial, pO2Venous, Potassium, Promyelocytes, PTT, Pulse, Rando-

mUrineChloride, RandomUrinePotassium, RandomUrineSodium, RBC, RDW, Respira-

toryRate, Sodium, SpecificGravity(Urine), SystolicBP, Temperature, TotalBilirubin, To-

talCO2, TotalProtein, Transferrin, TransferrinSaturation, Triglycerides, TroponinT, Tro-

poninT(HighSens.), TSH(2ndGeneration), Urea, UricAcid, UrinepH, UrineProtein, Uro-

bilinogen, Vit.B12, VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF, Age






































FreeT4, Glucose(Fasting), Glucose(Random), Hb, HbA1C, HCO3Arterial, HCO3Venous,

HCT, HDL, Height, INR, Iron, LDL, L-LacticAcid-Plasma, Lymp#, Lymp%, Lym-

phocytes(wbc), Magnesium, MCH, MCHC, MCV, Metamyelocytes, MEWSAVPUS-

cale, Mono#, Mono%, Monocytes(wbc), MPV, Myelocytes, Neut#, Neut%, Neu-

trophils(wbc), O2SaturationArterial, O2SaturationVenous, pCO2Arterial, pCO2Venous,

pHArterial, Phosphorus, pHVenous, Platelets, pO2Arterial, pO2Venous, Potassium,

Promyelocytes, PTT, Pulse, RandomUrineChloride, RandomUrinePotassium, Rando-

mUrineSodium, RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine), Sys-

tolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein, Transferrin, Transferrin-

Saturation, Triglycerides, TroponinT, TroponinT(HighSens.), TSH(2ndGeneration), Urea,

UricAcid, UrinepH, UrineProtein, Urobilinogen, Vit.B12, VSFiO2, WBC, Weight, CART,

MEWS, NEWS, ViEWS, CDF, Age





FreeT3, FreeT4, Glucose(Fasting), Glucose(Random), Hb, HCO3Arterial, HCO3Venous,


phocytes(wbc), Magnesium, MCH, MCHC, MCV, Metamyelocytes, MEWSAVPUScale,

Mono#, Mono%, Monocytes(wbc), MPV, Myelocytes, Neut#, Neut%, Neutrophils(wbc),


Phosphorus, pHVenous, Platelets, pO2Arterial, pO2Venous, Potassium, Promyelocytes,

PTT, Pulse, RandomUrineChloride, RandomUrinePotassium, RandomUrineSodium,

RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine), SystolicBP, Tempera-

ture, TotalBilirubin, TotalCO2, TotalProtein, Transferrin, Triglycerides, TroponinT, Tro-










phocytes(wbc), Magnesium, MCH, MCHC, MCV, Metamyelocytes, MEWSAVPUS-

cale, Mono#, Mono%, Monocytes(wbc), MPV, Myelocytes, Neut#, Neut%, Neu-

trophils(wbc), O2SaturationArterial, O2SaturationVenous, pCO2Arterial, pCO2Venous,

pHArterial, Phosphorus, pHVenous, Platelets, pO2Arterial, pO2Venous, Potassium,

Promyelocytes, PTT, Pulse, RandomUrineChloride, RandomUrinePotassium, Rando-

mUrineSodium, RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine), Sys-

tolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein, Triglycerides, TroponinT,

TroponinT(HighSens.), TSH(2ndGeneration), Urea, UricAcid, UrinepH, UrineProtein,

Urobilinogen, Vit.B12, VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF,

Age






HCT, HDL, Height, INR, LDL, L-LacticAcid-Plasma, Lymp#, Lymp%, Lympho-

cytes(wbc), Magnesium, MCH, MCHC, MCV, Metamyelocytes, MEWSAVPUScale,

Mono#, Mono%, Monocytes(wbc), MPV, Myelocytes, Neut#, Neut%, Neutrophils(wbc),


Phosphorus, pHVenous, Platelets, pO2Arterial, pO2Venous, Potassium, Promyelocytes,

PTT, Pulse, RandomUrineChloride, RandomUrinePotassium, RandomUrineSodium,

RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine), SystolicBP, Temperature,

TotalBilirubin, TotalCO2, TotalProtein, Triglycerides, TroponinT, TroponinT(HighSens.),

TSH(2ndGeneration), Urea, UrinepH, UrineProtein, Urobilinogen, Vit.B12, VSFiO2,

WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF, Age





Glucose(Fasting), Glucose(Random), Hb, HCO3Arterial, HCO3Venous, HCT, HDL,

Height, INR, LDL, L-LacticAcid-Plasma, Lymp#, Lymp%, Lymphocytes(wbc), Magne-


cytes(wbc), MPV, Myelocytes, Neut#, Neut%, Neutrophils(wbc), O2SaturationArterial,


Platelets, pO2Arterial, pO2Venous, Potassium, Promyelocytes, PTT, Pulse, RBC,

RDW, RespiratoryRate, Sodium, SpecificGravity(Urine), SystolicBP, Temperature, To-

talBilirubin, TotalCO2, TotalProtein, Triglycerides, TroponinT, TroponinT(HighSens.),








FreeT4, Glucose(Fasting), Glucose(Random), Hb, HCO3Arterial, HCO3Venous, HCT,

HDL, INR, LDL, L-LacticAcid-Plasma, Lymp#, Lymp%, Lymphocytes(wbc), Magne-


cytes(wbc), MPV, Myelocytes, Neut#, Neut%, Neutrophils(wbc), O2SaturationArterial,

pCO2Arterial, pCO2Venous, pHArterial, Phosphorus, pHVenous, Platelets, pO2Arterial,

pO2Venous, Potassium, Promyelocytes, PTT, Pulse, RBC, RDW, RespiratoryRate,

Sodium, SpecificGravity(Urine), SystolicBP, Temperature, TotalBilirubin, TotalCO2, To-

talProtein, Triglycerides, TroponinT, TroponinT(HighSens.), TSH(2ndGeneration), Urea,

UrinepH, UrineProtein, Urobilinogen, Vit.B12, VSFiO2, WBC, Weight, CART, MEWS,

NEWS, ViEWS, CDF, Age


BaseExcessArterial, Baso#, Basophils(wbc), Blasts(wbc), Calcium, Carboxy-Hb, Chlo-

ride, Cholesterol, CHOL/HDLRatio, CK-MBQuantitative, CKRI, CPK, Creatinine,

DiastolicBP, Eos#, Eosinophilswbc), EstimatedGFR, Ferritin, FiO2, FreeT4, Glu-

cose(Fasting), Glucose(Random), Hb, HCO3Arterial, HCT, HDL, INR, LDL, L-

LacticAcid-Plasma, Lymp#, Lymphocytes(wbc), Magnesium, MCH, MCHC, MCV,

Metamyelocytes, MEWSAVPUScale, Mono#, Monocytes(wbc), MPV, Myelocytes,

Neut#, Neutrophils(wbc), O2SaturationArterial, pCO2Arterial, pHArterial, Phospho-

rus, Platelets, pO2Arterial, Potassium, Promyelocytes, PTT, Pulse, RBC, RDW,

RespiratoryRate, Sodium, SpecificGravity(Urine), SystolicBP, Temperature, Total-

Bilirubin, TotalCO2, TotalProtein, Triglycerides, TroponinT, TroponinT(HighSens.),





ride, Cholesterol, CK-MBQuantitative, CKRI, CPK, Creatinine, DiastolicBP, Eos#,

Eosinophilswbc), EstimatedGFR, FiO2, FreeT4, Glucose(Fasting), Glucose(Random),

Hb, HCO3Arterial, HCT, INR, L-LacticAcid-Plasma, Lymp#, Lymphocytes(wbc),

Magnesium, MCH, MCHC, MCV, Metamyelocytes, MEWSAVPUScale, Mono#,

Monocytes(wbc), MPV, Myelocytes, Neut#, Neutrophils(wbc), O2SaturationArterial,

pCO2Arterial, pHArterial, Phosphorus, Platelets, pO2Arterial, Potassium, Promyelo-

cytes, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine), Sys-


TroponinT(HighSens.), TSH(2ndGeneration), Urea, UrinepH, UrineProtein, Urobilinogen,

Vit.B12, VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF, Age













VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF, Age












VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF, Age








pCO2Arterial, pHArterial, Phosphorus, Platelets, pO2Arterial, Potassium, Promye-

locytes, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine),

SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein, Triglycerides, Tro-

poninT, TSH(2ndGeneration), Urea, UrinepH, UrineProtein, Urobilinogen, VSFiO2,









pCO2Arterial, pHArterial, Phosphorus, Platelets, pO2Arterial, Potassium, Promye-

locytes, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine),

SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein, Triglycerides, Tro-






ride, CK-MBQuantitative, CKRI, CPK, Creatinine, DiastolicBP, Eos#, Eosinophilswbc),

EstimatedGFR, FiO2, FreeT4, Glucose(Fasting), Glucose(Random), Hb, HCO3Arterial,

HCT, INR, L-LacticAcid-Plasma, Lymp#, Lymphocytes(wbc), Magnesium, MCH,

MCHC, MCV, Metamyelocytes, MEWSAVPUScale, Mono#, Monocytes(wbc), MPV,

Myelocytes, Neut#, Neutrophils(wbc), O2SaturationArterial, pCO2Arterial, pHArterial,

Phosphorus, Platelets, pO2Arterial, Potassium, Promyelocytes, PTT, Pulse, RBC, RDW,

RespiratoryRate, Sodium, SpecificGravity(Urine), SystolicBP, Temperature, TotalBiliru-

bin, TotalCO2, TotalProtein, TroponinT, TSH(2ndGeneration), Urea, UrinepH, Urine-

Protein, Urobilinogen, VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF,

Age

541 80.83 79 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, Bands, BaseExcessArte-

rial, Baso#, Basophils(wbc), Blasts(wbc), Calcium, Carboxy-Hb, Chloride, CK-

MBQuantitative, CKRI, CPK, Creatinine, DiastolicBP, Eos#, Eosinophilswbc), Es-

timatedGFR, FiO2, Glucose(Fasting), Glucose(Random), Hb, HCO3Arterial, HCT,

INR, L-LacticAcid-Plasma, Lymp#, Lymphocytes(wbc), Magnesium, MCH, MCHC,

MCV, Metamyelocytes, MEWSAVPUScale, Mono#, Monocytes(wbc), MPV, Myelocytes,

Neut#, Neutrophils(wbc), O2SaturationArterial, pCO2Arterial, pHArterial, Phosphorus,

Platelets, pO2Arterial, Potassium, Promyelocytes, PTT, Pulse, RBC, RDW, Respira-


talCO2, TotalProtein, TroponinT, TSH(2ndGeneration), Urea, UrinepH, UrineProtein,

Urobilinogen, VSFiO2, WBC, Weight, CART, MEWS, NEWS, ViEWS, CDF, Age

561 81.07 78 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, Bands, BaseExcessArte-

rial, Baso#, Basophils(wbc), Blasts(wbc), Calcium, Carboxy-Hb, Chloride, CK-

MBQuantitative, CKRI, CPK, Creatinine, DiastolicBP, Eos#, Eosinophilswbc), Es-

timatedGFR, FiO2, Glucose(Fasting), Glucose(Random), Hb, HCO3Arterial, HCT,


MCV, Metamyelocytes, MEWSAVPUScale, Mono#, Monocytes(wbc), MPV, Myelocytes,


Platelets, pO2Arterial, Potassium, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium,

SpecificGravity(Urine), SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein,

TroponinT, TSH(2ndGeneration), Urea, UrinepH, UrineProtein, Urobilinogen, VSFiO2,


581 81.8 75 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, Bands, BaseExcessArterial,

Baso#, Basophils(wbc), Blasts(wbc), Calcium, Carboxy-Hb, Chloride, CPK, Creati-

nine, DiastolicBP, Eos#, Eosinophilswbc), EstimatedGFR, FiO2, Glucose(Fasting), Glu-

cose(Random), Hb, HCO3Arterial, HCT, INR, L-LacticAcid-Plasma, Lymp#, Lympho-

cytes(wbc), Magnesium, MCH, MCHC, MCV, Metamyelocytes, Mono#, Monocytes(wbc),

MPV, Myelocytes, Neut#, Neutrophils(wbc), O2SaturationArterial, pCO2Arterial, pHAr-

terial, Phosphorus, Platelets, pO2Arterial, Potassium, PTT, Pulse, RBC, RDW, Respira-


talCO2, TotalProtein, TroponinT, TSH(2ndGeneration), Urea, UrinepH, UrineProtein,



601 82.04 74 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, Bands, Baso#, Basophils(wbc),

Blasts(wbc), Calcium, Carboxy-Hb, Chloride, CPK, Creatinine, DiastolicBP, Eos#,

Eosinophilswbc), EstimatedGFR, FiO2, Glucose(Fasting), Glucose(Random), Hb,

HCO3Arterial, HCT, INR, L-LacticAcid-Plasma, Lymp#, Lymphocytes(wbc), Magne-

sium, MCH, MCHC, MCV, Metamyelocytes, Mono#, Monocytes(wbc), MPV, Myelocytes,






621 82.52 72 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, Bands, Baso#, Ba-

sophils(wbc), Blasts(wbc), Calcium, Carboxy-Hb, Chloride, CPK, Creatinine, Di-

astolicBP, Eos#, Eosinophilswbc), EstimatedGFR, FiO2, Glucose(Fasting), Glu-

cose(Random), Hb, HCO3Arterial, HCT, INR, L-LacticAcid-Plasma, Lymp#, Lym-

phocytes(wbc), Magnesium, MCH, MCHC, MCV, Mono#, Monocytes(wbc), MPV,






641 82.52 72 Albumin, ALP, ALT, Amylase(Serum), AnionGap, AST, Bands, Baso#, Ba-

sophils(wbc), Blasts(wbc), Calcium, Carboxy-Hb, Chloride, CPK, Creatinine, Di-

astolicBP, Eos#, Eosinophilswbc), EstimatedGFR, FiO2, Glucose(Fasting), Glu-

cose(Random), Hb, HCO3Arterial, HCT, INR, L-LacticAcid-Plasma, Lymp#, Lym-

phocytes(wbc), Magnesium, MCH, MCHC, MCV, Mono#, Monocytes(wbc), MPV,






661 83.01 70 Albumin, ALP, ALT, AnionGap, AST, Baso#, Basophils(wbc), Blasts(wbc), Cal-

cium, Carboxy-Hb, Chloride, CPK, Creatinine, DiastolicBP, Eos#, Eosinophilswbc),

EstimatedGFR, FiO2, Glucose(Fasting), Glucose(Random), Hb, HCO3Arterial, HCT,


MCV, Mono#, Monocytes(wbc), MPV, Neut#, Neutrophils(wbc), O2SaturationArterial,

pCO2Arterial, pHArterial, Phosphorus, Platelets, pO2Arterial, Potassium, PTT, Pulse,

RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine), SystolicBP, Tempera-

ture, TotalBilirubin, TotalCO2, TotalProtein, TroponinT, TSH(2ndGeneration), Urea,

UrinepH, UrineProtein, Urobilinogen, VSFiO2, WBC, Weight, CART, MEWS, NEWS,

ViEWS, CDF, Age

681 84.47 64 Albumin, ALP, ALT, AnionGap, AST, Baso#, Calcium, Carboxy-Hb, Chloride,

CPK, Creatinine, DiastolicBP, Eos#, EstimatedGFR, FiO2, Glucose(Fasting), Glu-

cose(Random), Hb, HCO3Arterial, HCT, INR, L-LacticAcid-Plasma, Lymp#, Magne-

sium, MCH, MCHC, MCV, Mono#, MPV, Neut#, O2SaturationArterial, pCO2Arterial,

pHArterial, Phosphorus, Platelets, pO2Arterial, Potassium, PTT, Pulse, RBC, RDW, Res-

piratoryRate, Sodium, SpecificGravity(Urine), SystolicBP, Temperature, TotalBilirubin,

TotalCO2, TotalProtein, TroponinT, TSH(2ndGeneration), Urea, UrinepH, UrineProtein,



821 85.19 61 Albumin, ALP, ALT, AnionGap, AST, Baso#, Calcium, Chloride, CPK, Creati-

nine, DiastolicBP, Eos#, EstimatedGFR, FiO2, Glucose(Random), Hb, HCO3Arterial,

HCT, INR, L-LacticAcid-Plasma, Lymp#, Magnesium, MCH, MCHC, MCV, Mono#,

MPV, Neut#, O2SaturationArterial, pCO2Arterial, pHArterial, Phosphorus, Platelets,

pO2Arterial, Potassium, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium, Specific-

Gravity(Urine), SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein, Tro-


WBC, CART, MEWS, NEWS, ViEWS, CDF, Age

























901 85.44 60 Albumin, ALP, ALT, AnionGap, AST, Baso#, Calcium, Chloride, CPK, Creatinine, Dias-

tolicBP, Eos#, EstimatedGFR, FiO2, Glucose(Random), Hb, HCO3Arterial, HCT, INR,

L-LacticAcid-Plasma, Lymp#, Magnesium, MCH, MCHC, MCV, Mono#, MPV, Neut#,

O2SaturationArterial, pCO2Arterial, pHArterial, Phosphorus, Platelets, pO2Arterial,

Potassium, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium, SpecificGravity(Urine),

SystolicBP, Temperature, TotalBilirubin, TotalCO2, TotalProtein, TroponinT, Urea,

UrinepH, UrineProtein, Urobilinogen, VSFiO2, WBC, CART, MEWS, NEWS, ViEWS,

CDF, Age








CDF, Age









CDF, Age








CDF, Age








CDF, Age





Potassium, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium, SystolicBP, Tempera-

ture, TotalBilirubin, TotalCO2, TotalProtein, TroponinT, Urea, UrineProtein, VSFiO2,







ture, TotalBilirubin, TotalCO2, TotalProtein, TroponinT, Urea, UrineProtein, VSFiO2,


1041 87.62 51 Albumin, ALP, ALT, AnionGap, AST, Baso#, Calcium, Chloride, CPK, Creatinine, Di-

astolicBP, Eos#, EstimatedGFR, FiO2, Glucose(Random), Hb, HCT, INR, L-LacticAcid-

Plasma, Lymp#, Magnesium, MCH, MCHC, MCV, Mono#, MPV, Neut#, Phosphorus,

Platelets, Potassium, PTT, Pulse, RBC, RDW, RespiratoryRate, Sodium, SystolicBP,

Temperature, TotalBilirubin, TotalCO2, TotalProtein, TroponinT, Urea, VSFiO2, WBC,


1061 87.62 51 Albumin, ALP, ALT, AnionGap, AST, Baso#, Calcium, Chloride, CPK, Creatinine, Di-

astolicBP, Eos#, EstimatedGFR, FiO2, Glucose(Random), Hb, HCT, INR, L-LacticAcid-






1081 87.86 50 Albumin, ALP, ALT, AnionGap, AST, Baso#, Calcium, Chloride, Creatinine, Dias-

tolicBP, Eos#, EstimatedGFR, FiO2, Glucose(Random), Hb, HCT, INR, L-LacticAcid-






tolicBP, Eos#, EstimatedGFR, Glucose(Random), Hb, HCT, INR, L-LacticAcid-Plasma,

Lymp#, Magnesium, MCH, MCHC, MCV, Mono#, MPV, Neut#, Phosphorus, Platelets,


ture, TotalBilirubin, TotalCO2, TotalProtein, TroponinT, Urea, VSFiO2, WBC, CART,

MEWS, NEWS, ViEWS, CDF, Age


tolicBP, Eos#, EstimatedGFR, Glucose(Random), Hb, HCT, INR, Lymp#, Magnesium,

MCH, MCHC, MCV, Mono#, MPV, Neut#, Phosphorus, Platelets, Potassium, PTT,

Pulse, RBC, RDW, RespiratoryRate, Sodium, SystolicBP, Temperature, TotalBiliru-

bin, TotalCO2, TotalProtein, TroponinT, Urea, VSFiO2, WBC, CART, MEWS, NEWS,

ViEWS, CDF, Age






ViEWS, CDF, Age






ViEWS, CDF, Age




Pulse, RBC, RDW, RespiratoryRate, Sodium, SystolicBP, Temperature, TotalBilirubin,

TotalCO2, TotalProtein, Urea, VSFiO2, WBC, CART, MEWS, NEWS, ViEWS, CDF,

Age






Age






Age







Age






Age






Age






Age






Age






Age





TotalCO2, Urea, VSFiO2, WBC, CART, MEWS, NEWS, ViEWS, CDF, Age


















tolicBP, Eos#, EstimatedGFR, Hb, HCT, INR, Lymp#, Magnesium, MCH, MCHC,

MCV, Mono#, MPV, Neut#, Phosphorus, Platelets, Potassium, PTT, Pulse, RBC, RDW,

RespiratoryRate, Sodium, SystolicBP, Temperature, TotalBilirubin, TotalCO2, Urea, VS-

FiO2, WBC, CART, MEWS, NEWS, ViEWS, CDF, Age


























1561 90.05 41 Albumin, AnionGap, Baso#, Calcium, Chloride, Creatinine, DiastolicBP, Eos#, Esti-

matedGFR, Hb, HCT, INR, Lymp#, Magnesium, MCH, MCHC, MCV, Mono#, MPV,

Neut#, Phosphorus, Platelets, Potassium, PTT, Pulse, RBC, RDW, RespiratoryRate,

Sodium, SystolicBP, Temperature, TotalCO2, Urea, VSFiO2, WBC, CART, MEWS,













1621 90.53 39 AnionGap, Baso#, Calcium, Chloride, Creatinine, DiastolicBP, Eos#, EstimatedGFR,

Hb, HCT, INR, Lymp#, Magnesium, MCH, MCHC, MCV, Mono#, MPV, Neut#, Phos-

phorus, Platelets, Potassium, Pulse, RBC, RDW, RespiratoryRate, Sodium, SystolicBP,

Temperature, TotalCO2, Urea, VSFiO2, WBC, CART, MEWS, NEWS, ViEWS, CDF,

Age

1641 90.53 39 AnionGap, Baso#, Calcium, Chloride, Creatinine, DiastolicBP, Eos#, EstimatedGFR,

Hb, HCT, INR, Lymp#, Magnesium, MCH, MCHC, MCV, Mono#, MPV, Neut#, Phos-

phorus, Platelets, Potassium, Pulse, RBC, RDW, RespiratoryRate, Sodium, SystolicBP,

Temperature, TotalCO2, Urea, VSFiO2, WBC, CART, MEWS, NEWS, ViEWS, CDF,

Age

1661 91.26 36 AnionGap, Baso#, Chloride, Creatinine, DiastolicBP, Eos#, EstimatedGFR, Hb, HCT,

INR, Lymp#, MCH, MCHC, MCV, Mono#, MPV, Neut#, Platelets, Potassium, Pulse,

RBC, RDW, RespiratoryRate, Sodium, SystolicBP, Temperature, TotalCO2, Urea, VS-



Lymp#, MCH, MCHC, MCV, Mono#, MPV, Neut#, Platelets, Potassium, Pulse, RBC,

RDW, RespiratoryRate, Sodium, SystolicBP, Temperature, TotalCO2, Urea, VSFiO2,

































































RDW, RespiratoryRate, Sodium, SystolicBP, Temperature, TotalCO2, VSFiO2, WBC,


2021 92.96 29 AnionGap, Chloride, Creatinine, DiastolicBP, EstimatedGFR, Hb, HCT, MCH, MCHC,

MCV, MPV, Platelets, Potassium, Pulse, RBC, RDW, RespiratoryRate, Sodium, Sys-

tolicBP, Temperature, TotalCO2, VSFiO2, WBC, CART, MEWS, NEWS, ViEWS, CDF,

Age

2041 96.12 16 AnionGap, Chloride, DiastolicBP, Pulse, RespiratoryRate, Sodium, SystolicBP, Temper-

ature, TotalCO2, VSFiO2, CART, MEWS, NEWS, ViEWS, CDF, Age

2061 97.09 12 DiastolicBP, Pulse, RespiratoryRate, SystolicBP, Temperature, VSFiO2, CART, MEWS,



Table B.6: Encounter Data Characteristics: Variables Selected by Num-

ber of Encounters Threshold

Approx. Sample Fuzzy Shannon ChaoShen

MEWS Min 0.04 0.05 0.10 3.66 4.41

ViEWS Min 0.06 0.06 0.11 3.96 4.72

NEWS Min 0.09 0.09 0.14 4.36 5.01

CDF Max 1.19 0.24 0.48 7.61 7.61

CDF Mean 0.79 0.63 0.62 7.63 7.63

MEWS Mean 0.37 0.68 0.61 7.54 10.19

CDF Min 1.01 0.70 0.35 10.58 6.97

ViEWS Mean 0.50 0.70 0.60 7.52 9.90

NEWS Mean 0.63 0.71 0.61 7.52 9.22

MEWS Max 1.68 0.74 0.60 7.58 7.68

CART Min 0.76 0.75 0.37 7.42 7.44

CDF Median 0.81 0.76 0.59 7.64 7.64

CART Mean 1.87 0.80 0.61 7.49 7.55

CART Max 1.88 0.81 0.58 7.54 7.54

MEWS Median 0.83 0.83 0.42 7.41 8.32

ViEWS Max 1.76 0.86 0.55 7.57 7.64

CART Median 1.01 0.90 0.49 7.40 7.42

NEWS Max 1.81 0.92 0.56 7.58 7.62

ViEWS Median 1.03 1.03 0.55 7.31 8.07

NEWS Median 1.17 1.20 0.51 7.37 8.01

Table B.7: Estimators of Entropy for Early Warning Scores Values,

Sorted by Sample Entropy

Outcome Harrel’s C Somer’s D

Composite 0.50809 0.01619

Death 0.50852 0.01704

CodeBlue 0.50859 0.01717

ICUTransfer 0.50915 0.0183

CardiacArrest 0.51056 0.02112

Table B.8: Summary of Discrimination Indices Per Outcome


Variable Description and UnitsMBP Mean Blood Pressure (mmHg)

PaO2/FiO2 Partial Pressure of Oxygen to Fraction of Inspired Oxygen Ratio (calculated)P(A-a)O2 Alveolar-Arterial Gradient (calculated)

Hct Hematocrit (percentage)WBC White Blood Cell Count (number of cells)

Cr [Serum] Creatinine (mg/dL)BUN [Serum] Blood Urea Nitrogen (mg/dL)Na [Serum] Sodium (mEq/L)Alb [Serum] Albumin (g/dL)Bili Bilirubin (mg/dL)SGc [Serum] Glucose (mg/dL)

HbA1c Glycated Hemoglobin (Hemoglobin A1c) (percent)pH Blood pH (pH scale)

HRVR Heart Rate Ventricular Response (binary atrial fibrillation)Lac [Serum] Lactate (mEq/L)

PaCO2 Partial Pressure of Carbon Dioxide (mmHg)AMY [Serum] Amylase (units/L)ALP [Serum] Alkaline Phosphatase (units/L)BSS Braden Scale/Score (units)Plat Platelet Count (number of platelets)PT Prothrombin Time No-Anticoagulants (seconds > control)CSF CSF-Positive Culture (binary)BPC Blood Positive Culture (binary)FPC Fungal Positive Culture (binary)Ca [Serum] Calcium (mg/dL)K [Serum] Potassium (mEq/L)

HCO3 [Serum] Bicarbonate (mEq/L)ALT [Serum] Alanine Aminotransferase (units/L)GGT Gamma-glutamyl Transferase (units/L)LDH [Serum] Lactate Dehydrogenase (units/L)PO4 [Serum] Phosphate (mmol/L)NH3 [Serum] Ammonia (umol/L)Mg [Serum] Magnesium (mmol/L)A Age (years)W Weight (lbs)PR Pulse Rate (beats per minute)

Temp Temperature (degrees Celsius)RR Respiratory Rate (beats per minute)SBP Systolic Blood Pressure (mmHg)DBP Diastolic Blood Pressure (mmHg)SaO2 Oxygen Saturation (percent)GCS Glasgow Coma Score (units)

AVPU Alertness (units)LOC Level of Consciousness Score (MSH) (units)

Figure B.2: Parameters Selected from Predictive Health Scores and Other Literature


SummativeScore

3 2 1 0 1 2 3

Systolic Bloodpressure(mmHg)

<70 71-80 81-100 101-199 ≥200

Heart rate(bpm)

<40 41-50 51-100 101-110 111-129 ≥130

Respiratoryrate (bpm)

<9 9-14 15-20 21-29 ≥30

Temperature(degrees C)

<35 35-38.4 ≥38.5

AVPU Score Alert Reacting:Voice

Reacting: Pain Unresponsive

Figure B.3: Modified Early Warning Scores (MEWS)

Summative Score 3 2 1 0 1 2 3

Pulse (bpm) ≤ 40 41-50 51-90 91-110 111-130 ≥131Breathing Rate(bpm)

≤8 9-11 12-20 21-24 ≥25

Temperature (de-grees C)

≤35.0 35.1-36.0 36.1-38.0 38.1-39.0 ≥39.1

Systolic BP(mmHg)

≤90 91-100 101-110 111-249 ≥250

SaO2 (%) ≤91 92-93 94-95 ≥96Inspired O2 Air Any O2

CNS (use AVPUScale)

Alert (A) Voice (V), Pain(P), Unresponsive(U)

Figure B.4: VitalPAC™Early Warning Score (ViEWS)

Summative Score 3 2 1 0 1 2 3

Respiration Rate(bpm)

≤8 9-11 12-20 21-24 ≥25

Oxygen Satura-tions (%)

≤91 92-93 94-95 ≥96

Supplemental Oxy-gen

Yes No

Temperature (de-grees C)

≤35.0 35.1-36.0 36.1-38.0 38.1-39.0 ≥39.1

Systolic BP(mmHg)

≤90 91-100 101-110 111-219 ≥220

Heart Rate ≤40 41-50 51-90 91-110 111-130 ≥131AVPU Score Alert (A) Voice (V), Pain

(P), Unresponsive(U)

Figure B.5: National Early Warning Score (NEWS)


f1 = 0.030 ∗ Pulse+ 0.085 ∗RespiratoryRate− 0.003 ∗ SystolicBP−0.430 ∗ Temperature− 0.163 ∗O2SaturationArterial + 27.583 (B.1)

f2 = 0.027 ∗ Pulse+ 0.091 ∗RespiratoryRate− 0.155 ∗O2SaturationArterial + 10.613 (B.2)

f3 = 0.030 ∗ Pulse+ 0.121 ∗RespiratoryRate− 5.240 (B.3)

Figure B.6: Cuthbertson Discriminant Functions (CDF)

Vital Sign Summative Score

Respiratory rate<21 021-23 824-25 1226-29 15>29 22

Heart rate<110 0

110-139 4>139 13

Diastolic blood presure>49 040-49 435-39 6<35 13Age<55 055-69 4>69 9

Figure B.7: Cardiac Arrest Risk Triage Scoreing Criteria (CART)


Call CCRT staff when patient exhibits signs of the following:

Airway

• Threatened

• Stridor

• Excessive secretions

Breathing

• Respiratory rate ≤ 8 or ≥ 30

• Distressed breathing

• O2 Saturations < 90 on = 50% O2 or 6 litres/min

Circulation

• Systolic blood pressure ≤ 90 mmHg or ≥ 200 mmHg or decrease 40 mmHg

• Heart rate ≤ 40 or ≥ 130

Disability

• Decreased level of consciousness

• Decrease in Glasgow Coma Scale > 2 points

• Signs or symptoms of a stroke

• Prolonged seizures

Other

• Urine output ≤ 100 ml over 4 hours (except when on dialysis)

• Serious concerns about your patient

Figure B.8: Ministry of Health and Long Term Care Calling Criteria (MOH)


Figure B.9: Parameters with Amount of Data within First Quantile


Figure B.10: Parameters with Amount of Data within Second Quantile


Figure B.11: Parameters with Amount of Data within Third Quantile


Figure B.12: Parameters with Amount of Data within Fourth Quantile


Figure B.13: Cleveland Dot Plots of Entropy Estimators Grouped by Early Warning Score


Figure B.14: Cleveland Dot Plots of Entropy Estimators Grouped by Early Warning Score Value Type


Figure B.15: Forest Plots of Cox Proportional Hazard Results: Hazard Ratios


Figure B.16: Cleveland Dot Plot of Cox Proportional Hazard Results: p-Values


Figure B.17: Cleveland Dot Plot of Cumulative Incidence from Comparative Risk: p-Values


Figure B.18: Receiver Operating Curves of CardiacArrest Prediction by Time to Event Thresholds for EarlyWarning Scores


Figure B.19: Receiver Operating Curves of CardiacArrest Prediction by Early Warning Score for Time toEvent Thresholds


Figure B.20: Receiver Operating Curves of CodeBlue Prediction by Time to Event Thresholds for EarlyWarning Scores


Figure B.21: Receiver Operating Curves of CodeBlue Prediction by Early Warning Score for Time to EventThresholds


Figure B.22: Receiver Operating Curves of Composite Prediction by Time to Event Thresholds for EarlyWarning Scores


Figure B.23: Receiver Operating Curves of Composite Prediction by Early Warning Score for Time to EventThresholds


Figure B.24: Receiver Operating Curves of Death Prediction by Time to Event Thresholds for Early WarningScores


Figure B.25: Receiver Operating Curves of Death Prediction by Early Warning Score for Time to EventThresholds


Figure B.26: Receiver Operating Curves of ICUTransfer Prediction by Time to Event Thresholds for EarlyWarning Scores


Figure B.27: Receiver Operating Curves of ICUTransfer Prediction by Early Warning Score for Time toEvent Thresholds


Outcome dT Threshold EWS AUC 95% CI

CodeBlue (0,16] CART 0.618 0.586-0.651

CodeBlue (0,24] CART 0.621 0.593-0.649

CodeBlue (0,12] CART 0.624 0.587-0.66

ICUTransfer (0,6] CART 0.628 0.609-0.647


CardiacArrest (0,16] CART 0.63 0.593-0.667


CodeBlue (0,8] CART 0.63 0.589-0.671





CodeBlue (0,4] CART 0.636 0.584-0.688

CodeBlue (0,6] CART 0.638 0.593-0.683





Composite (0,4] CART 0.657 0.641-0.673

Composite (0,6] CART 0.657 0.642-0.672

Composite (0,8] CART 0.663 0.649-0.677

Composite (0,12] CART 0.671 0.658-0.684

Death (0,4] CART 0.672 0.653-0.691

Death (0,6] CART 0.675 0.658-0.692

Composite (0,16] CART 0.679 0.667-0.691

Death (0,8] CART 0.684 0.668-0.7

Composite (0,24] CART 0.685 0.673-0.697

Death (0,12] CART 0.693 0.679-0.708

Death (0,16] CART 0.699 0.685-0.712

Death (0,24] CART 0.703 0.691-0.716

Death (0,4] ViEWS 0.734 0.717-0.752

Death (0,4] NEWS 0.743 0.726-0.76

Death (0,4] MEWS 0.744 0.727-0.761

Death (0,4] CDF 0.759 0.743-0.776

CodeBlue (0,6] CDF 0.773 0.731-0.815

Composite (0,4] CDF 0.773 0.761-0.786

Death (0,6] ViEWS 0.773 0.758-0.787

Death (0,6] NEWS 0.779 0.764-0.793

Death (0,6] MEWS 0.784 0.77-0.798

CardiacArrest (0,6] NEWS 0.786 0.743-0.829

CodeBlue (0,6] NEWS 0.786 0.751-0.822

CodeBlue (0,6] ViEWS 0.789 0.753-0.825

CardiacArrest (0,6] ViEWS 0.79 0.748-0.833

CodeBlue (0,4] NEWS 0.79 0.748-0.832

CodeBlue (0,4] ViEWS 0.792 0.75-0.835

CodeBlue (0,4] CDF 0.792 0.745-0.839

Death (0,6] CDF 0.792 0.778-0.806




CardiacArrest (0,6] CDF 0.794 0.748-0.84

Composite (0,6] CDF 0.794 0.783-0.805

Death (0,8] ViEWS 0.794 0.781-0.807

ICUTransfer (0,4] CDF 0.794 0.777-0.811


Composite (0,4] ViEWS 0.796 0.783-0.808


Death (0,8] NEWS 0.798 0.785-0.811

Composite (0,4] NEWS 0.799 0.786-0.811

CodeBlue (0,8] CDF 0.8 0.765-0.836

CodeBlue (0,8] NEWS 0.802 0.771-0.832

CodeBlue (0,8] ViEWS 0.805 0.775-0.836

CodeBlue (0,4] MEWS 0.806 0.763-0.848

Composite (0,4] MEWS 0.806 0.794-0.818

Death (0,8] MEWS 0.806 0.794-0.818

ICUTransfer (0,6] CDF 0.806 0.791-0.821

CardiacArrest (0,6] MEWS 0.809 0.766-0.851

CodeBlue (0,6] MEWS 0.81 0.774-0.846



Composite (0,8] CDF 0.811 0.801-0.821

Death (0,8] CDF 0.811 0.799-0.823

Composite (0,6] ViEWS 0.813 0.802-0.824


Composite (0,6] NEWS 0.815 0.803-0.826

CodeBlue (0,12] CDF 0.817 0.788-0.846


ICUTransfer (0,8] CDF 0.821 0.807-0.834

Death (0,12] ViEWS 0.824 0.813-0.835

CodeBlue (0,8] MEWS 0.825 0.794-0.855

Composite (0,6] MEWS 0.825 0.815-0.836

Composite (0,8] NEWS 0.825 0.815-0.836

Composite (0,8] ViEWS 0.825 0.815-0.836


Composite (0,12] CDF 0.826 0.817-0.834

Death (0,12] NEWS 0.827 0.816-0.838


Death (0,12] CDF 0.829 0.819-0.839

CodeBlue (0,12] NEWS 0.832 0.808-0.856

ICUTransfer (0,12] CDF 0.832 0.82-0.844


Composite (0,8] MEWS 0.837 0.827-0.847

Death (0,12] MEWS 0.837 0.826-0.847

CodeBlue (0,12] ViEWS 0.838 0.814-0.861

CodeBlue (0,16] CDF 0.838 0.814-0.862

Composite (0,16] CDF 0.839 0.831-0.847

Death (0,16] ViEWS 0.842 0.833-0.852

Composite (0,12] NEWS 0.843 0.833-0.853

Composite (0,12] ViEWS 0.843 0.834-0.853


Death (0,16] NEWS 0.843 0.833-0.853

Death (0,16] CDF 0.843 0.834-0.851


ICUTransfer (0,16] CDF 0.844 0.833-0.854

CodeBlue (0,16] NEWS 0.845 0.826-0.865



CodeBlue (0,16] ViEWS 0.854 0.834-0.874

Composite (0,16] NEWS 0.854 0.845-0.863


Composite (0,12] MEWS 0.855 0.847-0.864

Composite (0,16] ViEWS 0.855 0.846-0.864

Composite (0,24] CDF 0.855 0.848-0.862

Death (0,16] MEWS 0.855 0.846-0.864

CodeBlue (0,12] MEWS 0.856 0.833-0.88

CodeBlue (0,24] CDF 0.856 0.837-0.875

ICUTransfer (0,24] CDF 0.858 0.849-0.867

Death (0,24] CDF 0.859 0.852-0.867

Death (0,24] NEWS 0.861 0.852-0.87

Death (0,24] ViEWS 0.862 0.853-0.871

CodeBlue (0,24] NEWS 0.866 0.85-0.882


Composite (0,16] MEWS 0.868 0.86-0.876

Composite (0,24] NEWS 0.868 0.859-0.876

ICUTransfer (0,4] NEWS 0.869 0.857-0.881


Composite (0,24] ViEWS 0.87 0.862-0.878


CodeBlue (0,16] MEWS 0.871 0.852-0.891



Death (0,24] MEWS 0.875 0.867-0.883

CodeBlue (0,24] ViEWS 0.877 0.862-0.892


ICUTransfer (0,4] ViEWS 0.878 0.866-0.89



Composite (0,24] MEWS 0.882 0.875-0.89





ICUTransfer (0,4] MEWS 0.889 0.878-0.901



CodeBlue (0,24] MEWS 0.893 0.878-0.907








Table B.9: Table of Area Under the Curve Summary Across Receiver

Operating Curves Sorted by Area Under the Curve

Outcome Lambda Mean

Cross-

Validated

Error

Non-zero Coefficient Variables

CardiacArrest.Max,Min 0.01 0.27

CodeBlue.Max,Min 0.01 0.34 AnionGap.Max, Creatinine.Min, Esti-

matedGFR.Max, Hb.Min, INR.Min,

MCHC.Min, MCV.Max, Mono#.Min,

Neut#.Max, RespiratoryRate.Max, Respira-

toryRate.Min, TotalCO2.Min, Urea.Max

Composite.Max,Min 0.00 1.17 AnionGap.Max, AnionGap.Min,

Baso#.Max, Baso#.Min, Creatinine.Max,

Creatinine.Min, DiastolicBP.Max, Di-

astolicBP.Min, Eos#.Max, Eos#.Min,

EstimatedGFR.Max, EstimatedGFR.Min,

Hb.Max, Hb.Min, HCT.Max, INR.Max,

INR.Min, Lymp#.Max, Lymp#.Min,

MCH.Max, MCH.Min, MCHC.Max,

MCHC.Min, MCV.Max, MCV.Min,

Mono#.Max, Mono#.Min, MPV.Min,

Neut#.Max, Neut#.Min, Platelets.Max,

Platelets.Min, Potassium.Max, Potas-

sium.Min, Pulse.Max, Pulse.Min,

RBC.Min, RDW.Max, RDW.Min, Res-

piratoryRate.Max, RespiratoryRate.Min,

Sodium.Max, Sodium.Min, SystolicBP.Max,

Temperature.Max, Temperature.Min, To-

talCO2.Max, TotalCO2.Min, Urea.Min,

VSFiO2.Max, VSFiO2.Min, WBC.Max,

WBC.Min, Age.Min

Death.Max,Min 0.01 0.98 AnionGap.Max, Baso#.Max, Chloride.Max,

Chloride.Min, Creatinine.Min, Dias-

tolicBP.Max, DiastolicBP.Min, Eos#.Max,

Eos#.Min, EstimatedGFR.Max, Estimat-

edGFR.Min, Hb.Max, HCT.Max, INR.Max,

INR.Min, Lymp#.Max, MCHC.Min,

MCV.Max, Mono#.Min, MPV.Max,

MPV.Min, Neut#.Max, Neut#.Min,

Platelets.Max, Potassium.Max, Potas-

sium.Min, Pulse.Min, RDW.Min, Respira-

toryRate.Max, Sodium.Max, Sodium.Min,

SystolicBP.Max, SystolicBP.Min, Tempera-

ture.Max, Temperature.Min, TotalCO2.Max,

TotalCO2.Min, Urea.Min, Age.Min


ICUTransfer.Max,Min 0.00 0.92 AnionGap.Max, AnionGap.Min,

Baso#.Max, DiastolicBP.Max, Dias-

tolicBP.Min, Eos#.Max, Eos#.Min,

EstimatedGFR.Min, Hb.Max, Hb.Min,

INR.Min, Lymp#.Min, MCH.Max,

MCH.Min, MCHC.Max, Mono#.Max,

Mono#.Min, Neut#.Min, Potassium.Max,

Potassium.Min, Pulse.Max, Pulse.Min,

RDW.Max, RespiratoryRate.Max, Respira-

toryRate.Min, Sodium.Max, Sodium.Min,

SystolicBP.Min, Temperature.Max, Temper-

ature.Min, TotalCO2.Max, TotalCO2.Min,

Urea.Max, Urea.Min, VSFiO2.Max, VS-

FiO2.Min, WBC.Max, Age.Min

Table B.10: Table of Logistic Regression LASSO Optimization for Min-

imum Cross-Validation Error per Encounter

Outcome Lambda Mean

Cross-

Validated

Error

Non-zero Coefficient Variables

CardiacArrest 0.02 0.34

CodeBlue 0.01 0.43 Potassium.Max, Urea.Max, SystolicBP.Min, Respi-

ratoryRate.Max, Hb.Max, Hb.Min, AnionGap.Max,

MCV.Max, Neut#.Max, Platelets.Min

Composite 0.01 1.24 Potassium.Max, DiastolicBP.Min, MCHC.Max,

MCHC.Min, Lymp#.Min, RDW.Max, Chlo-

ride.Max, Age.Max, Urea.Max, WBC.Max,

INR.Max, SystolicBP.Max, RespiratoryRate.Max,

RespiratoryRate.Min, MCH.Max, Eos#.Max,

HCT.Max, TotalCO2.Max, TotalCO2.Min,

Pulse.Min, AnionGap.Max, Temperature.Max,

MCV.Max, Neut#.Max, Neut#.Min, Platelets.Max,

O2SaturationArterial.Min

Death 0.01 1.07 Potassium.Max, Potassium.Min, MPV.Min,

MCHC.Max, MCHC.Min, Baso#.Max, Lymp#.Max,

RDW.Min, Chloride.Max, Chloride.Min, Sodium.Min,

Age.Max, Urea.Max, INR.Max, SystolicBP.Max, Sys-

tolicBP.Min, RespiratoryRate.Max, Eos#.Max,

RBC.Max, TotalCO2.Min, Pulse.Min, Anion-

Gap.Max, Temperature.Min, MCV.Max, Neut#.Max,

Neut#.Min, Platelets.Max, Mono#.Max,

O2SaturationArterial.Min


ICUTransfer 0.01 1.20 Potassium.Max, Potassium.Min, DiastolicBP.Max,

DiastolicBP.Min, RDW.Min, Sodium.Max, Age.Max,

WBC.Max, SystolicBP.Min, RespiratoryRate.Min,

Hb.Max, Hb.Min, Eos#.Max, HCT.Min, To-

talCO2.Max, TotalCO2.Min, Pulse.Max, Pulse.Min,

Neut#.Max, Platelets.Min, Platelets.Max,

Mono#.Max, O2SaturationArterial.Min

Table B.11: Table of Logistic Regression LASSO Optimization for Min-

imum Cross-Validation Error per Encounter, Accounting for Direction-

ality

Estimate Std. Error z value Pr(>|z|)MCV.Max 0.15 0.04 3.86 0.00

MCV.Min -0.18 0.06 -2.90 0.00

Hb.Max -0.03 0.01 -2.58 0.01

SystolicBP.Min -0.01 0.01 -2.31 0.02

RBC.Min 2.83 1.24 2.28 0.02

AnionGap.Max 0.06 0.03 2.18 0.03

Hb.Min -0.09 0.04 -2.14 0.03

Neut#.Max 0.03 0.02 2.03 0.04

MCH.Min 0.36 0.19 1.91 0.06

RespiratoryRate.Max 0.00 0.00 1.73 0.08

RDW.Max -0.11 0.07 -1.66 0.10

Platelets.Max -0.00 0.00 -1.62 0.11

(Intercept) -6.44 4.85 -1.33 0.18

Table B.12: Table of Stepwise Optimized Logistic Regression Model Pa-

rameters Accounting for Directionality for Outcome: CodeBlue


MCV.Min -0.27 0.08 -3.22 0.00

MCH.Min 0.82 0.27 3.06 0.00

RBC.Min 3.99 1.46 2.74 0.01

Hb.Min -0.13 0.05 -2.60 0.01

(Intercept) -12.46 5.80 -2.15 0.03

Hb.Max -0.10 0.05 -2.05 0.04

SystolicBP.Min -0.01 0.01 -2.03 0.04

HCT.Max 28.91 15.86 1.82 0.07

RDW.Min -0.20 0.12 -1.73 0.08

AnionGap.Max 0.05 0.03 1.60 0.11

Potassium.Min 0.57 0.39 1.47 0.14

Platelets.Max -0.00 0.00 -1.39 0.17


rameters Accounting for Directionality for Outcome: CardiacArrest



RBC.Min 5.54 2.16 2.56 0.01

Potassium.Min 1.15 0.53 2.19 0.03

MCH.Min 0.85 0.42 2.02 0.04

AnionGap.Max 0.12 0.06 1.90 0.06

Hb.Max -0.12 0.07 -1.72 0.09

SystolicBP.Min -0.01 0.01 -1.70 0.09

(Intercept) -39.32 23.13 -1.70 0.09

Platelets.Max -0.00 0.00 -1.63 0.10

MCV.Min -0.22 0.14 -1.58 0.11

Hb.Min -0.16 0.10 -1.56 0.12

Platelets.Min 0.01 0.00 1.52 0.13

TotalCO2.Max 0.08 0.06 1.50 0.13

HCT.Max 28.66 19.89 1.44 0.15

RDW.Min -0.26 0.18 -1.39 0.16

Chloride.Max 0.08 0.06 1.34 0.18


Potassium.Max -0.38 0.31 -1.25 0.21

Pulse.Max 0.00 0.00 1.18 0.24

MCHC.Min 0.04 0.04 1.08 0.28

Sodium.Max -0.06 0.06 -1.03 0.30

Temperature.Max 0.02 0.02 1.03 0.30

INR.Max -0.14 0.14 -0.98 0.33

O2SaturationArterial.Min 0.02 0.02 0.98 0.33

Temperature.Min 0.03 0.03 0.97 0.33

MCH.Max -0.31 0.34 -0.90 0.37

Sodium.Min -0.05 0.07 -0.82 0.41

MPV.Min 0.23 0.30 0.75 0.45

DiastolicBP.Min -0.01 0.02 -0.74 0.46

VSFiO2.Max -0.01 0.01 -0.74 0.46

Eos#.Max 0.39 0.56 0.71 0.48

Mono#.Max -0.31 0.45 -0.69 0.49

Neut#.Min -0.18 0.27 -0.67 0.50

EstimatedGFR.Min -0.01 0.01 -0.65 0.51

SystolicBP.Max -0.00 0.01 -0.63 0.53

Baso#.Max 0.42 0.74 0.57 0.57

RBC.Max 0.85 1.49 0.57 0.57

DiastolicBP.Max 0.00 0.00 0.55 0.58

Chloride.Min 0.04 0.07 0.54 0.59

Lymp#.Max -0.07 0.16 -0.47 0.64

Urea.Max 0.01 0.03 0.36 0.72

Neut#.Max 0.02 0.06 0.35 0.73

HCT.Min -9.65 33.76 -0.29 0.77

WBC.Min 0.06 0.25 0.25 0.80

TotalCO2.Min 0.02 0.07 0.23 0.82

RDW.Max 0.03 0.13 0.20 0.84

WBC.Max 0.01 0.05 0.13 0.89

MCHC.Max 0.00 0.04 0.10 0.92


MPV.Max 0.02 0.19 0.09 0.93

RespiratoryRate.Min 0.00 0.04 0.09 0.93

Pulse.Min 0.00 0.01 0.05 0.96

Age.Max 0.00 0.02 0.04 0.97

Lymp#.Min -0.01 0.35 -0.02 0.98

Creatinine.Max 0.00 0.00 0.01 0.99

Table B.14: Table of Base Logistic Regression Model Parameters Ac-

counting for Directionality for Outcome: CardiacArrest

Estimate Std. Error z value Pr(>|z|)Age.Max 0.04 0.01 5.81 0.00

(Intercept) -19.44 3.65 -5.32 0.00

Neut#.Min 0.14 0.03 4.97 0.00

MCV.Max 0.12 0.03 4.08 0.00

RDW.Min 0.21 0.05 4.00 0.00

TotalCO2.Min -0.07 0.02 -3.84 0.00

Pulse.Min 0.02 0.00 3.02 0.00

Platelets.Max -0.00 0.00 -2.91 0.00

Chloride.Min -0.05 0.02 -2.84 0.00

Potassium.Min 0.45 0.20 2.31 0.02

INR.Max 0.11 0.05 2.24 0.02

Chloride.Max 0.03 0.02 2.15 0.03

MPV.Min 0.22 0.10 2.07 0.04

Urea.Max 0.02 0.01 2.06 0.04

Potassium.Max 0.23 0.12 1.96 0.05

RBC.Max 1.09 0.56 1.95 0.05

Eos#.Max -0.50 0.26 -1.89 0.06

Creatinine.Max -0.00 0.00 -1.66 0.10

HCT.Max -9.44 6.14 -1.54 0.12

Baso#.Max 0.59 0.38 1.53 0.13

MCH.Max -0.10 0.07 -1.48 0.14


rameters Accounting for Directionality for Outcome: Death

Estimate Std. Error z value Pr(>|z|)Neut#.Min 0.20 0.05 3.77 0.00

RespiratoryRate.Min -0.06 0.02 -3.67 0.00

MCV.Max 0.11 0.03 3.61 0.00

Platelets.Max -0.00 0.00 -3.49 0.00

AnionGap.Max 0.06 0.02 3.24 0.00

MCH.Max -0.24 0.08 -3.21 0.00

Pulse.Min 0.01 0.00 2.72 0.01

DiastolicBP.Min 0.02 0.01 2.71 0.01

Eos#.Max -0.66 0.25 -2.69 0.01

Chloride.Max 0.05 0.02 2.60 0.01

Sodium.Max -0.05 0.02 -2.50 0.01

(Intercept) -6.19 2.58 -2.39 0.02


WBC.Min -0.10 0.05 -2.24 0.03

WBC.Max 0.02 0.01 2.13 0.03


TotalCO2.Max 0.03 0.02 1.95 0.05

Potassium.Max 0.20 0.11 1.87 0.06

Age.Max 0.01 0.01 1.86 0.06

INR.Max 0.09 0.05 1.69 0.09

MCH.Min 0.15 0.10 1.54 0.12

MCV.Min -0.05 0.04 -1.48 0.14

RBC.Max 0.19 0.13 1.48 0.14

RDW.Min 0.07 0.05 1.42 0.16


rameters Accounting for Directionality for Outcome: Composite

Estimate Std. Error z value Pr(>|z|)RespiratoryRate.Min -0.06 0.02 -3.46 0.00

Neut#.Min 0.34 0.10 3.23 0.00

Platelets.Max -0.00 0.00 -2.94 0.00

AnionGap.Max 0.07 0.03 2.42 0.02

WBC.Min -0.23 0.10 -2.40 0.02

Pulse.Min 0.01 0.00 2.32 0.02

Eos#.Max -0.59 0.26 -2.24 0.02

Sodium.Max -0.06 0.03 -2.22 0.03


TotalCO2.Max 0.05 0.02 2.13 0.03


Chloride.Max 0.05 0.03 1.85 0.06

MCH.Min 0.30 0.18 1.72 0.09

INR.Max 0.09 0.05 1.71 0.09

Age.Max 0.01 0.01 1.60 0.11

MCV.Min -0.09 0.06 -1.49 0.14

MCV.Max 0.07 0.05 1.32 0.19

WBC.Max 0.03 0.03 1.11 0.27

Chloride.Min 0.03 0.03 1.11 0.27

MCH.Max -0.15 0.14 -1.08 0.28

Potassium.Max 0.13 0.12 1.07 0.28

Platelets.Min 0.00 0.00 0.99 0.32

VSFiO2.Max 0.00 0.00 0.93 0.35

Urea.Max 0.01 0.01 0.87 0.38

Sodium.Min -0.02 0.03 -0.83 0.41

MCHC.Min -0.01 0.02 -0.79 0.43

Temperature.Max -0.01 0.01 -0.75 0.45

MCHC.Max -0.01 0.01 -0.72 0.47

O2SaturationArterial.Min -0.01 0.01 -0.68 0.49

MPV.Min 0.09 0.14 0.67 0.50

Neut#.Max -0.02 0.03 -0.62 0.54

Baso#.Max 0.24 0.41 0.60 0.55

Lymp#.Min 0.08 0.15 0.57 0.57


Creatinine.Max -0.00 0.00 -0.57 0.57

Lymp#.Max 0.04 0.08 0.54 0.59

Potassium.Min 0.11 0.21 0.53 0.60

DiastolicBP.Max -0.00 0.00 -0.47 0.64

SystolicBP.Min 0.00 0.00 0.43 0.67

Pulse.Max -0.00 0.00 -0.42 0.67

RBC.Max 0.27 0.71 0.38 0.70

SystolicBP.Max -0.00 0.00 -0.34 0.73

RDW.Max 0.02 0.06 0.34 0.73

RDW.Min 0.02 0.08 0.31 0.75

MPV.Max -0.03 0.09 -0.30 0.76

(Intercept) 2.92 9.65 0.30 0.76

EstimatedGFR.Min 0.00 0.00 0.25 0.80

TotalCO2.Min -0.01 0.03 -0.22 0.83

RBC.Min -0.14 0.97 -0.14 0.89

HCT.Max 1.09 9.37 0.12 0.91

Hb.Min -0.00 0.04 -0.10 0.92

Mono#.Max 0.01 0.13 0.09 0.93

Temperature.Min -0.00 0.01 -0.04 0.97

Hb.Max 0.00 0.03 0.03 0.98

HCT.Min 0.31 14.37 0.02 0.98


counting for Directionality for Outcome: Composite


AnionGap.Max 0.11 0.05 2.19 0.03

SystolicBP.Min -0.01 0.01 -1.95 0.05

RBC.Min 3.10 1.74 1.78 0.07

MCV.Min -0.18 0.11 -1.65 0.10


(Intercept) -26.84 16.90 -1.59 0.11

Hb.Max -0.08 0.06 -1.53 0.13

Hb.Min -0.13 0.08 -1.52 0.13

Neut#.Max 0.09 0.07 1.31 0.19

MCHC.Min 0.03 0.03 1.20 0.23

Lymp#.Min 0.29 0.25 1.19 0.24

Sodium.Max -0.06 0.05 -1.15 0.25

Potassium.Min 0.47 0.42 1.12 0.26

Chloride.Max 0.05 0.05 1.07 0.28

Age.Max 0.01 0.01 1.03 0.30

Urea.Max 0.02 0.02 1.03 0.30

MCH.Min 0.32 0.31 1.03 0.31

WBC.Min -0.12 0.12 -1.01 0.31

MPV.Max -0.17 0.17 -0.99 0.32

Mono#.Max 0.17 0.18 0.95 0.34

TotalCO2.Min 0.06 0.06 0.94 0.35

Pulse.Max 0.00 0.00 0.93 0.35


RBC.Max 1.11 1.23 0.91 0.36

SystolicBP.Max -0.00 0.00 -0.83 0.41


RDW.Max -0.08 0.10 -0.74 0.46


WBC.Max -0.04 0.06 -0.66 0.51

TotalCO2.Max 0.03 0.05 0.64 0.52


Neut#.Min 0.08 0.13 0.62 0.54

Platelets.Min -0.00 0.00 -0.58 0.56

Platelets.Max -0.00 0.00 -0.58 0.56

INR.Max -0.06 0.11 -0.55 0.58

RDW.Min -0.08 0.14 -0.55 0.58

MPV.Min 0.14 0.27 0.53 0.60

Chloride.Min 0.03 0.05 0.53 0.60

Eos#.Max -0.24 0.46 -0.53 0.60

Sodium.Min -0.03 0.06 -0.51 0.61

Lymp#.Max -0.07 0.14 -0.48 0.63

HCT.Max 7.73 17.29 0.45 0.65

O2SaturationArterial.Min 0.01 0.01 0.44 0.66

Creatinine.Max 0.00 0.00 0.42 0.67

Baso#.Max -0.21 0.65 -0.32 0.75

MCHC.Max 0.01 0.02 0.30 0.76


HCT.Min 7.35 28.62 0.26 0.80

DiastolicBP.Min -0.00 0.02 -0.23 0.81

MCH.Max -0.04 0.17 -0.23 0.82

Pulse.Min 0.00 0.01 0.14 0.89

VSFiO2.Max -0.00 0.01 -0.13 0.90


Potassium.Max 0.02 0.22 0.07 0.94


counting for Directionality for Outcome: CodeBlue


MCH.Max -0.31 0.08 -3.95 0.00


MCH.Min 0.36 0.10 3.64 0.00

TotalCO2.Max 0.06 0.02 3.25 0.00

Sodium.Max -0.05 0.01 -3.08 0.00

Hb.Max 0.02 0.01 2.79 0.01

SystolicBP.Min 0.01 0.00 2.79 0.01

WBC.Max 0.02 0.01 2.72 0.01

Hb.Min -0.01 0.01 -2.53 0.01


MCV.Max 0.05 0.03 1.97 0.05

TotalCO2.Min -0.04 0.02 -1.94 0.05


Pulse.Max -0.00 0.00 -1.89 0.06

MCV.Min -0.07 0.04 -1.88 0.06

Chloride.Min 0.03 0.02 1.84 0.07

Mono#.Max -0.20 0.13 -1.54 0.12

Age.Max -0.01 0.01 -1.50 0.13

Lymp#.Min -0.08 0.05 -1.40 0.16

DiastolicBP.Max -0.00 0.00 -1.40 0.16

(Intercept) 2.84 2.43 1.17 0.24


rameters Accounting for Directionality for Outcome: ICUTransfer



Sodium.Max -0.08 0.03 -2.93 0.00

TotalCO2.Max 0.06 0.02 2.62 0.01

SystolicBP.Min 0.01 0.00 2.41 0.02

MCH.Max -0.32 0.15 -2.19 0.03


WBC.Max 0.04 0.02 1.75 0.08

Pulse.Max -0.00 0.00 -1.65 0.10

DiastolicBP.Max -0.00 0.00 -1.44 0.15

Mono#.Max -0.19 0.14 -1.39 0.17

Age.Max -0.01 0.01 -1.39 0.17

MCV.Max 0.07 0.05 1.33 0.18

Chloride.Max 0.03 0.03 1.32 0.19

Pulse.Min 0.01 0.00 1.25 0.21

MCH.Min 0.22 0.18 1.25 0.21

Potassium.Min -0.26 0.21 -1.21 0.23

Platelets.Max -0.00 0.00 -1.19 0.24

Neut#.Max -0.03 0.03 -1.12 0.26

Hb.Max 0.03 0.03 1.09 0.27

Potassium.Max 0.13 0.12 1.06 0.29

Lymp#.Min -0.13 0.13 -1.04 0.30

VSFiO2.Max 0.00 0.00 1.04 0.30

Chloride.Min 0.03 0.03 0.94 0.35


Urea.Max -0.01 0.01 -0.85 0.40

MCV.Min -0.05 0.06 -0.79 0.43

Creatinine.Max 0.00 0.00 0.77 0.44

SystolicBP.Max 0.00 0.00 0.70 0.49


RDW.Min -0.05 0.08 -0.62 0.53

RBC.Max -0.44 0.72 -0.61 0.54

AnionGap.Max 0.02 0.03 0.61 0.55

Neut#.Min 0.04 0.08 0.56 0.57

Eos#.Max -0.14 0.25 -0.55 0.58

Hb.Min -0.02 0.04 -0.46 0.65


MCHC.Min 0.01 0.02 0.43 0.66

(Intercept) 3.75 9.80 0.38 0.70

RBC.Min 0.33 0.98 0.34 0.73

Baso#.Max 0.12 0.35 0.33 0.74

MPV.Max -0.03 0.09 -0.29 0.77

Temperature.Min -0.00 0.01 -0.26 0.79

Platelets.Min -0.00 0.00 -0.25 0.80

Lymp#.Max 0.02 0.07 0.24 0.81

INR.Max 0.01 0.05 0.23 0.82

Temperature.Max -0.00 0.01 -0.20 0.84

Sodium.Min 0.01 0.03 0.20 0.84

MPV.Min -0.02 0.14 -0.15 0.88

HCT.Min -2.20 14.47 -0.15 0.88

TotalCO2.Min -0.00 0.03 -0.15 0.88

MCHC.Max -0.00 0.01 -0.12 0.90

RDW.Max -0.01 0.06 -0.11 0.92

WBC.Min 0.00 0.07 0.04 0.97

HCT.Max -0.19 9.16 -0.02 0.98


counting for Directionality for Outcome: ICUTransfer

Estimate Std. Error z value Pr(>|z|)Age.Max 0.04 0.01 5.64 0.00

RDW.Min 0.26 0.08 3.14 0.00

Platelets.Max -0.00 0.00 -2.96 0.00

Neut#.Min 0.25 0.09 2.64 0.01

Pulse.Min 0.01 0.01 2.63 0.01

INR.Max 0.12 0.05 2.32 0.02

MCV.Max 0.11 0.05 2.13 0.03

Potassium.Min 0.41 0.22 1.84 0.07

(Intercept) -18.84 10.37 -1.82 0.07

MPV.Min 0.25 0.15 1.72 0.09

Potassium.Max 0.22 0.13 1.70 0.09

Urea.Max 0.02 0.01 1.69 0.09

Chloride.Max 0.05 0.03 1.68 0.09

Platelets.Min 0.00 0.00 1.57 0.12

Creatinine.Max -0.00 0.00 -1.55 0.12

Baso#.Max 0.76 0.50 1.53 0.13

RBC.Max 1.13 0.75 1.52 0.13

Hb.Min -0.07 0.04 -1.47 0.14

WBC.Min -0.13 0.09 -1.44 0.15

Eos#.Max -0.40 0.28 -1.43 0.15

AnionGap.Max 0.04 0.03 1.40 0.16

MCH.Min 0.26 0.19 1.37 0.17

Chloride.Min -0.04 0.03 -1.32 0.19

Mono#.Max 0.17 0.13 1.26 0.21

HCT.Min 18.77 15.43 1.22 0.22

Lymp#.Min 0.16 0.13 1.22 0.22



SystolicBP.Max -0.00 0.00 -1.14 0.25

MCV.Min -0.07 0.07 -1.07 0.29

Temperature.Min -0.01 0.01 -0.89 0.37


SystolicBP.Min -0.00 0.00 -0.87 0.38

Lymp#.Max -0.06 0.06 -0.87 0.39

TotalCO2.Min -0.03 0.03 -0.85 0.40

TotalCO2.Max -0.02 0.02 -0.78 0.44


HCT.Max -6.93 9.51 -0.73 0.47

MCH.Max -0.11 0.15 -0.71 0.48

RDW.Max -0.04 0.06 -0.66 0.51

Pulse.Max -0.00 0.00 -0.59 0.55

VSFiO2.Max -0.00 0.00 -0.44 0.66

Sodium.Min -0.01 0.03 -0.37 0.71

Sodium.Max -0.01 0.03 -0.35 0.72

DiastolicBP.Min -0.00 0.01 -0.26 0.79

WBC.Max 0.00 0.01 0.25 0.80


MCHC.Max 0.00 0.02 0.21 0.83

MCHC.Min -0.00 0.02 -0.19 0.85

Hb.Max -0.00 0.03 -0.16 0.87

Neut#.Max 0.00 0.02 0.08 0.94


MPV.Max -0.01 0.09 -0.06 0.95


RBC.Min 0.04 1.04 0.04 0.97


counting for Directionality for Outcome: Death


MCV.Min -0.22 0.07 -3.29 0.00

MCH.Min 0.68 0.22 3.03 0.00

EstimatedGFR.Max 0.01 0.00 2.76 0.01

(Intercept) -13.32 5.30 -2.51 0.01


Eos#.Min -13.86 6.30 -2.20 0.03


Hb.Min -0.10 0.05 -2.07 0.04

RBC.Min 2.77 1.36 2.04 0.04

INR.Min 0.67 0.33 2.03 0.04

Baso#.Min 7.47 3.71 2.01 0.04


Baso#.Max 0.77 0.39 1.97 0.05

Potassium.Min 0.66 0.35 1.87 0.06

Platelets.Min 0.00 0.00 1.80 0.07


Creatinine.Min 0.00 0.00 1.67 0.09

MCH.Max -0.31 0.19 -1.64 0.10

Mono#.Max -0.56 0.34 -1.63 0.10

Platelets.Max -0.00 0.00 -1.56 0.12

Potassium.Max -0.35 0.24 -1.50 0.13


rameters for Outcome: CardiacArrest

Estimate Std. Error z value Pr(>|z|)INR.Min 1.14 0.44 2.61 0.01


MCH.Min 0.87 0.37 2.37 0.02

Potassium.Min 0.98 0.42 2.35 0.02

Platelets.Min 0.01 0.00 2.23 0.03

Eos#.Min -14.23 6.85 -2.08 0.04

Platelets.Max -0.00 0.00 -2.06 0.04

MCV.Min -0.25 0.12 -2.02 0.04


Potassium.Max -0.53 0.27 -1.95 0.05



Creatinine.Min 0.01 0.00 1.78 0.07

Baso#.Min 9.34 5.28 1.77 0.08

MCV.Max 0.16 0.09 1.76 0.08

RBC.Min 2.77 1.68 1.65 0.10


INR.Max -0.18 0.13 -1.41 0.16


Pulse.Max 0.00 0.00 1.24 0.22

Hb.Min -0.08 0.07 -1.14 0.26

TotalCO2.Max 0.06 0.06 1.00 0.32

Mono#.Max -0.45 0.46 -0.98 0.33

Creatinine.Max -0.00 0.00 -0.98 0.33

AnionGap.Max 0.06 0.06 0.97 0.33

Eos#.Max 0.44 0.46 0.96 0.34

Baso#.Max 0.60 0.64 0.94 0.35

SystolicBP.Min -0.01 0.01 -0.91 0.36

MCH.Max -0.26 0.29 -0.87 0.38


MCHC.Min -0.03 0.03 -0.79 0.43

Age.Min 0.27 0.36 0.76 0.45

MPV.Min 0.20 0.26 0.75 0.45

Urea.Max 0.02 0.03 0.74 0.46

Age.Max -0.26 0.36 -0.72 0.47

(Intercept) -12.77 20.44 -0.62 0.53

Chloride.Max 0.03 0.06 0.57 0.57

Hb.Max -0.03 0.06 -0.53 0.59

Mono#.Min -0.58 1.22 -0.48 0.63


Sodium.Min -0.03 0.07 -0.45 0.65

VSFiO2.Min -2.05 4.53 -0.45 0.65

Chloride.Min 0.03 0.06 0.42 0.68

Sodium.Max -0.02 0.06 -0.39 0.70

SystolicBP.Max -0.00 0.00 -0.38 0.70

RDW.Max -0.04 0.11 -0.37 0.71

Urea.Min -0.02 0.04 -0.36 0.72

Lymp#.Max -0.05 0.15 -0.35 0.72

MPV.Max -0.07 0.19 -0.35 0.73

RBC.Max 0.44 1.31 0.34 0.74

RDW.Min -0.05 0.15 -0.33 0.74

WBC.Min -0.05 0.17 -0.31 0.75

HCT.Max 5.28 17.07 0.31 0.76

Lymp#.Min 0.08 0.25 0.30 0.76

HCT.Min -5.12 17.49 -0.29 0.77

Pulse.Min 0.00 0.01 0.29 0.77

MCHC.Max 0.01 0.03 0.27 0.79

WBC.Max 0.01 0.03 0.25 0.81

TotalCO2.Min -0.01 0.06 -0.17 0.86

Neut#.Min 0.03 0.17 0.17 0.87

Neut#.Max -0.01 0.05 -0.12 0.90

AnionGap.Min -0.01 0.09 -0.10 0.92

VSFiO2.Max 0.00 0.01 0.02 0.98


Table B.23: Table of Base Logistic Regression Model Parameters for

Outcome: CardiacArrest

Estimate Std. Error z value Pr(>|z|)(Intercept) -13.97 4.10 -3.40 0.00

MCV.Max 0.11 0.03 3.36 0.00

INR.Min 0.97 0.30 3.23 0.00

Creatinine.Min 0.00 0.00 3.18 0.00

MCV.Min -0.16 0.05 -2.99 0.00

Baso#.Min 8.79 3.51 2.50 0.01

MCH.Min 0.40 0.16 2.42 0.02

RBC.Min 2.28 1.08 2.12 0.03


Mono#.Min -1.56 0.77 -2.04 0.04

Hb.Min -0.07 0.04 -1.96 0.05

Neut#.Max 0.03 0.01 1.96 0.05

Eos#.Min -9.37 5.11 -1.83 0.07

Age.Min 0.02 0.01 1.76 0.08

AnionGap.Max 0.05 0.03 1.76 0.08


Potassium.Min 0.50 0.29 1.69 0.09

HCT.Max -4.88 2.98 -1.64 0.10




rameters for Outcome: CodeBlue

Estimate Std. Error z value Pr(>|z|)INR.Min 1.16 0.37 3.13 0.00

Creatinine.Min 0.01 0.00 2.55 0.01

MCV.Max 0.16 0.07 2.22 0.03

MCV.Min -0.19 0.10 -2.01 0.04

Mono#.Min -1.80 0.95 -1.89 0.06




Baso#.Min 7.79 4.37 1.78 0.07


Eos#.Min -9.65 5.72 -1.69 0.09


Potassium.Min 0.55 0.35 1.55 0.12

RDW.Max -0.14 0.10 -1.44 0.15

AnionGap.Max 0.07 0.05 1.38 0.17

Urea.Max 0.03 0.02 1.33 0.18

Lymp#.Min 0.27 0.20 1.32 0.19

MCH.Min 0.37 0.28 1.31 0.19

Neut#.Min 0.14 0.11 1.21 0.23

SystolicBP.Min -0.01 0.01 -1.17 0.24

INR.Max -0.12 0.10 -1.15 0.25

Neut#.Max 0.04 0.04 1.06 0.29

Creatinine.Max -0.00 0.00 -1.05 0.29

WBC.Min -0.12 0.11 -1.05 0.29

Age.Min 0.35 0.35 1.01 0.31

Age.Max -0.33 0.35 -0.96 0.33

(Intercept) -15.04 15.77 -0.95 0.34

Hb.Min -0.06 0.07 -0.92 0.36

RBC.Min 1.35 1.49 0.91 0.36

RDW.Min 0.11 0.13 0.86 0.39

Mono#.Max 0.15 0.18 0.84 0.40

RBC.Max 0.88 1.08 0.81 0.42

Pulse.Max 0.00 0.00 0.78 0.43

HCT.Max -11.42 14.99 -0.76 0.45

Potassium.Max -0.15 0.21 -0.70 0.48

MPV.Max -0.11 0.16 -0.66 0.51


WBC.Max -0.02 0.03 -0.60 0.55

Sodium.Max -0.03 0.05 -0.59 0.56

SystolicBP.Max -0.00 0.00 -0.55 0.58

TotalCO2.Max 0.03 0.05 0.52 0.60

AnionGap.Min -0.04 0.08 -0.49 0.62


Platelets.Max -0.00 0.00 -0.48 0.63


Lymp#.Max -0.06 0.13 -0.48 0.63

VSFiO2.Min 1.23 2.69 0.46 0.65

Chloride.Max 0.02 0.05 0.45 0.65

Urea.Min -0.01 0.03 -0.42 0.67

Pulse.Min 0.00 0.01 0.42 0.67

Eos#.Max -0.16 0.42 -0.38 0.70

MCHC.Max 0.01 0.02 0.38 0.71

MPV.Min 0.08 0.23 0.34 0.74

VSFiO2.Max 0.00 0.01 0.32 0.75

HCT.Min 6.95 22.63 0.31 0.76


MCH.Max -0.05 0.20 -0.24 0.81

Hb.Max -0.01 0.05 -0.22 0.83

Chloride.Min 0.01 0.05 0.21 0.83

MCHC.Min -0.00 0.03 -0.16 0.87

Platelets.Min 0.00 0.00 0.05 0.96

TotalCO2.Min -0.00 0.05 -0.03 0.98

Sodium.Min 0.00 0.06 0.02 0.98

Baso#.Max 0.00 0.60 0.01 0.99


Outcome: CodeBlue

Estimate Std. Error z value Pr(>|z|)RDW.Min 0.25 0.06 3.92 0.00

Neut#.Min 0.28 0.08 3.47 0.00

Potassium.Max 0.35 0.11 3.23 0.00

Pulse.Min 0.01 0.00 3.21 0.00

Urea.Min 0.07 0.02 3.14 0.00

Platelets.Max -0.00 0.00 -2.72 0.01


INR.Min 0.77 0.29 2.64 0.01

INR.Max 0.12 0.04 2.63 0.01

Potassium.Min 0.45 0.17 2.58 0.01

MCV.Min -0.13 0.06 -2.36 0.02

Eos#.Min -3.79 1.67 -2.28 0.02

Age.Min 0.26 0.13 2.02 0.04

MCH.Min 0.30 0.16 1.95 0.05

MPV.Min 0.21 0.11 1.93 0.05

WBC.Min -0.15 0.08 -1.92 0.06

AnionGap.Max 0.05 0.03 1.91 0.06

MCV.Max 0.08 0.04 1.88 0.06

TotalCO2.Max -0.05 0.03 -1.82 0.07

Lymp#.Min 0.20 0.11 1.77 0.08

Mono#.Min -0.80 0.46 -1.72 0.08

Age.Max -0.22 0.13 -1.72 0.09


Eos#.Max -0.40 0.24 -1.65 0.10

Baso#.Max 0.69 0.44 1.57 0.12


Chloride.Min -0.04 0.03 -1.57 0.12

Creatinine.Min -0.00 0.00 -1.53 0.13

MCHC.Min -0.02 0.02 -1.45 0.15

RDW.Max -0.07 0.05 -1.44 0.15

DiastolicBP.Min -0.01 0.01 -1.42 0.16

Sodium.Max 0.03 0.02 1.39 0.17

Platelets.Min 0.00 0.00 1.35 0.18

HCT.Min 14.37 11.77 1.22 0.22



SystolicBP.Max -0.00 0.00 -1.06 0.29

SystolicBP.Min -0.00 0.00 -1.04 0.30

MCH.Max -0.13 0.13 -1.03 0.31

Temperature.Min -0.01 0.01 -0.95 0.34

(Intercept) -8.19 8.65 -0.95 0.34

Hb.Min -0.03 0.04 -0.82 0.41

Baso#.Min 1.80 2.36 0.76 0.45

RBC.Min -0.60 0.80 -0.75 0.45

Mono#.Max 0.09 0.13 0.67 0.50

Lymp#.Max -0.03 0.06 -0.63 0.53

VSFiO2.Max 0.00 0.00 0.56 0.57

TotalCO2.Min -0.02 0.03 -0.56 0.58

AnionGap.Min -0.02 0.04 -0.53 0.60

WBC.Max -0.01 0.01 -0.52 0.60

Neut#.Max 0.01 0.02 0.50 0.62

DiastolicBP.Max -0.00 0.00 -0.47 0.64

RBC.Max 0.24 0.61 0.40 0.69

Urea.Max -0.00 0.01 -0.35 0.72

Hb.Max 0.01 0.02 0.30 0.77

Creatinine.Max -0.00 0.00 -0.29 0.77

MCHC.Max 0.00 0.01 0.28 0.78

MPV.Max 0.02 0.08 0.24 0.81

HCT.Max -1.75 7.93 -0.22 0.83

Chloride.Max -0.00 0.02 -0.15 0.88

Pulse.Max 0.00 0.00 0.12 0.90

VSFiO2.Min 0.13 1.09 0.12 0.90


Sodium.Min -0.00 0.03 -0.01 0.99


Outcome: Death


Neut#.Min 0.36 0.08 4.35 0.00

MCV.Min -0.14 0.04 -3.92 0.00

AnionGap.Max 0.06 0.02 3.75 0.00

Urea.Min 0.05 0.01 3.65 0.00

WBC.Min -0.27 0.08 -3.58 0.00


Potassium.Max 0.29 0.09 3.23 0.00

MCH.Min 0.40 0.12 3.23 0.00

Pulse.Min 0.01 0.00 3.16 0.00


RDW.Min 0.16 0.05 2.88 0.00

Eos#.Min -3.35 1.20 -2.78 0.01

Eos#.Max -0.56 0.21 -2.62 0.01

INR.Min 0.62 0.25 2.48 0.01

MPV.Min 0.17 0.07 2.42 0.02

INR.Max 0.10 0.04 2.40 0.02

Lymp#.Min 0.23 0.10 2.38 0.02

MCH.Max -0.16 0.07 -2.30 0.02


MCHC.Min -0.03 0.01 -2.08 0.04

Age.Min 0.01 0.00 1.90 0.06

MCV.Max 0.06 0.03 1.87 0.06

WBC.Max 0.01 0.01 1.86 0.06

VSFiO2.Max 0.01 0.00 1.82 0.07

VSFiO2.Min 2.80 1.56 1.80 0.07

AnionGap.Min -0.05 0.03 -1.78 0.07

RDW.Max -0.07 0.04 -1.61 0.11

HCT.Max 1.68 1.10 1.52 0.13


(Intercept) -0.46 4.15 -0.11 0.91


rameters for Outcome: Composite


Neut#.Min 0.35 0.09 3.90 0.00

MCV.Min -0.17 0.05 -3.39 0.00


MCH.Min 0.43 0.14 3.01 0.00

Pulse.Min 0.01 0.00 2.92 0.00

WBC.Min -0.25 0.09 -2.91 0.00

Eos#.Min -3.83 1.33 -2.88 0.00

RDW.Min 0.15 0.06 2.66 0.01

Eos#.Max -0.55 0.22 -2.54 0.01

Urea.Min 0.05 0.02 2.51 0.01

AnionGap.Max 0.06 0.02 2.37 0.02

INR.Min 0.60 0.26 2.30 0.02

INR.Max 0.09 0.04 2.21 0.03

Potassium.Max 0.22 0.10 2.18 0.03


VSFiO2.Max 0.01 0.00 1.94 0.05

MCHC.Min -0.03 0.01 -1.89 0.06

VSFiO2.Min 2.73 1.64 1.67 0.10

MPV.Min 0.16 0.10 1.61 0.11


WBC.Max 0.03 0.02 1.60 0.11

RDW.Max -0.07 0.04 -1.60 0.11

Potassium.Min 0.24 0.16 1.52 0.13


Platelets.Max -0.00 0.00 -1.22 0.22

Neut#.Max -0.03 0.02 -1.13 0.26


Platelets.Min 0.00 0.00 1.07 0.29

HCT.Min 7.76 7.30 1.06 0.29

Baso#.Min 1.90 1.93 0.98 0.32

DiastolicBP.Max -0.00 0.00 -0.96 0.34

TotalCO2.Max 0.02 0.02 0.90 0.37

TotalCO2.Min -0.02 0.02 -0.90 0.37

Lymp#.Min 0.11 0.12 0.88 0.38

MCV.Max 0.03 0.04 0.88 0.38

Baso#.Max 0.35 0.40 0.87 0.38

AnionGap.Min -0.03 0.04 -0.87 0.38

MCH.Max -0.09 0.10 -0.87 0.39

Mono#.Min -0.32 0.40 -0.80 0.42

Hb.Min -0.02 0.03 -0.80 0.42

(Intercept) 5.42 7.56 0.72 0.47

RBC.Max -0.40 0.57 -0.71 0.48

Creatinine.Min -0.00 0.00 -0.68 0.50

Mono#.Max -0.08 0.12 -0.67 0.51

MCHC.Max -0.01 0.01 -0.66 0.51

Sodium.Min -0.01 0.02 -0.61 0.54

HCT.Max 3.91 7.46 0.52 0.60

Lymp#.Max 0.03 0.07 0.51 0.61

Temperature.Min -0.00 0.01 -0.46 0.65

Creatinine.Max -0.00 0.00 -0.43 0.67

Hb.Max 0.01 0.02 0.42 0.67

Chloride.Min 0.01 0.02 0.37 0.71

Pulse.Max 0.00 0.00 0.34 0.74

Age.Min 0.03 0.10 0.31 0.76

Temperature.Max -0.00 0.01 -0.28 0.78

SystolicBP.Max -0.00 0.00 -0.27 0.79

Urea.Max 0.00 0.01 0.24 0.81

Age.Max -0.02 0.10 -0.23 0.82

RBC.Min -0.14 0.68 -0.20 0.84

MPV.Max 0.00 0.07 0.06 0.95

Chloride.Max 0.00 0.02 0.05 0.96

Sodium.Max -0.00 0.02 -0.04 0.97

SystolicBP.Min 0.00 0.00 0.01 0.99


Outcome: Composite



TotalCO2.Max 0.07 0.01 4.67 0.00

MCH.Max -0.27 0.06 -4.43 0.00


MCH.Min 0.44 0.12 3.56 0.00

TotalCO2.Min -0.06 0.02 -3.18 0.00

Sodium.Max -0.04 0.01 -2.87 0.00

WBC.Max 0.02 0.01 2.77 0.01

SystolicBP.Min 0.01 0.00 2.50 0.01

Eos#.Min -3.72 1.52 -2.45 0.01

Age.Min -0.01 0.00 -2.44 0.01

(Intercept) 8.16 3.59 2.27 0.02

Hb.Min -0.01 0.01 -2.19 0.03

Pulse.Min 0.01 0.00 2.19 0.03


DiastolicBP.Max -0.00 0.00 -2.06 0.04

Hb.Max 0.01 0.00 1.95 0.05

Mono#.Max -0.23 0.12 -1.88 0.06


MCHC.Min -0.02 0.01 -1.71 0.09

VSFiO2.Max 0.00 0.00 1.59 0.11

MCV.Min -0.05 0.04 -1.48 0.14

Lymp#.Min -0.08 0.06 -1.44 0.15


rameters for Outcome: ICUTransfer



MCH.Min 0.44 0.16 2.73 0.01

Eos#.Min -4.10 1.67 -2.45 0.01


SystolicBP.Min 0.01 0.00 2.26 0.02

MCV.Min -0.11 0.06 -2.07 0.04

WBC.Max 0.05 0.02 2.04 0.04

DiastolicBP.Max -0.00 0.00 -2.04 0.04

TotalCO2.Max 0.05 0.03 2.01 0.04

Sodium.Max -0.05 0.03 -1.93 0.05

Pulse.Min 0.01 0.00 1.91 0.06

TotalCO2.Min -0.05 0.03 -1.75 0.08

Age.Min -0.20 0.12 -1.71 0.09

Age.Max 0.19 0.12 1.60 0.11

VSFiO2.Max 0.01 0.00 1.57 0.12

AnionGap.Min -0.06 0.04 -1.52 0.13

Neut#.Max -0.04 0.03 -1.50 0.13

MCH.Max -0.18 0.13 -1.38 0.17

Hb.Min -0.05 0.03 -1.37 0.17

Mono#.Max -0.18 0.14 -1.30 0.19

RDW.Max -0.06 0.05 -1.24 0.22


Urea.Min 0.03 0.02 1.24 0.22

INR.Min -0.38 0.32 -1.18 0.24


Temperature.Min -0.01 0.01 -1.12 0.26

(Intercept) 9.89 8.90 1.11 0.27

Potassium.Min -0.19 0.19 -1.00 0.32

Pulse.Max -0.00 0.00 -0.96 0.34

Neut#.Min 0.07 0.07 0.94 0.35

Lymp#.Min -0.12 0.12 -0.93 0.35

Baso#.Max 0.30 0.35 0.85 0.39

Potassium.Max 0.09 0.11 0.82 0.41

Urea.Max -0.01 0.01 -0.75 0.45

AnionGap.Max 0.02 0.03 0.74 0.46

Hb.Max 0.02 0.02 0.73 0.46

RBC.Min 0.60 0.82 0.73 0.47

RBC.Max -0.47 0.66 -0.72 0.47

Eos#.Max -0.16 0.22 -0.72 0.47

RDW.Min 0.04 0.07 0.68 0.50

MCHC.Max -0.01 0.01 -0.67 0.50

SystolicBP.Max 0.00 0.00 0.67 0.50

Sodium.Min 0.02 0.03 0.62 0.54

WBC.Min -0.04 0.07 -0.60 0.55

Baso#.Min 1.33 2.28 0.58 0.56

HCT.Min 5.02 9.03 0.56 0.58

VSFiO2.Min 0.87 1.59 0.55 0.59

Temperature.Max -0.01 0.01 -0.51 0.61

MCHC.Min -0.01 0.02 -0.51 0.61

MCV.Max 0.02 0.04 0.50 0.62

MPV.Min 0.05 0.12 0.43 0.67

Mono#.Min -0.15 0.39 -0.38 0.70

Platelets.Max 0.00 0.00 0.30 0.76

HCT.Max 2.39 8.31 0.29 0.77

EstimatedGFR.Max -0.00 0.00 -0.26 0.80

Creatinine.Min 0.00 0.00 0.25 0.80

Chloride.Max 0.01 0.02 0.25 0.81

INR.Max 0.01 0.05 0.21 0.84

Platelets.Min 0.00 0.00 0.18 0.86

Creatinine.Max -0.00 0.00 -0.18 0.86

Lymp#.Max 0.01 0.06 0.12 0.91

Chloride.Min -0.00 0.03 -0.08 0.93

MPV.Max -0.00 0.08 -0.04 0.97


Outcome: ICUTransfer

Estimate Std. Error z value Pr(>|z|)RDW.Min 0.27 0.06 4.31 0.00

Neut#.Min 0.20 0.05 4.16 0.00

Potassium.Max 0.38 0.10 3.84 0.00


Urea.Min 0.06 0.02 3.83 0.00

MCV.Min -0.15 0.04 -3.69 0.00

Pulse.Min 0.01 0.00 3.37 0.00

AnionGap.Max 0.05 0.02 2.96 0.00


TotalCO2.Max -0.04 0.02 -2.92 0.00

INR.Min 0.82 0.28 2.89 0.00

INR.Max 0.12 0.04 2.81 0.01

MCH.Min 0.37 0.14 2.75 0.01

MPV.Min 0.23 0.08 2.74 0.01

Chloride.Min -0.04 0.01 -2.60 0.01

Potassium.Min 0.43 0.17 2.59 0.01

Platelets.Max -0.00 0.00 -2.53 0.01

Sodium.Max 0.03 0.01 2.35 0.02

Eos#.Min -3.54 1.54 -2.30 0.02

Baso#.Max 0.83 0.36 2.30 0.02

MCV.Max 0.06 0.03 2.17 0.03

Creatinine.Min -0.00 0.00 -2.17 0.03

MCHC.Min -0.03 0.01 -2.09 0.04

DiastolicBP.Min -0.01 0.01 -2.07 0.04

Age.Min 0.26 0.12 2.06 0.04

RBC.Max 0.28 0.15 1.85 0.06

(Intercept) -8.95 4.97 -1.80 0.07

RDW.Max -0.08 0.05 -1.77 0.08

Hb.Min -0.05 0.03 -1.76 0.08

Age.Max -0.22 0.12 -1.74 0.08

Eos#.Max -0.40 0.23 -1.71 0.09


HCT.Min 15.57 9.94 1.57 0.12

Mono#.Min -0.63 0.41 -1.53 0.13

WBC.Min -0.06 0.05 -1.39 0.16


rameters for Outcome: Death


Figure B.28: Dependogram of Five Most Predictive Variables for Logistic Regression


Figure B.29: Dependogram of Five Most Predictive Variables for Logistic Regression, Accounting for Direc-tionality


Figure B.30: Dependogram of Five Most Predictive Variables for Random Forest Classification


Figure B.31: Dependogram of Five Most Predictive Variables for Random Forest Classification, Accountingfor Directionality


Figure B.32: Cleveland Dot Plot of p-Values for Random Forest Classification


Figure B.33: Cleveland Dot Plot of p-Values for Random Forest Classification, Accounting for Directionality


Figure B.34: Cleveland Dot Plot of p-Values for Logistic Regression, Accounting for Directionality


Figure B.35: Cleveland Dot Plot of p-Values for Stepwise Optimized Logistic Regression, Accounting forDirectionality


Figure B.36: Cleveland Dot Plot of p-Values for Logistic Regression


Figure B.37: Cleveland Dot Plot of p-Values for Stepwise Optimized Logistic Regression

Bibliography

[1] R. S. Braithwaite et al., “Use of medical emergency team (met) responses to detect medical errors,” Quality

and Safety in Health Care, vol. 13, pp. 255–259, 2004.

[2] A. Iyengar, A. Baxter, and A. Forster, “Using medical emergency teams to detect preventable adverse events,”

Critical Care, vol. 13, no. R126, 2009.

[3] D. Stockwell and A. Slonim, “Quality and safety in the intensive care unit,” Journal of Intensive Care Medicine,

vol. 21, no. 4, 2006.

[4] B. D. Winters et al., “Rapid-response systems as a patient safety strategy,” Annals of Internal Medicine, vol. 158,

pp. 417–425, 2013.

[5] M. A. Devita et al., ““identifying the hospitalised patient in crisis” aAT– a consensus conference on the afferent

limb of rapid response systems,” Resuscitation, vol. 81, pp. 375–382, 2010.

[6] M. A. Devita et al., “Findings of the first consensus conference on medical emergency teams,” Critical Care

Medicine, vol. 34, no. 9, pp. 2463–2478, 2006.

[7] B. Winters et al., “Rapid response teamsaATwalk, don’t run,” Journal of the American Medical Association,

vol. 296, no. 13, 2006.

[8] L. Esmonde, A. McDonnell, C. Ball, C. Waskett, R. Morgan, A. Rashidian, K. Bray, S. Adam, and S. Harvey,

“Investigating the effectiveness of critical care outreach services: a systematic review.,” Intensive care medicine,

vol. 32, no. 11, pp. 1713–21, 2006.

[9] S. Ranji et al., “Effects of rapid response systems on clinical outcomes: Systematic review and metaaARanaly-

sis,” Journal of Hospital Medicine, vol. 2, no. 6, pp. 422–432, 2007.

[10] J. McGaughey, F. Alderdice, R. Fowler, A. Kapila, A. Mayhew, and M. Moutray, “Outreach and early warning

systems (ews) for the prevention of intensive care admission and death of critically ill adult patients on general

hospital wards,” Cochrane Database of Systematic Reviews, vol. 3, no. CD005529, 2007.

[11] P. Chan, R. Jain, B. Nallmothu, R. Berg, and C. Sasson, “Rapid response teams: a systematic review and

meta-analysis,” Archives of internal medicine, vol. 170, no. 1, p. 18, 2010.

[12] GRADE Working Group, “Grading quality of evidence and strength of recommendations,” British Medical

Journal, vol. 328, no. 7454, 2004.

[13] D. A. Jones, M. A. DeVita, and R. Bellomo, “Rapid-response teams,” New England Journal of Medicine,

vol. 365, no. 2, pp. 139–146, 2011.

[14] J. C. on Accreditation of Healthcare Organizations, “The joint commission announces the 2009 national patient

safety goals,” Joint Commission Perspectives, vol. 28, no. 7, 2008.

[15] T. J. Commission, “National patient safety goals effective january 1, 2013:hospital accreditation program,” tech.

rep., The Joint Commission, 2012.

166

BIBLIOGRAPHY 167

[16] T. J. Commission, “National patient safety goals effective january 1, 2013: Summary presentation,” tech. rep.,

The Joint Commission, 2012.

[17] M. Armitage, “Acutely ill patients in hospital: Recognition of and response to acute illness in adults in hospital,”

tech. rep., Guideline Development Group, 2007.

[18] G. B. Smith, D. R. Prytherch, P. E. Schmidt, and P. I. Featherstone, “Review and performance evaluation of

aggregate weighted ‘track and trigger’ systems,” Resuscitation, vol. 77, no. 2, pp. 170 – 179, 2008.

[19] Australian Commission on Safety and Quality in Healthcare, “National consensus statement: Essential elements

for recognising and responding to clinical deterioration,” tech. rep., Commonwealth of Australia, 2010.

[20] R. McKay et al., “Ontario’s critical care strategy: Quarterly report,” tech. rep., Critical Care Secretariat,

Ontario Ministry of Health and Long-Term Care, March 2007.

[21] B. Lawless, R. McKay, S. Reynolds, et al., “Implementation of critical care response teams (ccrts) in ontario

hospitals — year one,” tech. rep., Critical Care Secretariat, Ontario Ministry of Health and Long-Term Care,

2007.

[22] College of Nurses of Ontario, “Medication, revised 2008,” tech. rep., Practice Standard, 2008.

[23] D. Prytherch et al., “ViEWS — towards a national early warning score for detecting adult inpatient deteriora-

tion,” Resuscitation, vol. 81, pp. 932–937, 2010.

[24] Report of a working party, “National early warning score (news): Standardising the assessment of acute-illness

severity in the nhs,” tech. rep., Royal College of Physicians, 2012.

[25] E. Loekito, J. Bailey, R. Bellomo, G. K. Hart, C. Hegarty, P. Davey, C. Bain, D. Pilcher, and H. Schneider,

“Common laboratory tests predict imminent death in ward patients,” Resuscitation, vol. 84, no. 3, pp. 280 –

285, 2013.

[26] S. W. Jarvis, C. Kovacs, T. Badriyah, J. Briggs, M. A. Mohammed, P. Meredith, P. E. Schmidt, P. I. Feath-

erstone, D. R. Prytherch, and G. B. Smith, “Development and validation of a decision tree early warning

score based on routine laboratory test results for the discrimination of hospital mortality in emergency medical

admissions,” Resuscitation, no. 0, pp. –, 2013.

[27] P. Calzavacca, E. Licari, A. Tee, and R. Bellomo, “Point-of-care testing during medical emergency team acti-

vations: A pilot study,” Resuscitation, vol. 83, no. 9, pp. 1119 – 1123, 2012.

[28] C. P. Subbe, “Effect of introducing the modified early warning score on clinical outcomes, cardio-pulmonary

arrests and intensive care utilisation in acute medical admissions,” Anesthesia, vol. 58, no. 8, pp. 797–802, 2003.

[29] C. Subbe et al., “Validation of a modified early warning score in medical admissions,” Quarterly Journal of

Medicine: An International Journal of Medicine, vol. 94, no. 10, 2001.

[30] G. B. Smith, “In-hospital cardiac arrest: Is it time for an in-hospital aAYchain of preventionaAZ?,” Resuscita-

tion, vol. 81, no. 9, pp. 1209 – 1211, 2010.

[31] A. G. Schneider, S. Warrillow, R. Robbins, D. A. Jones, and R. Bellomo, “An assessment of the triage perfor-

mance of the efferent arm of the rapid response system,” Resuscitation, vol. 84, no. 4, pp. 477 – 482, 2013.

[32] T. A. Brennan, A. R. Localio, L. L. Leape, N. M. Laird, L. Peterson, H. H. Hiatt, and B. A. Barnes, “Identifi-

cation of adverse events occurring during hospitalizationa cross-sectional study of litigation, quality assurance,

and medical records at two teaching hospitals,” Annals of Internal Medicine, vol. 112, no. 3, pp. 221–226, 1990.

[33] A. J. Forster, M. Taljaard, C. Bennett, and C. van Walraven, “Reliability of the peer-review process for adverse

event rating,” PLoS ONE, vol. 7, p. e41239, 07 2012.

BIBLIOGRAPHY 168

[34] Alvaro Silva, “Mortality assessment in intensive care units via adverse events using artificial neural networks,”

Artificial Intelligence in Medicine, vol. 36, no. 3, pp. 223–234, 2006.

[35] A. H. Taenzer et al., “A review of current and emerging approaches to address failure-to-rescue,” Anesthesiology,

vol. 115, no. 2, pp. 421–431, 2011.

[36] C. W. Hug, Detecting Hazardous Intensive Care Patient Episodes Using Real-time Mortality Models. PhD

thesis, Massachusetts Institute of Technology, 2009.

[37] W. A. Knaus et al., “Apache — acute physiology and chronic health evaluation: a physiologically based classi-

fication system,” Critical Care Medicine, vol. 9, no. 8, 1981.

[38] W. A. Knaus et al., “Apache ii: a severity of disease classification system,” Critical Care Medicine, vol. 13,

no. 10, 1985.

[39] W. A. Knaus et al., “The apache iii prognostic system. risk prediction of hospital mortality for critically ill

hospitalized adults,” Chest, vol. 100, no. 6, pp. 1619–1636, 1991.

[40] J. Zimmerman et al., “Acute physiology and chronic health evaluation (apache) iv: Hospital mortality assess-

ment for today’s critically ill patients,” Critical Care Medicine, vol. 34, no. 5, pp. 1297–1310, 2006.

[41] J.-R. L. Gall et al., “A simplified acute physiology score for icu patients,” Critical Care Medicine, vol. 12, no. 11,

1984.

[42] L. G. J, L. S, and S. F, “A new simplified acute physiology score (saps ii) based on a european/north american

multicenter study,” Journal of the American Medical Association, vol. 270, no. 24, pp. 2957–2963, 1993.

[43] P. G. Metnitz et al., “Saps 3 - from evaluation of the patient to evaluation of the intensive care unit. part 1:

Objectives, methods and cohort description,” Intensive Care Medicine, vol. 31, pp. 1336–1344, 2005.

[44] R. P. Moreno et al., “Saps 3–from evaluation of the patient to evaluation of the intensive care unit. part 2:

Development of a prognostic model for hospital mortality at icu admission,” Intensive Care Medicine, vol. 31,

pp. 1345–1355, 2005.

[45] S. Lemeshow et al., “A method for predicting survival and mortality of icu patients using objectively derived

weights,” Critical Care Medicine, vol. 13, no. 7, pp. 519–525, 1985.

[46] L. S, T. D, K. J, A. J, G. SH, and R. J, “Mortality probability models (mpm ii) based on an international cohort

of intensive care unit patients,” Journal of the American Medical Association, vol. 270, no. 20, pp. 2478–2486,

1993.

[47] T. Higgins et al., “Assessing contemporary intensive care unit outcome: An updated mortality probability

admission model (mpm0-iii),” Critical Care Medicine, vol. 35, no. 3, pp. 827–835, 2007.

[48] J. Abenstein and B. Narr, “An ounce of prevention may equate to a pound of cure: Can early detection and

intervention prevent adverse events?,” Anesthesiology, vol. 112, pp. 272–273, 2010.

[49] A. H. Taenzer et al., “Impact of pulse oximetry surveillance on rescue events and intensive care unit transfers,”

Anesthesiology, vol. 112, 2010.

[50] D. Jones, I. Mitchell, K. Hillman, and D. Story,“Defining clinical deterioration,”Resuscitation, vol. 84, pp. 1029–

1034, 2013.

[51] M. M. Churpek, T. C. Yuen, and D. P. Edelson, “Predicting clinical deterioration in the hospital: The impact

of outcome selection,” Resuscitation, vol. 84, no. 5, pp. 564 – 568, 2013.

BIBLIOGRAPHY 169

[52] A. Kho, D. Rotz, K. Alrahi, W. Cardenas, K. Ramsey, D. Liebovitz, G. Noskin, and C. Watts, “Utility of

commonly captured data from an EHR to identify hospitalized patients at risk for clinical deterioration.,”

AMIA Annual Symposium proceedings, no. Table 1, pp. 404–8, 2007.

[53] R. Bellomo et al., “A controlled trial of electronic automated advisory vital signs monitoring in general hospital

wards,” Critical Care Medicine, vol. 40, no. 8, 2012.

[54] L. Tarassenko et al., “Integrated monitoring and analysis for early warning of patient deterioration,” British

Journal of Anaesthesia, vol. 97, no. 1, pp. 64–68, 2006.

[55] M. Churpek et al., “Derivation of a cardiac arrest prediction model using ward vital signs,” Critical Care

Medicine, vol. 40, no. 7, 2012.

[56] B. H. Cuthbertson et al., “Can physiological variables and early warning scoring systems allow early recognition

of the deteriorating surgical patient?,” Critical Care Medicine, vol. 35, no. 2, pp. 402–409, 2007.

[57] M. H. Preece, A. Hill, M. S. Horswill, and M. O. Watson, “Supporting the detection of patient deterioration:

Observation chart design affects the recognition of abnormal vital signs,” Resuscitation, vol. 83, no. 9, pp. 1111

– 1118, 2012.

[58] J. Ludikhuize et al., “How nurses and physicians judge their own quality of care for deteriorating patients on

medical wards: Self-assessment of quality of care is suboptimal,” Critical Care Medicine, vol. 40, no. 11, 2012.

[59] J. H. Silber, S. V. Williams, H. Krakauer, and J. S. Schwartz, “Hospital and patient characteristics associated

with death after surgery: A study of adverse occurrence and failure to rescue,” Medical Care, vol. 30, no. 7,

pp. pp. 615–629, 1992.

[60] A. Schmid et al., “Failure to rescue: A literature review,” The Journal of Nursing Administration, vol. 37, no. 4,

pp. 188–198, 2007.

[61] C. on Quality of Health Care in America, “Crossing the quality chasm,” tech. rep., Institute of Medicine, 2001.

[62] P. S. Q. I. C. M. Workgroup, “Ahrq quality indicators: Patient safety indicators final report,” tech. rep., Agency

for Healthcare Research and Quality, 2008.

[63] P. S. Q. Indicators, “Death rate among surgical inpatients with serious treatable complications,” tech. rep.,

Agency for Healthcare Research and Quality, 2012.

[64] K. McDonald et al., “Measures of patient safety based on hospital administrative data (the agency for healthcare

research and quality patient safety indicators),” tech. rep., University of California San Francisco-Stanford

Evidence-based Practice Center, 2002.

[65] P. S. Romano, J. J. Geppert, S. Davies, M. R. Miller, A. Elixhauser, and K. M. McDonald, “A national profile

of patient safety in u.s. hospitals,” Health Affairs, vol. 22, no. 2, pp. 154–166, 2003.

[66] M. L. Graber, “The incidence of diagnostic error in medicine,” BMJ Quality & Safety, 2013.

[67] P. P. S. Advisory, “Diagnostic error in acute care,” Pennsylvania Patient Safety Advisory, vol. 7, pp. 76–86,

September 2010.

[68] V. Nangalia et al., “Health technology assessment review: Remote monitoring of vital signs - current status and

future challenges,” Critical Care, vol. 14, no. 233, 2010.

[69] Philips, “Philips introduces breakthrough graphical dashboard to advance its eicu telehealth offering,” tech.

rep., Royal Philips Electronics, 2013.

[70] S. Jones et al., “Bedside electronic capture of clinical observations and automated clinical alerts to improve

compliance with an early warning score protocol,” Critical Care Resuscitation, vol. 13, pp. 83–88, 2011.

BIBLIOGRAPHY 170

[71] T. Sinuff et al., “Mortality predictions in the intensive care unit: Comparing physicians with scoring systems,”

Critical Care Medicine, vol. 34, no. 3, 2006.

[72] J. N. Fullerton et al., “Is the modified early warning score (mews) superior to clinician judgement in detecting

critical illness in the pre-hospital environment?,” Resuscitation, vol. 83, no. 5, pp. 557–562, 2012.

[73] A. Lee et al., “The medical emergency team,” Anaesthesia and Intensive Care, vol. 23, pp. 183–186, 1995.

[74] J. B. Sobol, H. B. Gershengorn, H. Wunsch, and G. Li, “The surgical apgar score is strongly associated with

intensive care unit admission after high-risk intraabdominal surgery,” Anesthesia & Analgesia, vol. 117, no. 2,

pp. 438–446, 2013.

[75] A. A. Gawande, M. R. Kwaan, S. E. Regenbogen, S. A. Lipsitz, and M. J. Zinner, “An apgar score for surgery,”

Journal of the American College of Surgeons, vol. 204, no. 2, pp. 201–208, 2007.

[76] K. Palmer, “Acute upper gastrointestinal bleeding management,” tech. rep., National Clinical Guideline Centre,

2007.

[77] K. Hillman et al., “Redefining in-hospital resuscitation: the concept of the medical emergency team,” Resusci-

tation, vol. 48, pp. 105–110, 2001.

[78] K. Hillman et al., “Antecedents to hospital deaths,” Internal Medicine Journal, vol. 31, pp. 343–348, 2001.

[79] L. K. Ott, M. R. Pinsky, L. A. Hoffman, S. P. Clarke, S. Clark, D. Ren, and M. Hravnak, “Medical emergency

team calls in the radiology department: patient characteristics and outcomes,” BMJ Quality and Safety, 2012.

[80] A. Verceles, R. M. Schwarcz, P. Birnbaum, P. Mannam, and H. Patrick, “S.E.P.S.I.S: Sepsis Education Plus

Successful Implementation and Sustainability in the Absence of a Rapid Response Team,” Chest, vol. 128, no. 4,

pp. 181S–b–182, 2005.

[81] A. Joffe et al., “Reduction in hospital mortality over time in a hospital without a pediatric medical emergency

team,” Archives of Pediatrics and Adolescent Medicine, vol. 165, no. 5, 2011.

[82] J. Tirkkonen, J. YlAd’-Mattila, K. T. Olkkola, H. Huhtala, J. Tenhunen, and S. Hoppu, “Factors associated with

delayed activation of medical emergency team and excess mortality: An utstein-style analysis,” Resuscitation,

vol. 84, no. 2, pp. 173 – 178, 2013.

[83] K. Hillman et al., “Introduction of the medical emergency team (met) system: a cluster-randomised controlled

trial,” The Lancet, vol. 365, pp. 2091–2097, 2005.

[84] G. Chrysochoou and S. R. Gunn, “Demonstrating the benefit of medical emergency teams (met) proves more

difficult than anticipated,” Critical Care, vol. 10, p. 306, 2006.

[85] M. a. Cretikos, J. Chen, K. M. Hillman, R. Bellomo, S. R. Finfer, and A. Flabouris, “The effectiveness of

implementation of the medical emergency team (MET) system and factors associated with use during the

MERIT study.,” Critical care and resuscitation : journal of the Australasian Academy of Critical Care Medicine,

vol. 9, no. 2, pp. 206–12, 2007.

[86] R. Bellomo et al., “A prospective before-and-after trial of a medical emergency team,” The Medical Journal of

Australia, vol. 179, pp. 283–287, 2003.

[87] R. Bellomo, D. Goldsmith, S. Uchino, J. Buckmaster, G. Hart, H. Opdam, W. Silvester, L. Doolan, and

G. Gutteridge, “Prospective controlled trial of effect of medical emergency team on postoperative morbidity

and mortality rates,” Critical Care Medicine, vol. 32, no. 4, pp. 916–921, 2004.

[88] C. Karpman et al., “The impact of rapid response team on outcome of patients transferred from the ward to

the icu: A single-center study,” Critical Care Medicine, vol. 0, no. 0, 2013.

BIBLIOGRAPHY 171

[89] P. Chan et al., “Hospital-wide code rates and mortality before and after implementation of a rapid response

team,” Journal of the American Medical Association, vol. 300, no. 21, 2008.

[90] M. D. Buist, G. E. Moore, S. A. Bernard, B. P. Waxman, J. N. Anderson, and T. V. Nguyen, “Effects of a

medical emergency team on reduction of incidence of and mortality from unexpected cardiac arrests in hospital:

Preliminary study,” BMJ: British Medical Journal, vol. 324, no. 7334, pp. pp. 387–390, 2002.

[91] M. D. Howell et al., “Sustained effectiveness of a primary-team-based rapid response system,” Critical Care

Medicine, vol. 40, no. 9, 2012.

[92] D. Jones et al., “Effectiveness of the medical emergency team: the importance of dose,” Critical Care, vol. 13,

no. 5, 2009.

[93] T. W. Reader et al., “Developing a team performance framework for the intensive care unit,” Critical Care

Medicine, vol. 37, no. 5, pp. 1787–1793, 2009.

[94] D. S. Morris, W. Schweickert, D. Holena, R. Handzel, C. Sims, J. L. Pascual, and B. Sarani, “Differences

in outcomes between ICU attending and senior resident physician led medical emergency team responses,”

Resuscitation, vol. 83, no. 12, pp. 1434 – 1437, 2012.

[95] D. A. Jones et al., “The role of the medical emergency team in end-of-life care: A multicenter, prospective,

observational study,” Critical Care Medicine, vol. 40, no. 1, 2012.

[96] J. Downar, R. Barua, D. Rodin, B. Lejnieks, R. Gudimella, V. McCredie, C. Hayes, and A. Steel, “Changes

in end of life care 5 years after the introduction of a rapid response team: A multicentre retrospective study,”

Resuscitation, no. 0, pp. –, 2013.

[97] T. A. C. on Safety and Q. in Health Care, eds., Schedule of the 7th International Conference on Rapid Response

Systems and Medical Emergency Teams, May 2012.

[98] A. Garland et al., “The accuracy of administrative data for identifying the presence and timing of admission to

intensive care units in a canadian province,” Medical Care, vol. 50, no. 3, 2012.

[99] R Core Team, R: A Language and Environment for Statistical Computing. R Foundation for Statistical Com-

puting, Vienna, Austria, 2013. ISBN 3-900051-07-0.

[100] G. M. Fitzmaurice and N. M. Laird, “Regression models for mixed discrete and continuous responses with

potentially missing values,” Biometrics, vol. 53, no. 1, pp. pp. 110–122, 1997.

[101] K. B. Bayley et al., “Challenges in using electronic health record data for cer: Experience of 4 learning organi-

zations and solutions applied,” Medical Care, vol. 00, pp. 000–000, 2013.

[102] C. W. Hug, “Predicting the risk and trajectory of intensive care patients using survival models,” Master’s thesis,

Massachusetts Institute of Technology, 2006.

[103] M. W. Semler, D. G. Stover, A. P. Copland, G. Hong, M. J. Johnson, M. S. Kriss, H. Otepka, L. Wang,

B. W. Christman, and T. W. Rice, “Flash mob research: A single-day, multicenter, resident-directed study of

respiratory rate,” CHEST Journal, vol. 143, no. 6, pp. 1740–1744, 2013.

[104] V. K. Fieler et al., “Eliminating errors in vital signs documentation,” Computers, Informatics, Nursing, vol. 00,

no. 0, pp. 00–00, 2013.

[105] D. B. Dahl, xtable: Export tables to LaTeX or HTML, 2013. R package version 1.7-1.

[106] F. E. H. Jr, with contributions from Charles Dupont, and many others., Hmisc: Harrell Miscellaneous, 2013.

R package version 3.12-2.

[107] H. Wickham, ggplot2: elegant graphics for data analysis. Springer New York, 2009.

BIBLIOGRAPHY 172

[108] D. Sarkar, Lattice: Multivariate Data Visualization with R. New York: Springer, 2008. ISBN 978-0-387-75968-5.

[109] U. Ligges and M. Machler, “Scatterplot3d - an r package for visualizing multivariate data,” Journal of Statistical

Software, vol. 8, no. 11, pp. 1–20, 2003.

[110] T. Lumley, rmeta: Meta-analysis, 2012. R package version 2.16.

[111] A. Liaw and M. Wiener, “Classification and regression by randomforest,” R News, vol. 2, no. 3, pp. 18–22, 2002.

[112] F. E. H. Jr, rms: Regression Modeling Strategies, 2013. R package version 3.6-3.

[113] W. N. Venables and B. D. Ripley, Modern Applied Statistics with S. New York: Springer, fourth ed., 2002.

ISBN 0-387-95457-0.

[114] M. Hofert, I. Kojadinovic, M. Maechler, and J. Yan, copula: Multivariate Dependence with Copulas, 2012. R

package version 0.999-5.

[115] J. Hausser and K. Strimmer, entropy: Entropy and Mutual Information Estimation, 2012. R package version

1.1.7.

[116] P. Giordani and M. B. Ferraro, fclust: Fuzzy clustering, 2013. R package version 1.0.

[117] X. Robin, N. Turck, A. Hainard, N. Tiberti, F. Lisacek, J.-C. Sanchez, and M. MAijller, “proc: an open-source

package for r and s+ to analyze and compare roc curves,” BMC Bioinformatics, vol. 12, p. 77, 2011.

[118] B. Gray, cmprsk: Subdistribution Analysis of Competing Risks, 2012. R package version 2.2-3.

[119] J. Friedman, T. Hastie, and R. Tibshirani, “Regularization paths for generalized linear models via coordinate

descent,” Journal of Statistical Software, vol. 33, no. 1, pp. 1–22, 2010.

[120] S. R. Lele, J. L. Keim, and P. Solymos, ResourceSelection: Resource Selection (Probability) Functions for

Use-Availability Data, 2013. R package version 0.2-3.

[121] D. W. Hosmer and S. Lemeshow, Applied Logistic Regression. New York: John Wiley & Sons, Inc., 2005.

[122] R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal Statistical Society. Series

B (Methodological), vol. 58, no. 1, pp. pp. 267–288, 1996.

[123] I. C. on Machine Learning, ed., An Empirical Comparison of Supervised Learning Algorithms Using Different

Performance Metrics, June 2006.

[124] D. McNair, “Non-invasive assessment of acute deterioration likelihood in in patients,” in Proceedings of the 11th

International Conference on Complexity in Acute Illness, 2012.

[125] C. R. Rao et al., Linear Models and Generalizations: Least Squares and Alternatives. Springer Series in

Statistics, Springer, third extended ed., 2008.

[126] H. Akaike, “A new look at the statistical model identification,” Automatic Control, IEEE Transactions on,

vol. 19, no. 6, pp. 716–723, 1974.

[127] H. Bozdogan, “Model selection and akaike’s information criterion (aic): The general theory and its analytical

extensions,” Psychometrika, vol. 52, no. 3, pp. 345–370, 1987.

[128] R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” pp. 1137–

1143, Morgan Kaufmann, 1995.

[129] L. Breiman, “Random forests,” Machine Learning, vol. 45, pp. 5–32, 2001.

[130] T. Hatie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and

Prediction. Springer Series in Statistics, Springer, second ed., 2009.

BIBLIOGRAPHY 173

[131] L. Breiman, “Manual on setting up, using and understanding random forest v3.1,” tech. rep., University of

California, Berkley, 2002.

[132] J. E. Parrillo, “Pathogenetic mechanism of septic shock,” The New England Journal of Medicine, vol. 328,

no. 20, 1993.

[133] R. B. Nelsen, An Introduction to Copulas. New York: Springer, second ed., 2006.

[134] N. Kolev and D. Paiva, “Copula-based regression models: A survey,” Journal of Statistical Planning and Infer-

ence, vol. 139, no. 11, pp. 3847 – 3856, 2009.

[135] D. Beaudoin and L. Lakhal-Chaieb, “Archimedean copula model selection under dependent truncation,” Statis-

tics in Medicine, vol. 27, no. 22, pp. 4440–4454, 2008.

[136] Ivan Kojadinovic and Jun Yan, “Modeling multivariate distributions with continuous margins using the copula

R package,” Journal of Statistical Software, vol. 34, no. 9, pp. 1–20, 2010.

[137] P. Deheuvels, “An asymptotic decomposition for multivariate distribution-free tests of independence,” Journal

of Multivariate Analysis, vol. 11, no. 1, pp. 102 – 113, 1981.

[138] C. Genest and B. Remillard, “Test of independence and randomness based on the empirical copula process,”

Test, vol. 13, no. 2, pp. 335–369, 2004.

[139] S. Herget-Rosenthal et al., “How to estimate gfr-serum creatinine, serum cystatin c or equations?,” Clinical

Biochemistry, vol. 40, pp. 153–161, 2007.

[140] L. P. Fried, Q.-L. Xue, A. R. Cappola, L. Ferrucci, P. Chaves, R. Varadhan, J. M. Guralnik, S. X. Leng, R. D.

Semba, J. D. Walston, C. S. Blaum, and K. Bandeen-Roche, “Nonlinear multisystem physiological dysregulation

associated with frailty in older women: Implications for etiology and treatment,” The Journals of Gerontology

Series A: Biological Sciences and Medical Sciences, vol. 64A, no. 10, pp. 1049–1057, 2009.

[141] S. Searle, A. Mitnitski, E. Gahbauer, T. Gill, and K. Rockwood, “A standard procedure for creating a frailty

index,” BMC Geriatrics, vol. 8, no. 1, p. 24, 2008.

[142] K. Rockwood and A. Mitnitski, “Frailty in relation to the accumulation of deficits,” The Journals of Gerontology

Series A: Biological Sciences and Medical Sciences, vol. 62, no. 7, pp. 722–727, 2007.

[143] K. Rockwood, A. Mogilner, and A. Mitnitski, “Changes with age in the distribution of a frailty index,” Mecha-

nisms of Ageing and Development, vol. 125, no. 7, pp. 517 – 519, 2004.

[144] K. H. Zou, A. J. OaAZMalley, and L. Mauri, “Receiver-operating characteristic analysis for evaluating diagnostic

tests and predictive models,” Circulation, vol. 115, no. 5, pp. 654–657, 2007.

[145] T. Fawcett, “An introduction to roc analysis,” Pattern Recognition Letters, vol. 27, no. 8, pp. 861–874, 2006.

[146] S. de Vries et al., “Summary receiver operating characteristic curves as a technique for meta-analysis of the

diagnostic performance of duplex ultrasonography in peripheral arterial disease,” Statistics for Radiologists,

vol. 3, no. 4, 1996.

[147] A. Bravi et al., “Review and classification of variability analysis techniques with clinical applications,” BioMed-

ical Engineering OnLine, vol. 10, no. 90, 2011.

[148] S. M. Pincus, “Approximate entropy as a measure of system complexity,” Proceedings of the Natural Academy

of Science, vol. 88, pp. 2297–2301, 1991.

[149] A. Bravi et al., “The problem of parameter estimation within variability analysis: Generalization through

optimization,” Journal of Critical Care, vol. 27, no. 3, 2012.

BIBLIOGRAPHY 174

[150] J. S. Richman and J. R. Moorman, “Physiological time-series analysis using approximate entropy and sample

entropy,” American Journal of Physiology - Heart and Circulatory Physiology, vol. 278, no. 6, pp. H2039–H2049,

2000.

[151] N. Pal and J. Bezdek, “On cluster validity for the fuzzy c-means model,” Fuzzy Systems, IEEE Transactions

on, vol. 3, no. 3, pp. 370–379, 1995.

[152] A. Chao and T.-J. Shen, “Nonparametric estimation of shannonaAZs index of diversity when there are unseen

species in sample,” Environmental and Ecological Statistics, vol. 10, no. 4, pp. 429–443, 2003.

[153] D. W. Hosmer et al., “A comparison of goodness-of-fit tests for the logistic regression model,” Statistics in

Medicine, vol. 16, pp. 965–980, 1997.

[154] M. M. Boniatti et al., “Delayed medical emergency team calls and associated outcomes,” Critical Care Medicine,

vol. 0, no. 0, 2013.

[155] E. Kaplan and P. Meier, “Nonparametric estimation from incomplete observations,” Journal of the American

Statistical Association, vol. 53, no. 282, pp. 457–481, 1958.

[156] D. Kleinbaum and M. Klein, Survival Analysis: A Self-Learning Text. Statistics for Biology and Health,

Springer, second ed., 2005.

[157] D. R. Cox, “Regression Models and Life-Tables,” Journal of the Royal Statistical Society. Series B (Methodolog-

ical), vol. 34, no. 2, pp. pp. 187–220, 1972.

[158] R. J. Gray, “A class of k-sample tests for comparing the cumulative incidence of a competing risk,” The Annals

of Statistics, vol. 16, no. 3, pp. pp. 1141–1154, 1988.

[159] J. P. Fine and R. J. Gray, “A proportional hazards model for the subdistribution of a competing risk,” Journal

of the American Statistical Association, vol. 94, no. 446, pp. 496–509, 1999.

[160] V. Coviello and M. Boggess, “Cumulative incidence estimation in the presence of competing risks,” The Stata

Journal, vol. 4, no. 2, pp. 103–112, 2004.

[161] F. Harrell et al., “Multivariable prognostic models: Issues in developing models, evaluating assumptions and

adequacy, and measuring and reducing errors,” Statistics in Medicine, vol. 15, pp. 361–387, 1996.

[162] R. H. Somers, “A new asymmetric measure of association for ordinal variables,” American Sociological Review,

vol. 27, no. 6, pp. pp. 799–811, 1962.

[163] J. Soar and C. P. Subbe, “Identifying the patient at risk of deterioration, intensive care unit admission, or

cardiac arrest: Stop predicting, start preventing,” Critical Care Medicine, vol. 40, no. 7, 2012.

an analysis of rapid response team calling algorithms for ......an analysis of rapid response team...

Documents