improving biosurveillance protecting people as critical infrastructure
TRANSCRIPT
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
1/21
Improving Biosurveillance:Protecting People as Critical Infrastructure
Ronald D. Fricker, Jr. August 14, 2008
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
2/21
What is Biosurveillance?
Homeland Security Presidential DirectiveHSPD-21 (October 18, 2007): The term biosurveillance means the process of active data-
gathering of biosphere data in order to achieve earlywarning of health threats, early detection of health events, andoverall situational awareness of disease activity. [1]
The Secretary of Health and Human Services shall establishan operational national epidemiologic surveillance system forhuman health... [1]
Epidemiologic surveillance: surveillance using health-related data that precedediagnosis and signal a sufficient probability of a case or an
outbreak to warrant further public health response .
[2]
2
[1] www.whitehouse.gov/news/releases/2007/10/20071018-10.html[2] CDC ( www.cdc.gov/epo/dphsi/syndromic.htm , accessed 5/29/07)
http://www.cdc.gov/epo/dphsi/syndromic.htmhttp://www.cdc.gov/epo/dphsi/syndromic.htm -
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
3/21
An Existing System: BioSense
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
4/21
The Problem in Summary
Goal: Early detection ofdisease outbreak and/orbioterrorism
Issue: Currently detectionthresholds set naively Equally for all sensors Ignores differential
probability of attack
Result: High false alarm rates Loss of credibility
5
most health monitors
learned to ignore alarmstriggered by their system. Thisis due to the excessive falsealarm rate that is typical of
most systems - there is nearlyan alarm every day! [1]
[1] https://wiki.cirg.washington.edu/pub/bin/view/Isds/SurveillanceSystemsInPractice
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
5/21
6
Formal Description of the System
Each hospital sends data to CDC daily Let X it denote data from hospital i on day t If no attack anywhere X it ~ F 0 for all i and t If attack occurs on day t , X it ~ F
1, t =t, t +1,...
Assume only one location attacked Threshold detection: Signal on day t * if
for one or more hospitals Each hospital location has an estimated
probability of attack:
*it i X h
1,..., , 1n ii p p p
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
6/21
Idea of Threshold Detection
7
Distribution ofBackground Incidenceand Attack/Outbreak ( f 1 )
h
Distribution of BackgroundDisease Incidence ( f 0 )
0 0( ) 1 ( ) x h f x dx F h -
Probability of a false signal:
Probability of a true signal:
1 1( ) 1 ( ) x h
f x dx F h -
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
7/21
Its All About Choosing Thresholds
8
No Attack/Outbreak
Distribution
Attack/Outbreak
Distribution
Threshold ( h )
ROC Curve
Pr(signal | no attack)
P r ( s
i g n
a l | a
t t a c
k )
For each hospital, choice of h iscompromise between probabilityof true and false signals
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
8/21
9
Its simple to write out:
Express it as an optimization problem:
Mathematical Formulationof the Problem
1
0
max 1 ( ) s.t. 1 ( )
i ih i
ii
F h p F h
-
-
Pr(detection) Pr(signal|attack) Pr(attack)i
E(# false signals) Pr(signal|no attack)i
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
9/21
Some Assumptions
Hospitals are spatially independent Monitoring standardized residuals from model
Model accounts for (and removes) systematiceffects in the data
Result: Reasonable to assume F 0=N(0,1) An attack will result in a 2-sigma increase in
the mean of the residuals Result: F 1=N(2,1)
Then, problem is: min ( 2)
s.t. ( )
i ih i
ii
h p
h n
-
-
10
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
10/21
Ten Hospital Illustration
11
H ospital i
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
11/21
Simplifying to a One-dimensionalOptimization Problem
System of n hospitals means optimizationhas n free parameters Hard for to solve for large systems
Can simplify to one-parameter problem: Theorem : For F 0=N(0,1) and F 1=N(g,1), the
optimization simplifies to finding m to satisfy
and the optimal thresholds are then
12
1
1ln( ) ,
n
ii
p nm g
- -
1ln( ).i ih pm
g -
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
12/21
13
Consider (Hypothetical) System toMonitor 200 Largest Cities in US
Assume probability of attack is proportionalto the population in a city
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
13/21
Assume 2 magnitude event Constraint of 1 false signal system-wide / day
Result: Pr(signal | attack) = 0.388
Nave result: Pr(signal | attack) = 0.283
Optimal Solution for 200 Cities
Population Pr(attack) Threshold
14
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
14/21
P d False Alarm Trade-Off
15
( )1
0.388
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
15/21
Choosing and
Optimal probability of detection forvarious choices of g and
Choice of depends on available resources Setting g is subjective: what size mean
increase important to detect?
16
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
16/21
Sensitivity Analyses
Optimal probability of detection
Actual probability of detection
17
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
17/21
18
Optimizing a County-level System
Th h ld F i f
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
18/21
Thresholds as a Function ofProbability of Attack
19
Counties with low probabilityof attack high thresholds Unlikely to detect attack Few false signals
Counties with high probabilityof attack lower thresholds Better chance to detect attack Higher number of false signals
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
19/21
Take-Aways
BioSense and other biosurveillance systemsperformance can be improved now at no cost
Approach allows for customization E.g., increase in probability of detection at
individual location or add additional constraint tominimize false signals
Applies to other sensor system applications: Port surveillance, radiation/chem detection
systems, etc.
Details in Fricker and Banschbach (2008)
20
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
20/21
Future Research Directions
Assess data fusion techniques for usewhen multiple sensors in each region I.e., relax sensor (spatial) independence
assumption Generalize from threshold detectionmethods to other methods that usehistorical information I.e., relax temporal independence
assumption
21
-
8/10/2019 Improving Biosurveillance Protecting People as Critical Infrastructure
21/21
Selected References
Biosurveillance System Optimization:
Fricker, R.D., Jr., and D. Banschbach, Optimizing a System of Threshold Detection Sensors,in submission.
Background Information:
Fricker, R.D., Jr., and H. Rolka, Protecting Against Biological Terrorism: Statistical Issues inElectronic Biosurveillance, Chance , 91 , pp. 4-13, 2006.
Fricker, R.D., Jr., Syndromic Surveillance, in Encyclopedia of Quantitative Risk Assessment ,Melnick, E., and Everitt, B (eds.), John Wiley & Sons Ltd, pp. 1743-1752, 2008.
Detection Algorithm Development and Assessment: Fricker, R.D., Jr., and J.T. Chang, A Spatio-temporal Method for Real-time Biosurveillance,
Quality Engineering , (to appear, November 2008).
Fricker, R.D., Jr., Knitt, M.C., and C.X. Hu, Comparing Directionally Sensitive MCUSUM andMEWMA Procedures with Application to Biosurveillance, Quality Engineering (to appear,November 2008).
Joner, M.D., Jr., Woodall, W.H., Reynolds, M.R., Jr., and R.D. Fricker, Jr., A One-SidedMEWMA Chart for Health Surveillance, Quality and Reliability Engineering International,24 , pp. 503-519, 2008.
Fricker, R.D., Jr., Hegler, B.L., and D.A Dunfee, Assessing the Performance of the Early Aberration Reporting System (EARS) Syndromic Surveillance Algorithms, Statistics inMedicine, 27 , pp. 3407-3429, 2008.
Fricker, R.D., Jr., Directionally Sensitive Multivariate Statistical Process Control Methods with Application to Syndromic Surveillance, Advances in Disease Surveillance, 3:1, 2007.
22