an introduction to bayesisan decision analysis

Enterprise EHS Software Solutions 1

Enterprise EHS Software Solutions

Monica Melkonian, MS, CIHProduct Manager- Industrial

Hygiene

- THE BASICS OF BDA METHODOLOGY- BDA DECISION CHARTS- BDA AND THE AIHA EXPOSURE MODEL- QUANTIFYING PROFESSIONAL JUDGMENT

Paul Hewett, owner, Exposure Assessment Solutions Inc


ADVANTAGES OF BDA• Can set a plausible parameter space

• Output is a set of Decision Charts

• Can incorporate Professional Judgment

• Best applied to small datasets

• Provides feedback

• Consistent with …AIHA Exposure Banding Model

EU Control Banding Model

pharmaceutical Control Banding or PB-OEL Models

• Can be applied to datasets that contain non-detects

3


AIHA EXPOSURE ASSESSMENTStart

Basic Characterization

Exposure Assessment

Unacceptable Exposure

UncertainAcceptable Exposure

Reassessment

ControlFurther Information

Gathering

The exposure assessment and management program of most companies is based on the AIHA approach.

Category 0, 1, 2 or 3

Category 4 or 5


AIHA EXPOSURE CONTROL BANDING MODEL

Exposure Control Category Cutoff (%OEL) Confidence level

0 X0.95 < 1%

High

Medium

Low

1 1% < X0.95 < 10%

2 10% < X0.95 < 50%

3 50% < X0.95 < 100%

4 X0.95 > 100%

5


TYPICAL ACTIONS

Exposure Control Category Recommended Actions

0 (<1% of OEL) No action

1 (<10% of OEL) general HazCom

2 (10-50% of OEL) + chemical specific HazCom

3 (50-100% of OEL) + exposure surveillance, medical surveillance, work practice analysis

4 (>100% of OEL) + respirators & engineering controls, work practice controls, validate respirator selection

Multiples of OEL (e.g., based on respirator APFs)

+ immediate engineering controls or process shutdown, validate respirator selection

6


• The goal of an exposure assessment is to select the exposure category that most likely contains the true 95th percentile.

• The resulting “actions” will then be risk-based …

that is, proportional to the expected degree of risk,

as indicated by the exposure category.

• How do we pick the correct exposure category and certainty level?

7

EXPOSURE ASSESSMENT GOAL


THE “DATA DRIVEN” APPROACH

• Collect a large n dataset.

• Calculate the “sample” 95th percentile (X0.95)

plus the 95%LCL and 95%UCL.

• Compare sample X0.95 to the category cutoffs

Which category does it fall in?

• Determine a certainty level.

Compare LCL and UCL to category cutoffs.

• With large n little to no judgment is necessary. Nearly all IHs will reach the same decision.

8


EXAMPLE

• An IH collected 17 representative measurements and calculated standard statistics (OEL=1):

gm = 0.12 and gsd = 2.0

x0.95 = 0.38

38% of the OEL; suggests a Category 2 exposure profile

95%LCL = 0.26 and 95%UCL = 0.67

Category 3 is also a possibility. Categories 1 and 4 can be eliminated from consideration.

Final Rating : Category 2, Medium Certainty

9


How do we pick a category and certainty level… when we have only a few measurements?

• n=1; x = {0.05} (OEL=1)

• n=2; x = {0.05, 0.2}

X0.95 = 0.50 ( 95%LCL=0.16, 95%UCL=1.5x1010 )

• n=3; x = {0.05, 0.2, 0.1}

X0.95 = 0.31 ( 95%LCL=0.16, 95%UCL=20 )

• Which category: 0, 1, 2, 3, or 4 ?

10

PICKING A CATEGORY


PICKING A CATEGORY

• x = {<0.1, 0.2, <0.1} (OEL=1)

X0.95 = ?

• x = {<0.05, 0.2}

X0.95 = ?

• x = {<0.05} or {<0.25} or {<0.75}

X0.95 = ?

• Which category: 0, 1, 2, 3, or 4 ?11

…when some or most of the measurements are non-detects?


BAYESIAN DECISION ANALYSIS (BDA)

• An adjunct to the calculation and interpretation of traditional statistics.

• The goal of BDA is to estimate the probability that the true 95th percentile exposure falls within a particular category, or Exposure Rating.

• BDA can explicitly incorporate professional judgment.

12


ADVANTAGES

• Best suited for small datasets

• Can handle non-detects

• Professional judgment can be incorporated

• Encourages the improvement of professional judgment

• The output is a set of easy to interpret “decision charts”

13x = {0.05, 0.2, 0.1}


BDA DECISION CHARTS

• Prior decision chart Represents our professional judgment

regarding the probability of each of the five Exposure Ratings.

• Likelihood decision chart The set of probabilities of each exposure

rating calculated using only the collected data.

• Posterior decision chart The set of probabilities of each exposure

rating, updated using the information from the prior decision chart.

x = {0.05, 0.2, 0.1}


• In order to apply Bayesian analysis to industrial hygiene “decision making” we need the following:

A model for classifying occupational exposure profiles into exposure categories.e.g., the AIHA Exposure Control Banding Model

A distributional modele.g., the lognormal model

A decision statistice.g., the 95th percentile

15

BDA DECISION MAKING


When n is small, confidence intervals are often extremely broad.

• X = {0.20, 0.05, 0.10} (OEL=1)

• n = 3

• gm = 0.1 95%LCL = 0.03, 95%UCL = 0.32

• gsd = 2.0 95%LCL = 1.5, 95%UCL = 21

• x0.95 = 0.31 95%LCL = 0.16, 95%UCL = 20

16

CONFIDENCE INTERVALS


Exposure Rating Cutoff (%OEL)

0 X0.95 < 1%

1 1%< X0.95 <10%

2 10%< X0.95 <50%

3 50%< X0.95 <100%

4 X0.95 > 100%

An “Exposure Rating” represents a population of exposure profiles.

17

EXPOSURE RATINGS


GM

0.0001 0.001 0.01 0.1 1

GS

D

20

10

PARAMETER SPACE (OEL =1)

0 1 2 3 4

18


GM

0.001 0.01 0.1 1

GS

D

5

4

3

2

1

0 1 2 3 4

19

KEY CONCEPT: Parameter Space with a plausible upper limit for the true GSD (OEL=1)

KEY CONCEPTS


GM

0.001 0.01 0.1 1

GS

D

5

4

3

2

1

0 1 2 3 4

20

KEY CONCEPT: Decision Charts


Bayes’ Theorem –The foundation of Bayesian statistics

Posterior Likelihood Prior

Correction Factor21

BAYES’ THEOREM


Applied to all lognormal distributions within parameter space

22

BAYES’ THEOREM

KEY CONCEPT: Populationi = AIHA exposure category(i.e., all combinations of geometric mean (G) and geometric standard deviation (D) within the ith exposure category)


DECISION CHARTS

• Example Prior Decision Distributions

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.05

0.2

0.5

0.2

0.05

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2 0.2 0.2 0.2 0.2

Non-informative priorInformative prior:Category 2, Medium Certainty

23


GENERIC “PRIOR DECISION CHARTS”

• Professional Judgment prior

When the user picks an Initial Rating and Certainty Level a recommended Prior Decision Chart is loaded.

The default category probabilities represent a “best guess” as to what a generic prior should look like.

24


Prior

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0.6

0.2 0.150.04 0.01

Prior

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0.5

0.250.18

0.05 0.02

Prior

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0.40.3

0.21

0.06 0.03

Non-informative prior “decisionchart”

IR=Initial RatingCL=Certainty Level

IR=Category 0CL=low

IR=Category 0CL=medium

IR=Category 0CL=high

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2 0.2 0.2 0.2 0.2

25



IR=Category 1CL=low



Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2 0.2 0.2 0.2 0.2

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.30.4

0.180.07 0.05

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.25

0.5

0.160.06 0.03

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2

0.6

0.140.04 0.02




IR=Category 2CL=low



Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2 0.2 0.2 0.2 0.2


Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.07

0.23

0.4

0.23

0.07

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.05

0.2

0.5

0.2

0.05

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.03

0.17

0.6

0.17

0.03



IR=Category 3CL=low



Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2 0.2 0.2 0.2 0.2


Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.05 0.070.18

0.40.3

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.03 0.060.16

0.5

0.25

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.02 0.040.14

0.6

0.2



IR=Category 4CL=low



Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2 0.2 0.2 0.2 0.2

IR=Initial RatingCL=Certainty Level Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.03 0.06

0.210.3

0.4

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.02 0.050.18

0.25

0.5

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.01 0.040.15

0.2

0.6


X={ 0.20, 0.05, 0.10 }

Likelihood

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0 0.002

0.66

0.2290.109

30

EXAMPLE LIKELIHOOD DECISION CHART


EXAMPLE POSTERIOR DECISION CHARTS

Posterior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0 0.001

0.865

0.120.014

Posterior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0 0.002

0.66

0.2290.109

Using the non-informative prior

Using the informative prior

31


PRIOR

LIKELIHOOD

POSTERIOR

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.2 0.2 0.2 0.2 0.2

Prior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0.05

0.2

0.5

0.2

0.05

Likelihood

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0 0.002

0.66

0.2290.109

Posterior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0 0.001

0.865

0.120.014

Posterior

Exposure Rating

0 1 2 3 4

Decis

ion P

robability 1

0.8

0.6

0.4

0.2

0

0 0.002

0.66

0.2290.109

NON-INFORMATIVE INFORMATIVE


EXAMPLE APPLICATIONS OF BDA

• General analysis of occupational exposure data

• Reach a decision when n is small

• Leverage professional judgment

• Provide feedback

• Assist in respirator selection

• Analyze censored datasets

33


GENERAL ANALYSIS OF OCCUPATIONAL EXPOSURE DATA

• OEL=0.2 mg/m3

• n = 4

• x = {0.015, 0.008, 0.006, 0.016}

mg/m3

• In principle, BDA can be applied to any sample size (but is limited here to n<250).

34


REACH A DECISION WHEN N IS SMALL

• OEL=1 ppm

• n = 1

• x = 0.05 ppm

• BDA can be applied to sample sizes as low as n=1.

35


REACH A DECISION WHEN N IS SMALL

• OEL=1 ppm

• n = 1

• x = 0.99 ppm

• “Yes, the measurement is <OEL. But I strongly suspect that the exposure profile is not acceptable.”

• BDA leads to the same conclusion.

36


LEVERAGE PROFESSIONAL JUDGEMENT

• OEL=1 ppm

• n = 1

• x = 0.05 ppm

• Professional judgment can sharpen the decision.

37


PROVIDE FEEDBACK

• OEL=1 ppm

• n = 3

• x1 = 0.25 ppm

• x2 = 0.50 ppm

• x3 = 1.00 ppm

• The Prior is inconsistent with the Likelihood.

• BDA can be used to help improve professional judgment.

38


ASSIST IN RESPIRATOR SELECTION

• OEL=1 ppm

• n = 3

• x1 = 0.99 ppm

• x2 = 0.50 ppm

• x3 = 2.0 ppm

• Decision = Category 4

• BDA can be used to guide PPE selection.

39


ANALYZE CENSORED DATASETS

• OEL=1 ppm

• n = 1

• x < 0.05

• LOD = 0.05

• BDA can be applied to censored datasets, even datasets that are 100% censored and have different detection limits.

40


Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0 0.003

0.393

0.253

0.352

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0.05

0.804

0.13

0.012

0.005

ONE SAMPLE EXAMPLES (OEL=1)

• X = {0.005} ppm

• X = {0.01} ppm

• X = {0.05} ppm

• X = {0.25} ppm

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0.394

0.542

0.059

0.004

0.001

41

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0.000

0.394

0.459

0.082

0.064


OEL = 1 PPM

• X = {0.49} ppm

• X = {0.75} ppm

• X = {0.99} ppm

• X = {1.5} ppm

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0 0.000

0.056

0.351

0.593

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0 0.000

0.013

0.189

0.798

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y1

0.8

0.6

0.4

0.2

0

0 0 0.002

0.012

0.986

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0 0 0.006

0.048

0.946

42


LARGE SAMPLE EXAMPLE

• An IH collected 17 representative measurements and calculated standard statistics (OEL=1):

X0.95 = 0.38 95%LCL = 0.26 and 95%UCL = 0.67

Final Rating : Category 2, Medium Certainty

• When n is large the BDA

results and standard

statistical analysis generally

lead to the same decision.

Likelihood

Exposure Rating

0 1 2 3 4

De

cis

ion

Pro

ba

bilit

y

1

0.8

0.6

0.4

0.2

0

0 0

0.79

0.201

0.009

43


ASSUMPTIONS

• The lognormal distribution is a reasonable approximation of the true exposure profile.

• The true exposure profile falls somewhere within Parameter Space.

If the true GSD approaches or exceeds 4 the decision probabilities for the upper categories may be underestimates (and can potentially mislead).

44


CAUTIONS

• BDA does not give us a “data driven” decision from a small n dataset.

• BDA simply gives us a different way to look at the data, in terms of …

exposure categories and

decision probabilities that the true 95th percentile falls within a category.

45


CAUTIONS

• Training is necessary to properly use the BDA method and interpret the decision charts.

• Management should agree on …

the company risk management goals

the decision statistic

the use of the AIHA exposure rating method

how BDA fits into the overall program.

46


• BDA is new “decision making” tool for IHs.

• Use it in conjunction with your other tools: graphs and goodness-of-fit tests

descriptive statistics

compliance statistics.

• Do the BDA results suggest a different interpretation of your dataset?

• Which interpretation is most likely correct? The BDA interpretation or that reached using your current data analysis tools?

SUMMARY

47


WATCH THE WEBINAR

48

If you want to listen to the presentation that goes alongside this deck, you can do so for free here:http://www2.medgate.com/bayesian-webinar

http://www2.medgate.com/bayesian-webinar

an introduction to bayesisan decision analysis

Business