1

CEE 763 Fall 2011

Topic 4 – Before-After Studies

CEE 763

2

CEE 763 Fall 20112

BEFORE-AFTER STUDIES

Experiment Controlled environment e.g.: Physics, animal science

Observational Study Cross-Section (e.g., stop vs. yield) Before-After*

Ezra Hauer, “Observational Before-After Studies in Road Safety”, ISBN 0-08-043053-8

3

CEE 763 Fall 20113

WHAT IS THE QUESTION

Treatment – a measure implemented at a site for the purpose of achieving safety improvement.

The effectiveness of a treatment is the change in safety performance measures purely due to the treatment.

It is measured by the difference between “what would have been the safety of the site in the ‘after’ period had treatment not been applied” and “what the safety of the site in the ‘after’ period was”.

4

CEE 763 Fall 20114

AN EXAMPLE

R.I.D.E. (Reduce Impaired Driving Everywhere) Program

5

CEE 763 Fall 20115

FREQUENCY OR RATE?

AADT

Without Rumble Strip

With Rumble StripAB

C

What conclusions would you make by using rate or frequency?

6

CEE 763 Fall 20116

TARGET ACCIDENTS

Target accidents – Those accidents the occurrence of which can be materially affected by the treatment.

Case 1 – R.I.D.E: An enforcement program in Toronto to reduce alcohol-related injury accidents Target accidents: alcohol-impaired accidents or total accidents?

7

CEE 763 Fall 20117

TARGET ACCIDENTS (continued)

Case 2 – Sound-wall effect The study was to look at whether the construction of sound-

walls increased crashes or not. Target accidents: run-off-the-road accidents or total accidents?

8

CEE 763 Fall 20118

TARGET ACCIDENTS (continued)

Case 3 – Right-turn-on-red policy The study was to look at whether allowing vehicles to make right turns on

red increased crashes or not. Target accidents: accidents that involve at least one right-turn vehicle or

total accidents?

9

CEE 763 Fall 20119

RIGHT-TURN-ON-RED CASE

Case 3 – Right-turn-on-red policy

Target Comparison*

Before 167 3566

After 313 6121

*Comparison accidents are those that do not involve any right-turn vehicles

Right-turn Other* Total

Before 2192 28656 30848

After 2808 26344 29152

*Other accidents are those that do not involve any right-turn vehicles

10

CEE 763 Fall 201110

PREDICTION AND ESTIMATION

Prediction – to estimate what would have been the safety of the entity in the ‘after’ period had treatment not been applied.

Many ways to predict.

Estimation – to estimate what the safety of the treated entry in the ‘after’ period was.

11

CEE 763 Fall 201111

PREDICTION

One-year before (173)

Three-year before average (184)

Regression (165)

Comparison group (160)

12

CEE 763 Fall 201112

FOUR-STEP PROCESS FOR A B-A STUDY

Step 1 – Estimate λ and predict π λ is the expected number of target accidents in the ‘after’ period π is what the expected number of target accidents in an ‘after’ period would have been had it not been treated

Step 2 – Estimate VAR{λ} and VAR{π}

Step 3 – Estimate δ and θ δ is reduction in the expected number of accidents; θ is safety index of effectiveness

Step 4 – Estimate VAR{δ} and VAR{θ}

]/}{1/[ 2 VAR

13

CEE 763 Fall 201113

EQUATIONS

]/}{1/[ 2 VAR

}{}{}{ VARVARVAR

22222 ]/}{1/[]/}{/}{[}{ VARVARVARVAR

14

CEE 763 Fall 201114

EXAMPLENAÏVE BEFORE-AFTER STUDY

Consider a Naïve B-A study with 173 accidents in the ‘before’ year and 144 accidents in the ‘after’ year. Determine the effectiveness of the treatment.

15

CEE 763 Fall 201115

COMPARISON GROUP (C-G) B-A STUDY

Comparison group – a group of sites that did not receive the treatment

Assumptions Factors affecting safety have changed from “before” to “after” in the same manner for the treatment group and the

comparison group These factors influence both groups in the same way

Whatever happened to the subject group (except for the treatment itself) happened exactly the same way to the comparison group

16

CEE 763 Fall 201116

EXAMPLE

Where R.I.D.E. was implemented, alcohol-related crash was changed from 173 (before) to 144 (after). Where R.I.D.E. was NOT implemented, alcohol-related crash was changed from 225 (before) to 195 (after). What would be the crash in the after period had R.I.D.E. not been implemented?

17

CEE 763 Fall 201117

C-G METHOD

Treatment Group

Comparison Group

Before K M

After L N

M

NrC

KrT

TC rr KrC

T

C

r

r Odds ratio

18

CEE 763 Fall 201118

EQUATIONS

L

M

Nrr CT KrC

}]{VARNMK

[}{VAR 1112

L}{VAR

]/}{1/[ 2 VAR

}{}{}{ VARVARVAR

22222 ]/}{1/[]/}{/}{[}{ VARVARVARVAR

19

CEE 763 Fall 201119

EXAMPLE

Treatment Group

Comparison Group

Before K=173 M=897

After L=144 N=870

00550.}{VAR

The table shows the accident counts for the R.I.D.E. program at both treatment sites and comparison sites.

20

CEE 763 Fall 201120

THE EB METHOD

K)()k(E}K/k{E 1

}k{E}k{VAR

Y

1

1

}/{)1(.. KkEds

K}K{E}k{E If not giving, use the actual counts K (‘before’ period) to estimate population mean, E{k}

Ks}k{E}K{VAR}k{VAR 2

Variance if ‘before’ has multiple years

Y is the ratio between ‘before’ period and ‘after’ period

]Ks[Y}k{VAR 22

s2 is sample variance for the ‘before’ period

EB estimate of the expected number of ‘after’ accidents had the treatment not been implemented.

21

CEE 763 Fall 201121

EXAMPLE

Accidents recorded at 5 intersections over a two-year period are shown in the table. What is the weighting factor, α for the EB method?

Site Accident

1 0

2 3

3 2

4 0

5 1

22

CEE 763 Fall 201122

EQUATIONS

L

]K/k{E

L}{VAR

]/}{1/[ 2 VAR

}{}{}{ VARVARVAR

22222 ]/}{1/[]/}{/}{[}{ VARVARVARVAR

}K/k{E)(}{VAR 1

23

CEE 763 Fall 201123

EXAMPLE

Using the EB method to conduct the B-A study based on the information in the table.

1Site

2Before

3After

4K

5L

6K(acc/er yr)

7L (acc/ yr)

8E{k} - reference sites

Acc/yr

9S2

[acc/yr]2

10VAR{k}[acc/yr]2

11α

12E{k/K}

1 71-73 75-77 14 6 4.67 2.00 0.092 0.151 0.06 0.34 3.10

2 73-75 77-79 16 3 0.091 0.146

3 71-73 75-77 18 6

4 71-73 75-77 28 7

5 71-73 75-77 15 3

6 72-74 76-78 28 1 0.091 0.153

7 75-76 78-79 4 0 2.00 0.00 0.093 0.145 0.05 0.47 1.10

8 71-73 75-77 11 3

9 75-76 78-79 6 2

10 72-74 76-78 6 2