Validation of Highway Engineering Data Quality on Wisconsin Crash

Reports

Andrea Bill

Outline

• Introduction

• Project Goal

• Good Data Quality Characteristics

• Common Issues Leading to Poor Data Quality

• Research Methodology

• Results

• Conclusions/Recommendations

Introduction

• Anecdotal evidence & literature point to crash report interpretive errors being common place

• Quality data necessary for analysts to draw any meaningful conclusions

• Police officers have a limited background in highway engineering

– Training

– Instruction manuals

– Common sense

ehow.com

Project Goal

• Compare officer responses on MV4000 crash reports against researcher responses at the same location

• Determine where differences in data interpretation may require additional training or clarification

• Limited to highway engineering data fields

Good Data Quality Characteristics

• FHWA’s Crash Data Improvement Program Guide (CDIPG) published in April 2010

• According to CDIPG, crash databases should have the following “Six Pack” – Timeliness – Accuracy – Completeness – Consistency/Uniformity – Integration with other databases – Accessibility of data

• This project focused on accuracy, completeness, and consistency of engineering data fields of the Wisconsin crash database

safety.fhwa.dot.gov

Good Data Quality Characteristics

• Accuracy

– Internal validation (electronic forms)

– External validation (this research)

• Completeness—no “blank” entries

• Consistency—uniformity among files

– All WI agencies use MV4000

– National standard is Model Minimum Uniform Crash Criteria (MMUCC)

MMUCC

• Joint NHTSA & Governors’ Highway Safety Association publication

• Voluntary guideline, but compliance

necessary for States to receive Traffic Safety Information System improvement grants (Section 408)

• 2010 NHTSA Traffic Records

Assessment found WI MV4000 compliant in 90.6 percent of data fields and 53.1 percent in attributes – Field: Traffic Control – Attribute: Signal, Stop, Yield, etc.

Common Issues Leading to Poor Data Quality

• People & Training

• Processes

• Technology

People & Training

• According to the CDIPG, one refrain commonly heard from police is that “crash forms are being completed just for insurance companies”

• WisDOT MV4000 Instruction Manual

– Primary training resource for WI officers

– Last updated in 1998 – Brief and vague concerning

engineering fields • No baseline definition of when to flag

hills or curves • Poor definition of traffic barrier • No discussion of roundabouts

Processes

• Paper crash forms still used by approximately 45 percent of WI agencies in 2010

• Paper forms require manual data entry – Time consuming – Constrain resources – Lead to backlogs affecting

timeliness of data

• Personnel under pressure to process backlogs are prone to errors

Paper crash report backlogs in Texas. (GAO-10-454)

Technology

• CDPIG strongly recommends States adopt new and innovative technologies to improve crash data quality

• Two most commonly advocated technologies

– Electronic crash forms

– GPS based smart maps for location ID

Electronic Crash Forms

• Allow validity checks to catch inadvertent errors and blank fields

• Clean, clear crash diagramming

• Increases speed of submission, and therefore availability of data for analysis

• WI State Record planners predict 100 percent electronic reporting statewide by 2015

reportbeam.com

GPS/GIS Location Identification • MMUCC recommends that highway engineering data studied in this

research be generated by linking to roadway inventory data • Several States, including Maine and Ohio, already have such systems in

place • Badger TraCS (most common electronic report system in WI) has

interfaces for point-and-click location ID, but not yet used

Maine DOT Map Viewer System (GAO-10-454)

Other Crash Data Quality Studies

• Several studies have investigated commercial vehicle crash data quality, concluding that officer training needed improvement

• Mickee (2008) audited ~1000 Massachusetts crash reports – Used police, highway department & university auditors

– Reports were not verified by site visits

– Concluded that auditing created detailed statistical results, but was prohibitively time consuming

• Importantly, no other known studies incorporated investigation of verifiable conditions in the field

Research Methodology

• MV4000 data fields – Highway classification (derived based on municipality

population) – Access control – Traffic-way – Horizontal curvature – Vertical curvature – Traffic control

• Easily observable permanent highway features • Independent of other crash variables (e.g. date,

time, weather, driver, etc.)

Research Methodology

• Analysis timeframe: 1/1/09 to 12/31/10

• Geographic boundaries – Dane & Rock Counties – Representative of

municipality types in WI

• 27 intersection locations – Minimum of 5 crashes

2009-2010 – Apparent discrepancies

in three or more fields

• 664 total crash reports

Madison

Janesville

Beloit Bing.com

Site visit locations

Site Visits

• Observed ~250 foot radius around intersection

• Marked appropriate field attribute for each intersection approach

• In accordance with WisDOT Training Manual when possible—used engineering judgment when manual did not offer clear guidance – Curve: Alignment change of 15

degrees or more – Hill: Noticeable change in

elevation – Thinking from perspective of an

officer—they would not be measuring curve radii or grades

USH 51 at W Delavan Drive (Janesville)

Data Coding

• Binary system – 1 correct officer response

– 0 incorrect officer response

• Each crash report analyzed, using narratives and diagrams to determine correct approach where crash occurred

Field Report Coding

DOCTNMBR 1234567 N/A

HWYCLASS U CITY 1

ACSCNTL PART 0

ROADHOR C 1

ROADVERT 1

TRFCWAY ND 1

TRFCNTL1 TS OP 1

TRFCNTL2 NONE 0

Example of data coding for a report

Overall Results

• Highway classification – In general corresponded well – Exceptions often seen on

municipal boundaries (e.g. Fish Hatch at Greenway Cross)

• Traffic control – In general fairly accurate – Most common error was

reporting no control when in fact vehicles were subject to signals because of proximity to the intersection

• Access control, horizontal & vertical curvature, and traffic-way were more problematic

Correctly

Reported Percent of Total

TOTAL 664 100.0%

HWYCLASS 649 97.6%

ACSCNTL 480 72.3%

ROADHOR 602 90.8%

ROADVERT 613 92.9%

TRFCWAY 479 72.3%

TRFCNTL1 573 88.7%

TRFCNTL2 585 90.1%

Global Reporting Accuracy

Results: Access Control

• Overall accuracy only 72.3 percent • Researchers noted a wider variability in officer responses at locations

with at least one partially controlled facility (no private driveways, but at-grade intersections—e.g. USH 51 through Madison)

• Results confirm a sharp decrease in accuracy at partially controlled locations

Locations with Partial Access Control

Location

Reporting Accuracy (number

correct) Percentage

USH-14 & Pontiac 8 of 34 23.5%

STH-113 & CTH M 0 of 8 0%

US-51 & CTH CV 2 of 16 12.5%

W Main St. & O'Keeffe 2 of 13 15.4%

USH-51 & Pflaum 45 of 90 50.0%

USH-18 & CTH PD 22 of 50 44.0%

Total 79 of 211 37.4%

Locations with No Access Control

21 Remaining Locations

Reporting Accuracy (amount

correct) Percentage

Total 401 of 453 88.5%

USH 51 at Pflaum Rd (Madison)

Results: Horizontal Curvature

• Eleven sites had a curve present on at least one approach that should have been flagged

• Accuracy at locations with a curve present was much lower than those without a curve

Location Number of

Observations

ROADHOR

Accuracy

STH 26 & CTH N 23 69.6%

USH-14 & N Pontiac 34 97.0%

USH-51 & W Delavan 13 100.0%

Maple & Fourth 7 71.4%

US-51 & Anderson (CTH CV) 16 100.0%

W Main & O'Keeffe 12 91.7%

Century & Allen 14 78.6%

University & Allen 13 100.0%

Cumulative Total 132 89.3%

Other 16 Locations 481 97.1%

Results: Vertical Curvature

• Seven sites had a hill present on at least one approach that should have been flagged

• Accuracy at locations with a hill present was much lower than those without a hill

Location Number of

Observations

ROADVERT

Accuracy

STH 26 & CTH N 23 90.5%

USH-51 & W Delavan 13 84.6%

Johnson & Park 27 92.6%

Fish Hatchery & Post 18 72.2%

Fish Hatchery & Caddis 14 66.7%

US-14 & Deming Way 17 88.2%

University & Allen 13 38.5%

Cumulative Total 125 76.8%

Other 20 Locations 499 98.6%

Results: Traffic-way Divided without Barrier Totals Percentage

Correctly Reported 374 70.3%

Divided with Barrier 92 17.3%

Not Divided 59 11.1%

Blank 5 0.9%

One Way 2 0.4%

Total 532 100.0%

Not Divided Totals Percentage

Correctly Reported 18 41.9%

Divided without Barrier 24 55.8%

Blank 1 2.3%

Divided with Barrier 0 0.0%

One Way 0 0.0%

Total 43 100.0%

One Way Totals Percentage

Correctly Reported 36 97.3%

Not Divided 1 2.7%

Divided without Barrier 0 0.0%

Divided with Barrier 0 0.0%

Total 37 100.0%

True divided with barrier traffic-way http://forums.trinituner.com/upload/data/45/jersey%20barriers.jpg

• When facility was divided without a barrier, most common error was reporting a barrier • Roadways with no division (i.e. two lane road with painted centerline) were reported as divided over half the time

Results: Roundabouts

• Three roundabout intersections were studied

• The most inaccurate fields were traffic-way, horizontal curvature, and traffic control

S Towne &

Industrial

8th St &

Springdale

Commercial Ave

& N Thompson

Round-

about

Total

HWYCLASS 100.0% 100.0% 100.0% 100.0%

ACSCNTL 69.2% 100.0% 83.9% 82.0%

ROADHOR 30.8% 66.7% 29.0% 34.0%

ROADVERT 100.0% 100.0% 96.8% 98.0%

TRFCWAY 53.8% 50.0% 54.8% 54.0%

TRFCNTL1 92.3% 60.0% 86.2% 85.1%

TRFCNTL2 61.5% 100.0% 72.4% 72.3%

Results: Roundabouts

• Horizontal curvature reporting errors were tied to the circulating lane

• Majority of crashes flagged with a curve involved at least one vehicle in the circulating lane

Results: Roundabouts

• Traffic-way errors were most commonly associated with incorrectly using “not physically divided” or divided with a barrier

• Low traffic control accuracy most often resulted from officers indicating no traffic control, when the correct response should have been a yield sign

Results: Completeness & Agency Trends

• On the 664 crash reports in the study, there were only 16 “blank” entries (out of a possible 4,648)

• No notable trends were found based agency type (e.g. police, sheriff, state patrol), however, only 6 locations had multiple responding agencies

Conclusions

• Low accuracy for access control at partially controlled facilities indicates a lack of understanding by officers for what qualifies as partial control

• When hills or curves are present on at least one approach, officers are flagging them even when they are not the site of the crash, which is incorrect

• Low traffic-way accuracy is a result of misunderstanding what constitute divided roadways and barriers

• Roundabout-specific inaccuracies were especially noteworthy in the horizontal curve and traffic-way fields

Recommendations

• Update WisDOT Instruction Manual and develop pocket-sized “cheat sheet” versions

• In-service officer training with case-studies of commonly misinterpreted data fields

• Generate data fields by linking to roadway inventory and hardware data

Recommendations

• Implement 100 percent electronic crash reporting in Wisconsin

• Utilize GIS/GPS location identification technologies

• Update data fields to comply 100 percent with MMUCC, including roundabout-specific attributes

Thank You

We would be happy to answer any questions!