Wisconsin Traffic Operations and Safety Laboratory

Department of Civil and Environmental EngineeringUniversity of Wisconsin-Madison

Extracting Meaningful Results from

the National Performance

Management Research Data Set

Glenn Vorhes, PE, GISP

2018 AASHTO GIS for Transportation Symposium

March 20, 2018

gavorhes@wisc.edu

What is the NPMRDS?

• Fine grained vehicle travel time / speed information for all* of North America• Aggregated in 5 minute bins for passenger, freight, and combined.• Formerly originated from Navteq/HERE (V1) and later from INRIX (V2)• Derived from probe vehicle data. Passive, No fixed/moving sensors, “floating car”

• Cellular• Reidentification (Bluetooth)• Toll tags, license plates, MAC IDs, etc.• GPS breadcrumbs

• Time, location, speed, heading• Processing done internally; results from the black box. Typically consistent and

clean.

• Unit of analysis is the TMC (Traffic Message Channel)• Directional segments** with lengths from 0.4 to 3.4 miles, ~1 mile typically

• Free!*** GPS probe data costs hundreds, up to thousands for real time

* Limited to the NHS** Structure and definition of TMC segments changed with the current version, still changing*** No free lunch, data is available but comes with usage restrictions

What’s it Good For?• Performance Metrics, MAP-21 reporting requirements

• Travel Time Index peak-period travel time / free-flow travel time*• Planning Time Index 95th percentile travel time / free-flow travel time*

• Congestion Analysis• Delay Calculations• Work zone performance, impacts

• Let the data determine the baseline for each segment• *FHWA free flow – 85th percentile off-peak times, M-R: 9-4, 7-10, SU: 6am-10pm• Make your own! Ex. Before, during, after construction

Pros and Cons (challenges)

+ -

• Spatially and temporally granular• Compares favorably with other data sources

• No volume information• Other sources needed for delay

• NoData• No readings or too slow

• Low sample size• One vehicle?

• May have sampling bias• Seems high on freight vehicles

• Not useable as a navigable network for routing applications

• Version 2• Naming inconsistencies

• Ex. Name: “I-39 N”, Direction: “N”• Moving target

• Addition, removal, and modification of TMCs without formal release schedule

Project GoalsStorage, Access, and Visualization of NPMRDS

• NPMRDS is a valuable resource albeit with some processing and storage difficulties

• Received as a collection of shapefiles and CSVs• Quickly exceeds limits of Excel• Joins with spatial data for visualizations inefficient at best

• Two NPMRDS Versions with widely differing structures increase complexity• A database solution is necessary

• Efficient joins• Indexing on TMC id and time representation• In database stored procedures/functions for aggregation, statistics

• Data is right there• Beats any attempt at a full query with for subsequent analysis with

Python (NumPy/SciPy), R, Stata, MATLAB, etc.• Spatial indices for efficient queries by extent (PostgreSQL/PostGIS)• Power and storage for billions of travel time rows (Oracle)

Version Transition – Data AcquisitionVersion 1 Version 2

• FTP Site managed by HERE (formerly Navteq)

• Handy method for bulk downloads for whole of North America

• Huge files requiring considerable processing just to open to say nothing of database storage; gzipped tarball with gzipped CSVs

• Regional Integrated Transportation Information System (RITIS)

• NPMRDS component managed by Center for Advanced Transportation Technology (CATT) Laboratory at the University of Maryland

• Massive Data Downloader• Nice query tool, harder for bulk download

• Other analytics tools in dashboard, not free

8 GB+

RITIS NPMRDS Interface

npmrds.ritis.org

Version Transition – Spatial DataVersion 1 Version 2

• “Static” Table • Road Name, Direction• Complementary Information

• Links (Shapefile)• Geometries• Can be shared between TMCs• Digitization inconsistent with travel

• Lookup Table• Link Id, TMC, Digitization/Travel Flag

• Database procedure to join TMCs and links• Copy if necessary• Flip if necessary – travel and digitization

direction match• Dissolve by TMC• Simple spatial analysis and mapping

• Flat Structure – No more lookup table• Geometry and road information in the

same table• Travel and digitization consistent• No shared geometries

• Introduction of inner and outer TMCs

• Example 110_01235 where _ is:• P (N/E) or N (S/W), Internal Paths• + (N/E) or - (S/W), External Paths

Version Transition – Times/SpeedsVersion 1 Version 2

• Date-Time as separate columns• Date• Epoch, 0-287 5 minute bins per day

• Travel times for passenger, freight, and all/combined in one row

• TMC lengths can change• Typically small changes but some

considerably larger ones• Updated once or twice a year• Need mechanism to reference a specific

version

• All of North America loaded• 21,621,017,302 Travel Time Rows

• True datetime representation in CSV• mm/dd/yyyy HH:MM

• Selectable metrics in download interface• Speed• Historic Average Speed• Reference Speed• Travel Time• Data Density?

• Can choose any or all of passenger, freight, or all vehicles

• Each vehicle classification in one row• Requires additional indexing

• Only loaded for Wisconsin so far,• Plans for more, Surrounding States, beyond

Coverage Changes

•Version 2 is mostly a subset of Version 1

•Very few additions

•Version 2 has been a moving target –Coverage updates but no release schedule

•Both versions were/are intended to cover the Federal Highway Administration (FHWA) National Highway System (NHS) and nothing more

•Depends on State input

Coverage

NHS vs

NPMRDS

•Good but certainly not perfect

• Excellent resource but causes a lot of frustration for performance metrics

NPMRDS API

• RESTful interface to retrieve desired NPMRDS information• To date, focused on consumption by web applications, JSON format• Planned to allow an additional format request parameter to trigger

download of a zip file containing spatial and tabular information

• Eliminates need for version specific application code

• Only requirement is that the request doesn’t span the version break• Underlying geometries are different• Database queries against separate tables

• Development Efforts• Smooth the transition, TMC Matching• Improved query performance• Spatial and date range limits• API key and request logging

Architecture

Database

Server

Time/Speed Data Spatial Data

Version 1< 1/31/2017

Version 2> 2/1/2017

Dependent ApplicationsAPI Request

(Geo)JSON

Desktop GISArcMap Query LayerQGIS native support

npm install github:glennvorhes/webmapsjsExample: import {quickMap} from 'webmapsjs/dist/olHelpers/quickMap’;Documentation: https://transportal.cee.wisc.edu/gis/webmaps/webmapsstatic/docs/js/index.html

Statistics

Example Applications – API GUI

Heatmap GeneratorMajor Incident occurred on March 27, 2015 near Janesville, WIOverturned semi, chemical spillHeatmap provides visualization of travel speed relative to TMC free flow speed

Historic Travel TimesShow progression of delay for all segments in a region over the course of a dayMoving window average with user selectable range (20 minutes down to raw 5 minute bins

August 21, 2017 - Eclipse

Year ComparisonChange in a variety of metrics from one year to the next.Example: Measure of travel time reliability (80th / 50th percentile) speedLower is better; more consistent travel times

Conclusion

• Development work yet to be done• Improved query performance, smarter indices• Smooth transition between NPMRDS versions• More retrieval formats• New visualizations

• Reduced barrier to entry for web application and advanced desktop analytics

Thank you for coming!Questions?