State of the Practice for ITS Data Archiving:

Regional Data Archive and Performance Measures

Robert L. BertiniPortland State UniversityNorth American Travel Monitoring Exhibition & ConferenceSan Diego, California June 30, 2004

2

Objectives

Introduction to Data Archiving Project

National ITS Architecture and ADUS

Introduce Portland Regional ADUS

Describe Architecture

Describe Database Processing and Storage

Data Analysis and Visualization

Conclusions and Next Steps

3

IntroductionGuiding Principles

“Data are too valuable to only

use once.”

4

IntroductionGuiding Principles

“Management of the transportation system cannot be done without knowledge of its performance.”

5

National ITS Architecture Major Components

Travel and Traffic Management Public Transportation Management Electronic Payment Commercial Vehicles Operations Emergency Management Advanced Vehicle Safety Systems Information Management Maintenance and Construction Management

6

Overview: Relational Diagram of ITS Architecture

Source: Guidelines for Developing ITS Data Archiving Systems

7

Archived Data User Service (ADUS) is BornITS Architecture 1999

USDOT Vision for ADUS: “Improve transportation decisions through the archiving and sharing of ITS generated data.”

Principles of ADUS to Achieve Vision All ITS deployments should consider data archiving Archive data to maximize integration with other data

sources and systems Archive data in a way that eases retrieval for those with

access Provide information that is integral to transportation practice

8

Archived Data User Service (ADUS) is BornADUS Standards

Operational Data Control: by managing operations data integrity

Data Import and Verification: through historical data Automatic Data Historical Archive: with a permanent data

archive Data Warehouse Distribution: which integrates the planning,

safety, operations and research communities and processes data for these communities

ITS Community Interface: by providing a common interface to all ITS users for data products specification and retrieval

9

Who Can Use Archived ITS DataStakeholders

Transportation Planning Transportation System Monitoring Air Quality Analysis MPO/State Freight and Intermodal Planning Land Use/Growth Management Planning Transportation Administrators and Policy Analysis Traffic Management Transit Management Construction and Maintenance Safety Planning and Administration Commercial Vehicle Operations Emergency Management Transportation Research Private Sector

10

Data ResourcesPotential Archiving Applications

Traffic Surveillance

Fare/Toll Systems

Incident Management

Traffic Video

Environmental

CVO

Traffic Control

Highway/Rail

Emergency Response

ITSITSDataData

ArchivesArchives

ITSITSDataData

ArchivesArchives

Performance MonitoringNational reportingPerformance-based planningEvaluationsPublic Reactions

Long Range PlanningTRANSIMSIDASFour step modelsTransit routes

Operations PlanningIncident managementER deploymentSignal timingTransit service

Travel Time ForecastingCustomized route planningATIS Advisories

Other Stakeholder FunctionsSafetyLand useAir qualityMaintenance management

11

ADUS Architecture:Implementing a Successful Data Archive

A Few Examples of Existing Systems California PeMS - only statewide system Puget Sound (WSDOT/TRAC) - started small and

successfully expanded San Antonio, TX TransGuide, Datalink system

Lessons Learned from Existing Systems Begin with a single data source Provide data through the web or CD subscription Create a user friendly interface Save raw data Make aggregate data available to users Implement data quality control measures Create adequate documentation of system and metadata

12

Portland ADUSPSU Designated Regional Archive Center

13

Portland ADUSPSU Designated Regional Archive Center

Through regional cooperation, Portland State University is the regional center for collecting, coordinating and disseminating variable sources of transportation data and derived performance measures.

Portland ITS data will be warehoused at their most raw level.

A user interface for extracting relevant performance measures in real time and historical data is being implemented.

14

Portland ADUSData Sources

Oregon Department of Transportation (ODOT) Freeway loop detector data Incident data

National Oceanic and Atmospheric Administration (NOAA)

Weather data

Portland Metropolitan Transit Agency (Tri-Met) Automatic vehicle location (AVL) poll data Bus dispatch system (BDS) data

City of Portland Traffic signal count/speed data

Washington State Department of Transportation (WSDOT) ODOT Weigh-in-motion data

15

Portland ADUSPortland Regional Infrastructure 77 CCTV Cameras 18 Variable Message Signs

(VMS) 436 Inductive Loop Detectors 118 Ramp Meters TriMet Automatic Vehicle

Location (AVL) System and Bus Dispatch System (BDS)

Extensive Fiber Optics Network

16

ADUS ArchitectureFiber Optic Data Connection

Rich Johnson, City of Portland

17

Portland ADUS ArchitectureData Flows

ODOTTMOC

Loop Controllers

Fiber

PSUADUSServer

Fiber

WebServer

TriM

et

WS

DO

T

City

of

Por

tlan

dFuture

BackupServer

Operations Planning PublicUser Classes

18

Portland ADUS ArchitectureDatabase Back End/ Web Front End

Database Back End SQL relational database 200 MB per day of raw data 75 GB per year High capacity disk array 5 MB compressed for download

Off-site backup server Daily backups Uninterruptible power

Web-based interface Easily accessible User-friendly PHP language

Database Processing and StorageWeb Interface is Up and Running

20

Database Processing and Storage Data Fidelity

Loop health analysis Proven techniques to prevent propagation of error

Aggregation Levels Raw 20-second loop detector data collected

Useful for research Large data sets Previously difficult to manage and organize

High resolution aggregates 5 and 15 minute aggregates Flow, speed and occupancy characteristics retained VMT, VHT, travel time, delay added Useful for research and planning purposes

Lower resolutions aggregates Hourly or Daily VMT, VHT, travel time, delay Increased processing speed for long term overviews

Data Analysis and VisualizationContour Plots

Data Analysis and VisualizationTime Series Plots

Data Analysis and VisualizationData Fidelity

24

Next StepsExpanding Archive Functionality

Data from: TriMet Washington State DOT City of Portland ODOT WIM Data

Additional processing tools and performance measures

Acknowledgments

National Science Foundation Oregon Department of Transportation City of Portland TriMet Portland State University Oregon Engineering and Technology

Industry Council

26

FeedbackYour Opinions and Suggestions

Please let us know what you think about existing or future:

Data sources Performance measures Data visualizations and summaries

Contact us:

bertini@pdx.eduhttp://www.its.pdx.edu