tutorial on taffic management and ai
DESCRIPTION
This is the tutorial on AI techniques for traffic management presented at the AAAI 2012 conference, Toronto, Canada presented by Biplav Srivastava and Anand Ranganathan.TRANSCRIPT
![Page 1: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/1.jpg)
Tutorial: Traffic Management and AI
Biplav Srivastava, Anand RanganathanIBM Research
AAAI 2012 Tutorial
22nd July 2012
© 2012 IBM Corporation
Toronto, Canada
![Page 2: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/2.jpg)
What to Expect: Tutorial Objectives
� The aim of the tutorial is to make early and experienced researchers aware of the traffic management area, provide – an insightful overview of the current efforts using AI techniques
• in Research and • in practice (real world pilots), and
– whet interest for newer efforts on important open issues.
© 2010 IBM Corporation
� From the call for tutorial: “a second type of tutorial provides a broad overview for an AI area that potentially crosses boundaries with an interesting application area”.
� Disclaimer: we are only providing a sample of the traffic management space intended to
match audience profile in the available time.
2
![Page 3: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/3.jpg)
Outline
1. Traffic Management Problem
2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data
3. Interconnection1. Middleware 2. Traffic standards
© 2010 IBM Corporation
2. Traffic standards
4. Intelligence1. Path planning
1. Simple Illustration2. Path Planning for Individual Vehicles
2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors
5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots
3
![Page 4: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/4.jpg)
Acknowledgements
All our collaborators, and especially those in:
� City agencies around the world– Bengaluru, India– Boston, USA– Dublin, Ireland– Ho Chi Minh City, Vietnam– New Delhi, India– New York, USA
© 2010 IBM Corporation
– New York, USA– Stockholm, Sweden
� Academia
� IBM: Many including - Raj Gupta, Ullas Nambiar, Srikanth Tamilselvam, L V Subramaniam, Chai Wah Wu, Anand Paul, Milind Naphade, Jurij Paraszczak, Wei Sun, Laura Wynter, Olivier Verscheure, Eric Bouillet, Francesco Calabrese, Tsuyoshi Ide, Xuan Liu, Arun Hampapur, Nithya Rajamani, Vivek Tyagi, Raguram Krishnapuram, Shivkumar Kalyanraman, Manish Gupta, Niterndra Rajput, Krishna Kummamuru, Raymond Rudy, Brent Miller, Jane Xu, Steven Wysmuller, Alberto Giacomel, Vinod A Bijlani, Pankaj D Lunia, Tran Viet Huan, Wei Xiong Shang, Chen WC Wang, Bob Schloss, Rosario Usceda-Sosa, Anton Riabov, Magda Mourad
4
For discussions, ideas and contributions. Apologies to anyone unintentionally missed.
![Page 5: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/5.jpg)
Traffic Management and AI
Section: Traffic Problem
Speaker: Biplav Srivastava
AAAI 2012 Tutorial
July 2011
© 2012 IBM Corporation
Toronto, Canada
![Page 6: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/6.jpg)
Outline
1. Traffic Management Problem
2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data
3. Interconnection1. Middleware2. Traffic standards
© 2010 IBM Corporation
2. Traffic standards
4. Intelligence1. Path planning
1. Simple Illustration2. Path Planning for Individual Vehicles
2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors
5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots
6
![Page 7: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/7.jpg)
We All See Traffic Daily. An Illustration from Across the Globe
Characteristics New York
City, USA
New Delhi,
India
Beijing, China Moscow,
Russia
Ho Chi Minh City,
Vietnam
Sao Paolo, Brazil
1 How is traffic pre-dominantly managed
Automated control, manual
control
Manual control
Automated control, manual
control
Automated, manual control
Manual control Automated, manual control, Rotation
system (# plate based)
2 How is data collected Inductive loops, cops,
video, GPS
Traffic surveys, cops
Video, GPS, cops GPS, some video, cops
Traffic surveys, cops Video, GPS, cops
3 How can citizens manage their resources
GPS devices, alerts on radio,
web, road signs (variable)
Alerts on radio
alerts on radio, road signs
(variable), mobile alerts
GPS, radio, road signs,
mobile alerts
Alerts on radio GPS devices, alerts on radio, web
© 2010 IBM Corporation
Source: Google map for New York City and New Delhi; Search done on Aug 20, 2010
4 Traffic heterogeneity by vehicle types(Low: <10;
Medium 10-25; High: >25)
Low High Low Low Medium Low
5 Driving habit maturity (Low: <10 yrs; Medium:
10-20; High: > 20)
High Low Low Low Low Medium
6 Traffic movement Lane driving Chaotic Lane driving Lane driving Chaotic Lane Driving
![Page 8: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/8.jpg)
Is it Just Supply-Demand Mismatch?
� Supply – Roads and their capacity– Personnel available– Capital and operational budget
� Demands – Travel needs of citizens
© 2010 IBM Corporation
– Travel needs of citizens– Travel needs of organizations: businesses, governments
� Solution to mis-match?– Keep building supply (roads, bridges, …)– Keep reducing demand (restrict citizens, businesses, …)– May work for some of the cities, for some of the time,
• but not for all of the cities and all of the time
8
![Page 9: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/9.jpg)
Levers in Cities’ Hands for Traffic Management
� Physical infrastructure– New flyovers, roads– Expanding existing capacity– New metros, …
� Policies
© 2010 IBM Corporation
� Policies– Relocating businesses– Incentivising public transport
� IT enabled technologies– Intelligent Traffic/ Transportation Systems– Social networking
![Page 10: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/10.jpg)
Car (car +motorcycle) takes less time regardlessof distance unless
there is congestionon road.
(a) 3 km trip (b) 6 km trip
© 2010 IBM Corporation
Which Mode of Transport is Best for a Particular Distance – Door-to-Door Trip Times
See speaker notesfor more details.
Source: Mythologies, Metros & Future Urban Transport , by Prof. Dinesh Mohan, TRIPP, 2008
Data used:
• international data on things like access time to public transportation
was taken
•Studies were done for Delhi Metro
• Minimum assumptions were made.
•E.g., walking time for metro access (5 mins) is widely acceptable
•See speaker notes too
(c) 12 km trip (d) 24 km trip
![Page 11: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/11.jpg)
Why Focus on Mere Congestion Leads to Anomalies
� The Pigou-Knight-Downs paradox – States that adding extra road capacity to a road does not reduce travel time. – This occurs because traffic may simply shift to the upgraded road from the other, making the upgraded road more congested.
� The Downs-Thomson paradox – States that the equilibrium speed of car traffic on the road network is determined by the average door-to-door speed of equivalent journeys by public transport.
© 2010 IBM Corporation
public transport. – This occurs when the shift from public transport causes a disinvestment in the mode such that the operator either reduces frequency of service or raises fares to cover costs.
� The Braess' paradox – States that adding extra capacity to a network, when the moving entities selfishly choose their route, can in some cases reduce overall performance and increase the total commuting time.
11
Details:
Arnott, Richard and K.A. Small, 1994, “The Economics of Traffic Congestion,” American Scientist, Vol. 82, No. 5, pp. 446-455.
Chengri Ding and Shunfeng Song , Paradoxes of Traffic Flow and Congestion Pricing,
![Page 12: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/12.jpg)
Pigou-Knight-Downs Paradox
Highway
Bridge
� Bridge is the direct route between A-B, while highway is the circuitous route
� The average travel time on the (congested) bridge is a linear function of the flow-to-capacity ratio and the average travel time on the uncongested highway is a constant.
� a, b, and d are positive parameters with d > a; At equilibrium, travel times are same on both routes.
� Paradox exists for any bridge capacity less than C1, because expanding the bridge will only shift travelers to the bridge but not reduce travel time.
.
© 2010 IBM Corporation12
Details:
Arnott, Richard and K.A. Small, 1994, “The Economics of Traffic Congestion,” American Scientist, Vol. 82, No. 5, pp. 446-455.
Chengri Ding and Shunfeng Song , Paradoxes of Traffic Flow and Congestion Pricing,
� Using a=10, b=10, d=15, and F=1,000, Arnott and Small (1994) showed that the average travel time remains at 15 even if the bridge capacity continues to increase until it reaches 2,000, which is twice as large as the total traffic volume
� How to address paradoxes– Car commuters ignore how much delay they cause on other travelers but only pay attention to how long it takes them to commute.
– Delay cost to others must be quantified and accounted for as social cost.
– If social cost is introduced as tolls, the paradox goes away
+=
1
11
C
FbaT ; dT =2 ; FFF =+ 21 ;
21 TT = ( )adbF
C −= 11
![Page 13: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/13.jpg)
Key Insights for a New Traffic Problem Look
� Two category of stake-holders putting their resources– Public resources
• Example: $1M per month for a city with 5M population• Example: $100M available for roads and bridges for the next 3 years.
– Private resources (often ignored)• Example: Average time taken to travel 1 KM • Example: Average $ needed to travel 1 KM
© 2010 IBM Corporation
• Example: Average $ needed to travel 1 KM
� Impact of time-frame– In short-term, new physical supply cannot be created due to long construction time, time to hire personnel, etc
– Public resources have a short-term and a long-term cycle, while private resources for transportation need is time-frame insensitive
� Approach: optimize both public as well as private resources
13
![Page 14: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/14.jpg)
Problem Statement for Traffic Management from Engineering Perspective
� Problem (Short Term)Match traffic demand to supply with optimal usage of available public resources and
concomitant optimization of citizens’ private resources for travel needs
– Example: For a given day, • minimize over-time payments to traffic personnel , while • minimizing average commute time per km.
– Alternative: Service objectives can also be stated like average commute time per km be below 10 mins/ km.
© 2010 IBM Corporation
� Implications: Conflicting goals: optimality of public resources (e.g., road, traffic cops) v/s optimality of individual’s resources (e.g., travel time)
14
Details: Biplav Srivastava. A new look at the traffic management problem and where to start. In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.
![Page 15: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/15.jpg)
Problem Statement for Traffic Management from Engineering Perspective
� Problem (Short Term)Match traffic demand to supply with optimal usage of available public resources and concomitant optimization of citizens’ private resources for travel needs
– Example: For a given day, minimize over-time payments to traffic personnel while minimizing average commute time per km. Serviceobjectives can also be stated like average commute time per km be below 10 mins/ km.
– Implications: Conflicting goals: optimality of public resources (e.g., road, traffic cops) v/s optimality of individual’s resources (e.g., travel time)
� Problem (Long Term)For a known future period, match traffic demand to supply with optimal usage of available
and planned public resources and concomitant optimization of citizens’ private
© 2010 IBM Corporation
resources for travel needs.
– Example: For the next 3 years, minimize city’s expenses (annual operational budget and capital investment in the period) while minimizing average travel time per km and average fare per km. Service objectives can also be stated towards citizens like average commute time be below 10 mins/ km and average fare be below $1/ km.
� Implications: Information is used to plan city’s regions (including businesses, communities) and roads, thereby influencing traffic patterns
15
Details : Biplav Srivastava. A new look at the traffic management problem and where to start. In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.
![Page 16: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/16.jpg)
Starting Point from Problem Statement for Traffic Management
� Problem (Short Term)Match traffic demand to supply with optimal usage of available public resources and concomitant optimization of citizens’private resources for travel needs
– Example: For a given day, minimize over-time payments to traffic personnel while minimizing average commute time per km. Service objectives can also be stated like average commute time per km be below 10 mins/ km.
– Implications: Conflicting goals: optimality of public resources (e.g., road, traffic cops) v/s optimality of individual’s resources (e.g., travel time)
� Problem (Long Term)For a known future period, match traffic demand to supply with optimal usage of available and planned public resources
and concomitant optimization of citizens’ private resources for travel needs.
© 2010 IBM Corporation
– Example: For the next 3 years, minimize city’s expenses (annual operational budget and capital investment in the period) while minimizing average travel time per km and average fare per km. Service objectives can also be stated towards citizens like average commute time be below 10 mins/ km and average fare be below $1/ km.
– Implications: Information is used to plan city’s regions (including businesses, communities)and roads, thereby influencing traffic patterns
� Starting point for any solution requires traffic to be measured accurately at sustained, continuous basis, at a rate that helps effective traffic control
16
Details: Biplav Srivastava. A new look at the traffic management problem and where to start. In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.
![Page 17: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/17.jpg)
strategic
planning
Level 1Silo
Level 2Centralized
Level 3Partially Integrated
Level 4Multimodal Integrated
Level 5Multimodal Optimized
PlanningFunctional Area Planning (single mode)
Project-based Planning (single mode)
Integrated agency wide planning (single mode)
Integrated corridor-based multimodal planning
Integrated regional multimodal planning
Data Collection Limited or Manual Input
Near real-time for major routes
Real-time for major routes using multiple inputs
Real-time coverage for major corridors, all significant modes
System-wide real -time data collection across all modes
Performance
MeasurementMinimal
Defined metrics by mode
Limited integration across organizational silos
Shared multimodal system-wide metrics-
Continuous system-wide performance measurement
Customer
Management
Minimal capability, no customer accounts
Multi-channel account interaction per mode
Unified customer account across multiple modes
Integrated multimodal incentives to optimize multimodal use
Customer accounts managed separately for each system/mode
“Average” City
Top 3 City: range
Leading Practice
A Summary of where Global Cities are within the Transportation Maturity Model
Consulting Solution Approach
© 2010 IBM Corporation
real-time
information
creation
capability
real-time
intervention
capability
Data Integration Limited NetworkedCommon user interface
2-way system integration
Extended integration
Network Ops.
ResponseAd-Hoc, Single Mode
Centralized, Single Mode
Automated, Single Mode
Automated, Multimodal
Multimodal Real-time Optimized
Payment
MethodsManual Cash Collection
Automatic Cash Machines
Electronic PaymentsMultimodal integrated fare card
Multimodal, multi-media (fare cards, cell phones, etc)
Incident
Management
Manual detection, response and recovery
Manual detection, coordinated response, manual recovery
Automatic detection, coordinated response and manual recovery
Automated pre-planned multimodal recovery plans
Dynamic multimodal recovery plans based on real-time data
Demand
ManagementIndividual static measures
Individual measures, with long term variability
Coordinated measures, with short term variability
Dynamic pricingMultimodal dynamic pricing
Traveler
Information Static Information
Static trip planning with limited real-time alerts
Multi-channel trip planning and account-based alert subscription
Location-based, on-journey multimodal information
Location-based, multimodal proactive re-routing
Analytics Ad-hoc analysisPeriodic, Systematic analysis
High-level analysis in near real-time
Detailed analysis in real-time
Multi-modal analysis in real-time
Multimodal Network Management Maturity Model version 1.1 © Copyright IBM Corporation 2007
More details at: http://www-935.ibm.com/services/us/igs/pdf/transport-systems-white-paper.pdf
![Page 18: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/18.jpg)
An Intelligent Transportation Integration & Analytics Framework
Integration & AnalysisCollection
In-Vehicle Detectors
Mapping Services
(Creation and Maintenance of maps etc)
DisseminationCommand & Control
Roadside Infrastructure
(Variable Message Signs, Traffic Signals
etc)
Traveller Information Portal
In- vehicle
Devices
Mobile
Devices
Data Fusion(Creation of Single View)
Data Fusion(Creation of Single View)
Route Guidance &
Traffic Control
Center Dashboard
ITS Asset
Management,
Maintenance &
Optimization
Incident
Integration
Transport Management Subsystems
(Traffic Signals, Bridges, Parking, Mgt, etc)
Technology Approach
© 2010 IBM Corporation18
Mass Transit
Operational
Management (timetables, etc)
Infrastructure Detectors
(Loops, CCTV, Tag & Beacon etc)
In-Vehicle Detectors
(Probes etc)
Third Party
Information Feeds (e.g. Floating Cellular
Data)
Portal
(Info Display, Route Planning)
Traveler Information Gateway
(B2B info feeds etc)
Traveller Information Gateway (B2B info Feeds,
etc).
Self- service
Traveler Portal (view bill, pay charge,
etc)
External Systems (Vehicle Licensing,
Electronic Payments,
Bill Printing, etc)
Devices
Web
Browsers
Third Party
Systems
Mass Transit
Operational
Management (timetables, etc)
Data Warehouse(Reporting, Archiving)
Data Analytics(Forecasting , Journey Time
Determination)
Security
Systems Management
Route Guidance &
Trip Planning
Incident Management
(Process Execution)
Geographic
Information
System
Geographic
Information
System
Vehicle
Identification
Pricing ModelsPublic Access
Devices
(e.g. kiosks)
Integration
Integration
CRM (Call Centre, Interactive Voice)
Finance en& Paymts
Pricing Models
![Page 19: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/19.jpg)
References
� Mythologies, Metros & Future Urban Transport , by Prof. Dinesh Mohan, TRIPP, 2008
� A new look at the traffic management problem and where to start, by Biplav Srivastava, In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.
� Arnott, Richard and K.A. Small, 1994, “The Economics of Traffic Congestion,” American Scientist, Vol. 82, No. 5, pp. 446-455.
© 2010 IBM Corporation
� Chengri Ding and Shunfeng Song , Paradoxes of Traffic Flow and Congestion Pricing,
19
![Page 20: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/20.jpg)
Traffic Management and AI
Section: Traffic Sensing
Speaker: Biplav Srivastava
AAAI 2012 Tutorial
July 2011
© 2012 IBM Corporation
Toronto, Canada
![Page 21: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/21.jpg)
Outline
1. Traffic Management Problem
2. Instrumentation
1. Sensing traffic
2. Traffic state estimation
3. Optimizing and combining sensor data
3. Interconnection1. Middleware2. Traffic standards
© 2010 IBM Corporation
2. Traffic standards
4. Intelligence1. Path planning
1. For individual vehicles2. For public transportation
2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors
5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots
21
![Page 22: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/22.jpg)
A Feel For Traffic Sensing
No. Data type Format Ability for using in sensing
I1 Collected by manual
method
Document Can be used right after they
are integrated, cleaning and
conversion
I2 Mobile data CDR Data will be converted to
transport data in standard
format during project
implementation
I3 Data from mobile
which have GPS
Follow format
of data traffic
Data will be collected directly
during project deployment and
I3: 28km/hr
I2: 20km/hr
I1: 25km/hr
I2: 20km/hr
I2: 4 km/hr
© 2010 IBM Corporation
device switch to the format needed
I1: 25km/hr
Situation: Sensor readings from different types, various locations; for some links, no sensor present
Problem: What is the overall view of traffic?
![Page 23: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/23.jpg)
Sensed Traffic Metrics
� Traffic flow – number of vehicles* that pass a certain point during a specified time interval (vehicles/hour)
� Speed – rate of motion in distance/time (mph)
� Density – number of vehicles occupying a given length of highway or lane (vehicles per mile per lane)
© 2010 IBM Corporation
per lane)
� Spacing – the distance between successive vehicles in a traffic stream as they pass some common reference point on the vehicles
� Time headway – the time between successive vehicles in a traffic stream as they pass some common reference point on the vehicles
23
* What is a vehicle? There are 48 types in India! So measurement can be non-trivial!
![Page 24: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/24.jpg)
Traffic Flow and Time Headway
Traffic Flow given by:
t
nq =
q= traffic flow in vehicles per unit time
t= duration of time interval
n= number of vehicles passing some
designated roadway point during time interval t
© 2010 IBM Corporation
![Page 25: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/25.jpg)
Time Mean Speed
� Measure of speed at a certain point in space
Arithmetic mean of vehicles speeds is given by:
n
∑
© 2010 IBM Corporation
n
u
u
n
i
i
t
∑== 1
ut=time-mean speed in unit distance per unit time
ui=spot speed of the ith vehicle
n=number of measured vehicle spot speeds
![Page 26: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/26.jpg)
Space Mean Speed
� Time necessary for a vehicle to travel some known length of roadway
t
lus =
© 2010 IBM Corporation
∑=
=n
i
itn
t1
1
us= space-mean speed in unit distance per unit time
l = length of roadway used for travel time measurements of vehicles
t = vehicle travel time
ti= time necessary for vehicle i to travel a roadway section of length l
![Page 27: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/27.jpg)
Evolution of Traffic Flow Sensor Technology
Goal: Get traffic information (speed, volume) for city region on sustained, continuous basis, at a rate that helps effective traffic control at low cost
Latest Buzz: • Social Media• Mobile phones
© 2010 IBM Corporation27
1960s
Inductive-loop detector
1970s
Video image processor
1990s
GPS-based floating car
After 2000
Mobile phones as traffic probes
1920s
Auto traffic signal
• High spatial coverage• Cost effectiveness• Network-wide info.• 24x7 availability
• Weather insensitive• Rich traffic data• High flexibility
• High spatial coverage• Cost effectiveness• Network-wide info.• 24x7 availability
• Weather insensitive• Rich traffic data• High flexibility
• Network-wide info• Enlarged coverage• Increased flexibility• Rich traffic data through XFCD
• Network-wide info• Enlarged coverage• Increased flexibility• Rich traffic data through XFCD
Accuracy & reliabilityAccuracy & reliability
• Multiple lanes and zones monitoring• Rich traffic data
• Flexibility
• Multiple lanes and zones monitoring• Rich traffic data
• Flexibility
Source: IBM China Research Lab
![Page 28: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/28.jpg)
Conventional Sensing Approach
� Take whatever sensors are in place and in immediate control of the concerned department, ignoring other sensing data
� Sample References– Handbook reference for : inductive loop detectors, magnetic sensors and detectors, video image processors, microwave radar sensors, laser radars, passive infrared and passive acoustic array sensors, and ultrasonic sensors, plus combinations of sensor technologies. http://www.fhwa.dot.gov/publications/research/operations/its/06108/
– Inductive loop detectors: Ingram, J.W. The Inductive Loop Vehicle Detector: Installation Acceptance Criteria and Maintenance Techniques, Report No. TL 631387, prepared by California Department of Transportation, Sacramento, CA, for Federal Highway Administration, Washington, DC. U.S. Department of Commerce, National Technical Information Service, PB-263 948, Washington, DC, March 1976.
© 2010 IBM Corporation
National Technical Information Service, PB-263 948, Washington, DC, March 1976.– Manual: N. Bansal, and B. Srivastava. On Using Crowd for Measuring Traffic at Aggregate Level for Emerging Countries. IIWeb workshop, WWW 2011, Hyderabad, India, March 28, 2011.
– Audio: H. Bischof, M. Godec, C. Leistner, B. Rinner, and A. Starzacher. Autonomous Audio-Supported Learning of Visual Classifiers for Traffic Monitoring. IEEE Intell. Sys., 25(3) pg 15–23, May/June 2010.
– Video: M. Bramberger, R. Pflugfelder, A. Maier, B. Rinner, B. Strobl, and H. Schwabach. A Smart Camera for Traffic Surveillance. WISES03 pages 12, Vienna, Austria, June 2003.
– Mobile: Y. Zhao. Mobile Phone Location Determination and Its Impact on Intelligent Transportation Systems. IEEE Trans. ITS, col. 1, NO. 1, Mar. 2000.
– GPS: M. A. Quddus, W. Y. Ochieng, and R. B. Noland. Current mapmatching algorithms for transport applications: State-of-the art and future research directions. Transportation Research, Part C, vol. 15, no. 5, pp. 312–328, Oct. 2007.
– Hybrid: M. Prashanth, V. N. Padmanabhan, and R. Ramjee. Nericell: rich monitoring of road and traffic conditions using mobile smartphones. Proc. ACM Sensys, Pg. 323-336, 2008
![Page 29: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/29.jpg)
Technology Complementarity
Inductive-loop detector
Timeliness
Accuracy
Reliability
Security & Privacy
Temporal coverage
Spatial coverageLow operational cost
Resource
consolidation
Network-wide
information
Rich traffic data
Analysis easiness
Video image processor
Timeliness
Accuracy
Reliability
Security & Privacy
Temporal coverage
Spatial coverageLow operational cost
Resource
consolidation
Network-wide
information
Rich traffic data
Analysis easiness
No Single Panacea
© 2010 IBM Corporation29
Trustiness (legal)Low capital cost Trustiness (legal)Low capital cost
Floating car data
Timeliness
Accuracy
Reliability
Security & Privacy
Temporal coverage
Spatial coverage
Trustiness (legal)Low capital cost
Low operational cost
Resource
consolidation
Network-wide
information
Rich traffic data
Analysis easiness
Mobile traffic probe
Timeliness
Accuracy
Reliability
Security & Privacy
Temporal coverage
Spatial coverage
Trustiness (legal)Low capital cost
Low operational cost
Resource
consolidation
Network-wide
information
Rich traffic data
Analysis easiness
Source: IBM China Research Lab
![Page 30: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/30.jpg)
Illustration of Some Sensor Types
© 2010 IBM Corporation30
![Page 31: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/31.jpg)
Induction Loop System
© 2010 IBM Corporation
31
![Page 32: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/32.jpg)
Average Speed Calculation from Inductive Loop Detector
� Method to calculate the average speed of the links:
© 2010 IBM Corporation
� Reliability of method depends on the estimated value for g
32
![Page 33: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/33.jpg)
Video-based Vehicle Counting
© 2010 IBM Corporation33
http://www.youtube.com/watch?v=KWobYJzQl_k
![Page 34: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/34.jpg)
34
Available Mobility Data
1. Passive collection:
on-call phonesstandby phones
3. GPS Enabled Phones:
Network Based Terminal Based
Need to upload the DataPower Consumption Issue
© 2010 IBM Corporation
LU (Location Update)
Location Area Dimension: 3~10km
Continuous comm. behaviors (SMS/MMS/Call) and HO (Handover)
Location Area Dimension: 100m~2km
2. Active collection:
Positioning TechniquesAccuracy: 50~200m
Accuracy: 5~30m
4. Other Software Installed:
Accuracy: 50m~2km
Network Overhead
Source: IBM China Research Lab
Y. Zhao. Mobile Phone Location Determination and Its Impact on Intelligent Transportation Systems. IEEE Trans. ITS, col. 1, NO. 1, Mar. 2000.
![Page 35: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/35.jpg)
Road-side Acoustics based Sensing
� Honking
� Road sounds–Engine–Road–Air
© 2010 IBM Corporation
–Air–…
35
![Page 36: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/36.jpg)
Honk-Ok-Please [IIT Bombay]
© 2010 IBM Corporation36``Horn-Ok-Please'', Rijurekha Sen, Bhaskaran Raman, Prashima Sharma, The Eighth International Conference on Mobile Systems, Applications, and Services, ACM MobiSys 2010, Jun 2010, San Francisco, CA
Summary
• Can detect speed with high accuracy (relativeerror < 20% in most cases)
• Can detect congestion v/s free-flow
• Results heavily dependent on positioning of sensors (recorders),
![Page 37: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/37.jpg)
Traffic State: Audio spectrograms of various traffic density states, free flow, medium and jammed, and their corresponding cues
Road-side cumulative acoustics composed of various noise signals for eg: tire noise, engine noise, engine idling noise, occasional honks, air turbulence noise;
Traffic density state determines combination of noise signals. Therefore use
© 2010 IBM Corporation
combination of noise signals. Therefore use the spectral modulation acoustic features along with the Gaussian mixture Hidden Markov models to statistically characterize these traffic density states.
Initial experimentation have established the feasibility and accuracy of using this approach.
Vehicular Traffic Density State Estimation Based on Cumulative Road Acoustics, IEEE Transactions on Intelligent Transportation Systems, 2011
Classification accuracy using MFCC featuresframes covering T sec of audio, SVM classifier
![Page 38: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/38.jpg)
Sensor Subset Selection and Optimization
© 2010 IBM Corporation38
![Page 39: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/39.jpg)
Problem Considered
� City authorities need to decide what sensors to use to get traffic data for traffic of their region– Slew of techniques available varying in accuracy, coverage and cost to install and maintain– Diversity in how they can be set up – How sensing methods may complement each other
� City can make an initial decision but it will need to be re-visited over time
© 2010 IBM Corporation
� City can make an initial decision but it will need to be re-visited over time– Traffic patterns changes– Sensing technologies changes
Problem : find the subset of sensors from available types that give the best cost-benefit for a given traffic pattern
![Page 40: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/40.jpg)
Effect of Increasing Number of Sensors on RMSE
© 2010 IBM Corporation
[Traffic Pattern 1 on a Grid] Effect of increasing number of sensors of different type. a)Shows that RMSE decreases with increase in percentage of sensors from 10 to 100%. b)Shows increase in cost is highest for manual sensors whereas it is lowest for CDR.
(a) (b)
![Page 41: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/41.jpg)
Subset Selection Approach
� Model sensor types based on cost, accuracy and coverage
� Create a sample space of sensor combination choices
� Use a traffic simulator (MATSIM)– To measure the sensing error distribution entailed in each sensor
combination– To ensure physical characteristics of the city are taken into account
� Choose Pareto sensor combinations (non-dominated); Call this “Optimal Candidate Set (OCS)”
© 2010 IBM Corporation
� Optional filtering steps– Remove combinations above a give cost threshold– Remove combinations above an error threshold
� For a given set of ‘k’ optimal combinations to be returned– Select a preference function– Use OCS selection using ICP approximation (Carlyle et al)
� Return ‘k’ optimal sensor combinations
Manual Video CDR GPS RMSE Cost Norm RMSE Norm Cost
100 0 0 0 1.2 4200 0 1
0 0 0 100 2.25 2520 0.028074866 0.592034968
0 0 100 0 6 840 0.128342246 0.184069937
0 0 40 0 20.55 335 0.517379679 0.061437591
0 0 10 0 38.6 82 1 0
0 0 10 0 38.6 82 1.00 0.00
0 0 20 0 29.4 167 0.75 0.02
0 0 30 0 25.05 247 0.64 0.04
0 0 40 0 20.55 335 0.52 0.06
0 0 50 0 17.85 414 0.45 0.08
0 0 60 0 15.6 500 0.39 0.10
0 0 70 0 11.95 586 0.29 0.12
0 0 80 0 9.65 668 0.23 0.14
0 0 90 0 8.45 753 0.19 0.16
0 0 100 0 6 840 0.13 0.18
0 0 100 10 5.8 1088 0.12 0.24
10 0 100 0 5.4 1175 0.11 0.27
0 0 100 30 5.1 1600 0.10 0.37
0 0 100 40 4.65 1835 0.09 0.43
0 0 100 50 4.45 2093 0.09 0.49
0 0 100 60 3.95 2356 0.07 0.55
0 0 0 100 2.25 2520 0.03 0.59
10 0 0 100 2.2 2855 0.03 0.67
10 0 10 100 2.15 2937 0.03 0.69
20 0 0 100 2.05 3348 0.02 0.79
30 0 20 100 2 4005 0.02 0.95
0
5
10
15
20
25
30
35
40
45
0 2000 4000 6000
CostRMSE
Non SelectedPareto
Selected Pareto
Sensor Subset Selection for Traffic Management, R. Gupta and B. Srivastava, in 14th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2011), Washington DC, USA, Oct 5-7, 2011.
![Page 42: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/42.jpg)
Examples of City Preferences in Sensor Selection
� (Case A): a city which already has some sensors in place would want to add only those new sensor types which complement its investments– Maximize incremental accuracy improvement
� (Case B): a city building the ITS infrastructure for the first time would want the highest accuracy possible within its budget– Highest accuracy
© 2010 IBM Corporation
– Highest accuracy
� (Case C): a city may have low budgets and would want to have high coverage even at the loss of some accuracy (which it may compensate by putting more people on the field)– Minimize cost
� (Case D): a city may want to minimize operational costs, even trading it off with higher capital expenditure, possibly because they come from a different budget– Minimize operational cost
42
![Page 43: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/43.jpg)
Key Results from Sensing
� Low-cost, noisy, CDR-based sensing complements existing sensors in a city due to easy coverage it provides– Highly beneficial when other, more accurate, sensor types are few– Benefit diminishes as the number of more accurate sensors increases– Important result for traffic management of emerging countries
© 2010 IBM Corporation
� Sensor type combinations can be suggested for a city’s traffic patterns using a simulator balancing accuracy and cost of sensing– Returns non-dominated results– Return few results approximating the search space
43
![Page 44: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/44.jpg)
Extracting Real City Characteristics for Simulation to Choose Sensors
© 2010 IBM Corporation44
![Page 45: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/45.jpg)
Combining Multiple Sensor Data
An example from Boston, USA completed in June 2012
© 2010 IBM Corporation45
![Page 46: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/46.jpg)
Traffic Context in Boston: Background
Consumers
� Urban planning users– Needs counts done annually (max) or more frequently
– Currently uses 10 year old data
� Environment users– Needs counts every 15 mins (min) or longer time
� Transportation users
Data Sources for Traffic Count
� Data in hand– Inductive loop detectors (Transportation asset)
– Manual count by consultants during urban development (Urban planning asset)
– Tube count (by state authorities)
� Future
© 2010 IBM Corporation
� Transportation users– Collects and uses counts every 15 mins
� Others– Residents: Consume for their personal choices
– Businesses: identify new opportunities
� Future – Video camera (multiple organizations)– GPS (city vehicles)– Directly by citizens (e.g, Citizen Connect)– Social media
All consumers demand quality over quantity and frequency, since no sensor is perfect.
![Page 47: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/47.jpg)
Model Consolidation
• Objectives• Make it simple to consume (traffic count) data*; easily provide content that is needed
commonly by applications• Enable entitled users who want to see details to access them• Ease the process to import, process and manage data for IT staff
• Approach
© 2010 IBM Corporation
• Approach1. Identify requirements from relevant consumers, negotiate2. (Domain knowledge): know what information typical consumers (traffic and other domains) want3. Look at sample data4. Select attributes commonly of interest5. Have additional attributes for data quality, provenance and cost, as applicable6. Validate selected attributes with consumers and implementers7. Reconcile with applicable standards (NIEM, CAP, TMDD, Datex2) and enhance description8. Sign-off: Freeze for development and create documentation
* Set expectation for data quality and error checking
![Page 48: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/48.jpg)
Common Model
Standards Aligned,Uniform format, Uniform Error Semantics
A Snapshot of Common Model and Mapping to Data Sources
Source Models
© 2010 IBM Corporation
Mapping to Source
DataTransformation
Data Source Metadata
![Page 49: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/49.jpg)
References
� Handbook reference for : inductive loop detectors, magnetic sensors and detectors, video image processors, microwave radar sensors, laser radars, passive infrared and passive acoustic array sensors, and ultrasonic sensors, plus combinations of sensor technologies. http://www.fhwa.dot.gov/publications/research/operations/its/06108/
� Ingram, J.W. The Inductive Loop Vehicle Detector: Installation Acceptance Criteria and Maintenance Techniques, Report No. TL 631387, prepared by California Department of Transportation, Sacramento, CA, for Federal Highway Administration, Washington, DC. U.S. Department of Commerce, National Technical Information Service, PB-263 948, Washington, DC, March 1976.
� N. Bansal, and B. Srivastava. On Using Crowd for Measuring Traffic at Aggregate Level for Emerging Countries. IIWeb workshop, WWW 2011, Hyderabad, India, March 28, 2011.
� H. Bischof, M. Godec, C. Leistner, B. Rinner, and A. Starzacher. Autonomous Audio-Supported Learning of Visual Classifiers for Traffic Monitoring. IEEE Intell. Sys., 25(3) pg 15–23, May/June 2010.
� M. Bramberger, R. Pflugfelder, A. Maier, B. Rinner, B. Strobl, and H. Schwabach. A Smart Camera for Traffic Surveillance. WISES03 pages 12, Vienna, Austria, June 2003.
© 2010 IBM Corporation
WISES03 pages 12, Vienna, Austria, June 2003.� Y. Zhao. Mobile Phone Location Determination and Its Impact on Intelligent Transportation Systems. IEEE Trans. ITS, col.
1, NO. 1, Mar. 2000.� M. A. Quddus, W. Y. Ochieng, and R. B. Noland. Current mapmatching algorithms for transport applications: State-of-the
art and future research directions. Transportation Research, Part C, vol. 15, no. 5, pp. 312–328, Oct. 2007.� M. Prashanth, V. N. Padmanabhan, and R. Ramjee. Nericell: rich monitoring of road and traffic conditions using mobile
smartphones. Proc. ACM Sensys, Pg. 323-336, 2008� Sensor Subset Selection for Traffic Management, R. Gupta and B. Srivastava, in 14th
International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2011), Washington DC, USA, Oct 5-7, 2011.
� V. Tyagi, S. Kalyanaraman, R. Krishnapuram. Vehicular Traffic Density State Estimation Based on Cumulative Road Acoustics, IEEE Transactions on Intelligent Transportation Systems, 2011
� ``Horn-Ok-Please'', Rijurekha Sen, Bhaskaran Raman, Prashima Sharma, The Eighth International Conference on Mobile Systems, Applications, and Services, ACM MobiSys 2010, Jun 2010, San Francisco, CA
49
![Page 50: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/50.jpg)
Traffic Management and AI
Section: Interconnection
Speaker: Anand Ranganathan
AAAI 2012 Tutorial
July 2012
© 2012 IBM Corporation
Toronto, Canada
![Page 51: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/51.jpg)
Outline
1. Traffic Management Problem
2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data
3. Interconnection
1. Middleware
2. Traffic standards
© 2010 IBM Corporation
2. Traffic standards
4. Intelligence1. Path planning
1. Simple Illustration2. Path Planning for Individual Vehicles
2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors
5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots
51
![Page 52: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/52.jpg)
Challenges in Middleware
� Scalability
–processing large volumes of real-time and static data
� Ability to incorporate various kinds of specialized as well as reusable analytics–Classifiers, forecast/prediction algorithms, simulators, spatio-temporal processing, image/video processing, etc.
© 2010 IBM Corporation
processing, image/video processing, etc.
� Extensibility
–being able to add sources and data and new kinds of analyses on the data rapidly
– support different kinds of one-time and continuous queries from diverse end-users• commuters, highway patrols, public service vehicles like re-engines and ambulances, departments of transportation, urban planners, commercial vehicle operators, etc.
![Page 53: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/53.jpg)
Addressing these challenges on the IBM InfoSphere Streams platform
� Stream Processing–New data processing paradigm where large volume of real-time (sensor) data is processed on the fly, without storing in a database
–Continuous queries (and processing)• Queries are long running, while data keeps moving
© 2010 IBM Corporation
� Key Features of InfoSphere Streams–Parallel and high performance stream processing software platform –Analysis of structured and unstructured in information
![Page 54: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/54.jpg)
Data is being produced continuously in very large volumes in different domains
Stock market• Impact of weather on
securities prices
• Analyze market data at ultra-low latencies
Radio Astronomy
• Detection of transient events
Natural Systems
• Seismic monitoring
• Wildfire management
• Water management
© 2010 IBM Corporation
Transportation
• Intelligent traffic management
Law Enforcement
• Real-time multimodal surveillance
Fraud prevention
• Detecting multi-party fraud
• Real time fraud preventionManufacturing
• Process control for microchip fabrication
Health & Life Sciences
• Neonatal ICU monitoring
• Epidemic early warning system
• Remote healthcare monitoring
![Page 55: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/55.jpg)
� continuous ingestion � continuous analysis
How Streams Works
© 2010 IBM Corporation
achieve scale by
partitioning applications into components
![Page 56: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/56.jpg)
� continuous ingestion
� continuous analysis
How Streams Worksinfrastructure provides services for
scheduling analytics across h/w nodes
establishing streaming connectivity
…Transform
Filter
Annotate
© 2010 IBM Corporation
achieve scale
by partitioning applications into components
by distributing across stream-connected hardware nodes
Classify
Correlate
• where appropriate,
• elements can be ““““fused”””” together
• for lower communication latencies
![Page 57: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/57.jpg)
Application Programming
Source Adapters Sink AdaptersOperator Repository
MARIO: Automated Application Composition
© 2010 IBM Corporation
� Consumable
� Reusable set of operators
� Connectors to external static or streaming data sources and sinks
SPL: Stream processing dataflow scripting language
MARIO: Automated Application Composition
Platform Optimized Compilation
![Page 58: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/58.jpg)
SPL Language Highlights
� Intermediate language describing how a set of individual stream processing operators are connected to one another in a graph
� Each individual operator has zero or more input and output streams:–A Built-In Operator provided by the SPL language
• Filter, Aggregator, Join, Split, Barrier, –A User Defined Operator
• Written in C++ or Java
© 2010 IBM Corporation
• Written in C++ or Java• Can be written in a schema independent manner to enhance reusability
–A source or sink operator to read/write from/to external data sources
� Procedural constructs– To create distributed and parallel flow graphs
� Settings to influence placement of operators, fusion into processes, execution model, etc.
![Page 59: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/59.jpg)
A simple example
[Application]
SourceSink trace
[Typedefs]
typedef id_t Integer
typedef timestamp_t Long
[Program]
# a source stream is generated by a Source operator – in this case tuples come from an input file
stream SenSource ( id : id_t, location : Double, light : Float, temperature : Float, timestamp : timestamp_t )
:= Source( ) [ “file:///SenSource.dat” ] {}
SinkSource Aggregate Functor
SenSourceSenAggregator
SenFunctor
© 2010 IBM Corporation
:= Source( ) [ “file:///SenSource.dat” ] {}
# this intermediate stream is produced by an Aggregate operator, using the SenSource stream as input
stream SenAggregator ( id : id_t, location : Double, light : Float, temperature : Float, timestamp : timestamp_t )
:= Aggregate( SenSource <count(100),count(1)> ) [ id . location ]
{ Any(id), Any(location), Max(light), Min(temperature), Avg(timestamp) }
# this intermediate stream is produced by a functor operator
stream SenFunctor ( id: Integer, location: Double, message: String )
:= Functor( SenAggregator ) [ log(temperature,2.0)>6.0 ]
{ id, location, “Node ”+toString(id)+ “ at location ”+toString(location) }
# result management is done by a sink operator – in this case produced tuples are sent to a socket
Nil := Sink( SenFunctor ) [ “cudp://192.168.0.144:5500/” ] {}
![Page 60: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/60.jpg)
Infosphere Streams Runtime Services
Optimizing scheduler assigns operators to
processing nodes, and continually
manages resource allocation
Runs on commodity hardware – from single
node to blade centers to high performance
multi-rack clusters
© 2010 IBM Corporation
X86
Box
X86 Blade
Cell
Blade
X86 Blade
FPGA
Blade
X86
Blade
X86 Blade
X86
Blade
X86 Blade
X86
Blade
Transport
Infosphere Streams Data Fabric
Processing Element Container
Processing Element Container
Processing Element Container
Processing Element Container
Processing Element Container
![Page 61: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/61.jpg)
Infosphere Streams Runtime Services
Optimizing scheduler assigns operators to
processing nodes, and continually
manages resource allocationAdapts to changes in resources,
workload, data ratesRuns on commodity hardware – from single
node to blade centers to high performance
multi-rack clusters
© 2010 IBM Corporation
X86
Box
X86 Blade
Cell
Blade
X86 Blade
FPGA
Blade
X86
Blade
X86 Blade
X86
Blade
X86 Blade
X86
Blade
Transport
Infosphere Streams Data Fabric
Processing Element Container
Processing Element Container
Processing Element Container
Processing Element Container
Processing Element Container
Capable of exploiting specialized hardware
![Page 62: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/62.jpg)
Real-Time GPS Data Processing – Map Matching
� Inputs:• {GPS-id, Latitude, Longitude, Heading} tuples • Collection of poly-lines (the map)
� Output:• {GPS-id, Shape-id, distance} tuple, representing the line-segment that is the best match for the GPS reading
© 2010 IBM Corporation
the best match for the GPS reading• Filter based on heading and distance
� Our technique : Grid-Based Decomposition
![Page 63: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/63.jpg)
Scaling Map-Matching in InfoSphere Streams - how can it be done?
Source Map
Map
Map
Map
Map
Reduce
Map
Map
Map
(…)
Source
© 2010 IBM Corporation
Source Map
Map
Map
Map
Map (…)Reduce
(…)
(…)
(…)
Can implement a streaming version of map-reduce on the InfoSphere Streams platform
Given those operators, building and extending the application is just a matter of wiring those operators.
Source
![Page 64: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/64.jpg)
No. of Sources Map Size
(# links)
CPU % Throughput
(#GPS points
mapped)
1 200M 10 310K
2 200M 16 589K
3 200M 20 777K
4 200M 27 1003K
Results
© 2010 IBM Corporation
4 200M 27 1003K
1 500M 14 301K
2 500M 24 582K
4 500M 37 964K
1 1000M 18 294K
2 1000M 33 596K
4 1000M 53 941K
![Page 65: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/65.jpg)
Illustration: Streams for State Estimation
© 2010 IBM Corporation65
![Page 66: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/66.jpg)
Recall: Elements of Traffic State
� Traffic flow – number of vehicles that pass a certain point during a specified time interval (vehicles/hour)
� Speed – rate of motion in distance/time (mph)
� Density – number of vehicles occupying a given length of highway or lane (vehicles per mile per lane, vpmpl)
© 2010 IBM Corporation
� Spacing – the distance between successive vehicles in a traffic stream as they pass some common reference point on the vehicles
� Time headway – the time between successive vehicles in a traffic stream as they pass some common reference point on the vehicles
66
![Page 67: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/67.jpg)
Using GPS Data … Prototype for Stockholm
� Historic GPS data traces from Stockholm for the year 2008
–1500 taxis, 40 trucks– 170 million GPS points for the year–Each taxi produces a reading every 60 seconds–Trucks – every 30 seconds–Peak rate of about 1000 readings/min
© 2010 IBM Corporation
–Peak rate of about 1000 readings/min–Often replayed at a higher rate – over 100,000 readings per sec
� Road Network
–628,095 links, spread over a 80km x 80km area
![Page 68: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/68.jpg)
Overall Application Description
InteractiveStorage
GPS
Data
Streams
Real Time Transformation
Logic
Real Time Geo
Mapping
Real Time Speed & Heading Estimation
Real-time GPS data processing
Cleaning
Map-Matching
Per-Vehicle Speed Estimation
Real Time Aggregates & Statistics
Aggregated Statistics
Per-link / per-region
Stochastic Link Travel Time
© 2010 IBM Corporation
Interactive
visualization
Data
Warehouse
Offline
statistical
analysis
adapters
Web
Server
Earth
Calculation
Stochastic Path Travel Time
Calculation
Splitter
Shortest
Path 1
Shortest
Path n
User-driven computations
Travel times, Shortest Paths
![Page 69: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/69.jpg)
Matching map artifact
GPS probe
Real Time Geo Mapping & Speed Estimation
© 2010 IBM Corporation
Estimated path
Estimated speed & heading
![Page 70: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/70.jpg)
Traffic Statistics Generation
� Aggregation and Inter-Quartiles Generation
Output Stream Schema
Aggregation operation, specifying window boundaries
© 2010 IBM Corporation
specifying window boundaries and attributes to group by
Logic for computing fields in output schema
execution process
Specifies which execution container to run operation in. SPADE allows fusing multiple
operators into a single execution process
![Page 71: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/71.jpg)
User Specific Computations
� Time Dependent Shortest Paths
� Stochastic Travel Times
� Stochastic Shortest Paths
© 2010 IBM Corporation
![Page 72: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/72.jpg)
ITS Application Flow-graph (125k GPS/second)
5min Aggr.
KML Color map
Inter-quartiles (IQ) week-
© 2010 IBM Corporation
GPS source
ISO 8601 to timestamp
Time of day, day of week
month of year
Filter taxis and invalid
values
GeoMatch
GpsTrack
Clean Sort Adapt
Travel times
JoinQuery
DSPends
IQ
IQ
Sink
Current Traffic
Statistics
Sink
![Page 73: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/73.jpg)
Showing Placement of Operators on 4 nodes
© 2010 IBM Corporation
![Page 74: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/74.jpg)
Scaling of throughput as we add more nodes
© 2010 IBM Corporation
![Page 75: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/75.jpg)
Features of InfoSphere Streams / SPADE that helps development
� Component Based Programming Model
–Applications developed as a graph of reusable operators–Operators can be built-in or user defined
� Modular Application Development
–Applications can import and export streams–Allows composing entire applications
© 2010 IBM Corporation
–Allows composing entire applications
� Declarative Application Configuration within SPADE
–Degree of parallelization–Fusion of operators–Placement of operators onto nodes
� Rich toolkits of available Operators
–Relational, Geo-Spatial, Stream Mining,…
![Page 76: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/76.jpg)
References
� Alain Biem, Eric Bouillet, Hanhua Feng, Haris Koutsopoulos, Carlos Moran, Anand Ranganathan, Anton Riabov, Olivier Verscheure, IBM Infosphere Streams for Scalable, Real-Time, Intelligent Transportation Services, In ACM International Conference on Management of Data (SIGMOD 2010), June 6-11, 2010, Indianapolis, IN
� Eric Bouillet, Anand Ranganathan, Scalable, Real-time Map-Matching using IBM’s System S, In 11th International Conference on Mobile Data Management (MDM 2010),
© 2010 IBM Corporation
System S, In 11th International Conference on Mobile Data Management (MDM 2010), Kansas City, MO, May 23- 26, 2010
76
![Page 77: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/77.jpg)
Traffic Management and AI
Section: Useful Standards (or Commonly Used)
Speaker: Biplav Srivastava
AAAI 2012 Tutorial
July 2011
© 2012 IBM Corporation
Toronto, Canada
![Page 78: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/78.jpg)
Outline
1. Traffic Management Problem
2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data
3. Interconnection
1. Middleware 2. Traffic standards
© 2010 IBM Corporation
2. Traffic standards
4. Intelligence1. Path planning
1. Simple Illustration2. Path Planning for Individual Vehicles
2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors
5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots
78
![Page 79: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/79.jpg)
Outline
� Motivation: Why we are talking standards
� Overview of immediately relevant standards
� Recommendations on standards
© 2010 IBM Corporation
![Page 80: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/80.jpg)
Motivation: Assembling the Smarter City … today’s cities
The data/ information, services, structures and activities in cities are described in different ways by the agencies which manage them, by the denizens which use them and by companies which
provide them
The data/ information, services, structures and activities in cities are described in different ways by the agencies which manage them, by the denizens which use them and by companies which
provide them
This cacophony of views and understanding drive many of the inefficiencies, create uneconomic costs in cities and limit the This cacophony of views and understanding drive many of the inefficiencies, create uneconomic costs in cities and limit the
© 2010 IBM Corporation
inefficiencies, create uneconomic costs in cities and limit the growth of city quality of life
inefficiencies, create uneconomic costs in cities and limit the growth of city quality of life
Research efforts ongoing to provide structures for the data, and the city infrastructure and services which reduce this confusion, significantly reduce costs and
improve life in the city. Example: SCRIBE (IBM).
Research efforts ongoing to provide structures for the data, and the city infrastructure and services which reduce this confusion, significantly reduce costs and
improve life in the city. Example: SCRIBE (IBM).
80
![Page 81: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/81.jpg)
Motivation: Traffic
� All parts of a city affect each other – none can work in isolation–Sample: Traffic incidents may affect public safety, water management, environment monitoring.
–Sample: External events may affect traffic, like a broken water pipe.
� Issue: how do participants share common information about the domain
© 2010 IBM Corporation
� Issue: how do participants share common information about the domain unambiguously? –E.g., congestion by traffic personnel , fire dept, electrical utility, mayor office, citizens, civil contractors, IT companies implementing Intelligent Transportation Systems
–How are concepts related to the data actually being collected. E.g., congestionto (motorized) traffic count.
![Page 82: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/82.jpg)
Motivation: Reducing every agency’s software to understandingat most 2 representations, not n representations
Common Data Model
(supporting RDF/XML
orSQL)
Police
Applications
Police
Formats
Traffic
Applications
Traffic
Formats
Fire DeptFire WaterWater
© 2010 IBM Corporation82
orSQL)Fire Dept
Applications
Fire
Formats
Water
ApplicationsFormats
Traffic
Applications
Traffic
Formats
Building
Applications
Bldg
Formats
Recreation
Applications
Recr’tn
Formats Align with select standards
(put annotations, support data formats)
![Page 83: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/83.jpg)
Sample Transportation StandardsStandard Examples of supported
concepts
Current deployments
Advanced Traveler Information Systems (ATIS), SAE 2354
Messages defined for traveler information, trip guidance, parking, Mayday (emergency information)
International standard (SAE) with traction in North AmericaGainingmomentum in the US; Nebraska, Washington and Minnesota planning support. See Washington State Department of Transportation Advanced Traveler Information Systems Business Plan for details.
DATEX II Traffic elements, operator actions, impacts, non-road event information, elaborated data, and measured data
European standardVersion I deployed in several European countries. DATEX II deployment appears to be gaining momentum in several European countries. See DATEX Deployments for details.
IEEE 1512 Common incident management message sets for use by emergency management centers, traffic management, public safety, hazardous materials, and entities external to centers
International standardEarly deployments include Washington D.C., NYC, and Milwaukee. See 1512 Public Safety Early Deployment Projects for details.
Source: Smarter city data model standards landscape, Part 2: Transportation
© 2010 IBM Corporation
external to centers
National Transportation Communication for ITS Protocol (NTCIP) 1200 Series
Currently thirteen data dictionaries defined, including object definitions for: dynamic message signs, CCTV camera control, ramp meter control, transportation sensor systems
U.S. specific standardAccording to NTCIP web site, several cities and states in the U.S. have NTCIP projects underway. See NTCIP Deployment: Projects for details.
Service Interface for Real Time Information (SIRI)
Information exchange of real-time information about public transportation services and vehicles
European specific standardBased on best practises of various national and proprietary standards from across Europe.
Traffic Management Data Dictionary (TMDD)
OwnerCenter, ExternalCenter, Device, DateTime, Dynamic Message Sign, Event, Generic, Organization, and RoadNetwork
International standard (ITE and AASHTO) with traction in North AmericaIn early stages of deployment in the US. TMDD is backed by US DoT and multiple vendors (Transcom, Siemens). Multiple state DoTs are planning to move to TMDD in their next rev (including Florida and Utah). See "A Report to the ITS Standards Community ITS Standards Testing Program, For TMDD and Related Standards as Deployed by the Utah Department of Transportation" for TMDD testing details from the Utah DoT.
![Page 84: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/84.jpg)
Relevant Standards
� Information exchanged – Transportation and external agencies: Datex 2– Between agencies: NIEM
� Format of information given: CAP
� Public Transportation: GTFS
© 2010 IBM Corporation
84
� Public Transportation: GTFS
� In addendum: Road Conditions– Road Surface Conditions– Road types– Types of restriction
![Page 85: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/85.jpg)
Information Exchanged
Information exchanged with DATEX II systems is composed of different basic elements:
• Road and traffic related events (called “Traffic elements”)• Operator actions• Advice
– Different types of advice: speed, lane usage, winter driving …
• Impacts• Non road event information
© 2010 IBM Corporation
• Non road event information– It concerns information about events which are not directly on the road: transit information, service disruption, car parks.
• Elaborated data (derived/computed data, e.g. travel times, traffic status)• Measured data (direct measurement data from equipments or outstations, e.g. traffic and weather measurements)
85
Datex 2Version 1.0
![Page 86: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/86.jpg)
Road Events (Elements)
Road and traffic related events (called “Traffic elements”)These are all events which are not initiated by the traffic operator and force him to undertake(re)actions. They are classified in 6 main categories:• Abnormal traffic (long queues, stop and go, …)• Accidents - are events in which one or more vehicles lose control and do not recover. They include collisions between vehicle(s) or other road user(s), between vehicle(s) and any obstacle(s), or they result from a vehicle running off the road. Details can be given:
– cause type,– accident type,– overview of people involved (number, injury status, involvement role, type),– overview of vehicles involved per type (number, status, type, usage),
© 2010 IBM Corporation
– overview of vehicles involved per type (number, status, type, usage),– details of involved vehicles (type + individual vehicle characteristics)
• Obstructions :– animal presence,– vehicles presence,– obstructions due to environment (avalanches, flooding, fallen trees, rock falls, …),– obstructions due to equipments (fallen power cables, …)
• Activities• Incidents on infrastructure equipments (Variable message sign out of order, tunnel ventilation not working, emergency telephone not working, …)
• Specific conditions : road conditions related to weather (ice, snow, … ) or not (oil, …), environment conditions (precipitation, wind, …), …
86
Datex 2Version 1.0
![Page 87: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/87.jpg)
Operator actions
Operator actions are classified in 4 main categories:
� Network management :road closure, alternate traffic, contra flow …– traffic control : rerouting, temporary limits
� Roadworks: resurfacing, salting, grass cutting …
© 2010 IBM Corporation
� Roadside assistance: vehicle repair, helicopter rescue, food delivery …
� Sign settings: variable message signs, matrix signs.
87
Datex 2Version 1.0
![Page 88: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/88.jpg)
Impact
� Delays and traffic status (Impact) have 5 possible values:
– free flow,– heavy,– congested,– impossible,
© 2010 IBM Corporation
– impossible,– Unknown
88
Datex 2Version 1.0
![Page 89: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/89.jpg)
Elaborated Data - These sets of data are normally derived on a periodic basis by the Traffic Centre from input data for specified locations
� Traffic values (normally published as measured data, but can be derived on a periodic basis and published as elaborated data):
– flow, – speed, – headway, – concentration and – individual vehicle measurements.
© 2010 IBM Corporation
– individual vehicle measurements.
� Weather values (normally published as measured data, but can be derived on a periodic basis and published as elaborated data):
– precipitation, – wind, – temperature, – pollution, – Road surface condition and – visibility.
89
Datex 2 Version 1.0
![Page 90: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/90.jpg)
Measured Data - These data sets are normally derived from direct inputs from outstations or equipments at specific measurement sites (e.g. loop detection sites or weather stations) which are received on a regular (normally frequent) basis
� traffic values: – flow, speed, headway, concentration and individual vehicle measurements.
� weather values: – precipitation, wind, temperature, pollution, road surface condition and visibility.
� travel times: – elaborated time, free flow time, normally expected time
© 2010 IBM Corporation
� traffic status: – free flow, heavy, congested, impossible, Unknown
90
Datex 2Version 1.0
![Page 91: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/91.jpg)
National Information Exchange Model: Summary
� “NIEM provides a common vocabulary for consistent, repeatable exchanges of information between agencies and domains. The model is represented in a number of forms, including a data dictionary and a reference schema, and includes the body of concepts and rules that underlie its structure, maintain its consistency, and govern its use.
� NIEM:– Is a foundation for information exchange.– Offers a common vocabulary so that when two or more people talk to each
other they can exchange information based on common words that they both understand.
– Is community-driven.– Provides a data model, governance, methodologies, training, technical
© 2010 IBM Corporation
– Provides a data model, governance, methodologies, training, technical assistance, and an active community to assist users in adopting a standards-based approach to exchanging information.
– Provides technical tools to support development, discovery, dissemination, and reuse of information exchanges.
– Provides a forum for accelerating collaboration as well as identifying common approaches and challenges to exchanging information.
� NIEM is Not:– The actual exchange of information. NIEM describes the data that is in
motion in the exchange.– A database or system. NIEM does not store information.– Intrusive to existing systems. NIEM allows organizations to move information
quickly and effectively without rebuilding systems.– Software.– A technology stack. NIEM is technology agnostic and addresses the data
layer, which means you can use NIEM irrespective of the particular technologies used within an organization.
– Just for law enforcement and justice. Fourteen domains participate in NIEM with additional domains forming.
– Strictly for federal government. NIEM is used in all 50 states, as well as local and tribal governments and private industry.
– Limited by national borders. NIEM is used internationally.”
Source: NIEM Website
![Page 92: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/92.jpg)
Common Alerting Protocol (CAP)
� “The Common Alerting Protocol (CAP) is an XML-based data format for exchanging public warnings and emergencies between alerting technologies.– Alerts from the United States Geological Survey, the Department of Homeland Security, NOAA and
<?xml version = "1.0" encoding = "UTF-8"?><alert xmlns = "urn:oasis:names:tc:emergency:cap:1.2"><identifier>43b080713727</identifier><sender>[email protected]</sender><sent>2003-04-02T14:39:01-05:00</sent><status>Actual</status><msgType>Alert</msgType><scope>Public</scope> <info><category>Security</category><event>Homeland Security Advisory System Update</event><urgency>Immediate</urgency><severity>Severe</severity><certainty>Likely</certainty><senderName>U.S. Government, Department of Homeland Security</senderName>
© 2010 IBM Corporation
of Homeland Security, NOAA and the California Office of Emergency Services can all be received in the same format, by the same application.
– The CAP data structure is backward-compatible with existing alert formats including the Specific Area Message Encoding(SAME) used in Weatheradio and the broadcast Emergency Alert System as well as new technology such as theCommercial Mobile Alert System (CMAS)”.
Acknowledgement: Wiki - http://en.wikipedia.org/wiki/Common_Alerting_Protocol
<senderName>U.S. Government, Department of Homeland Security</senderName><headline>Homeland Security Sets Code ORANGE</headline><description>The Department of Homeland Security has elevated the Homeland Security
Advisory System threat level to ORANGE / High in response to intelligence which may indicate a heightened threat of terrorism.</description><instruction> A High Condition is declared when there is a high risk of terrorist attacks. In
addition to the Protective Measures taken in the previous Threat Conditions, Federal departments and agencies should consider agency-specific Protective Measures in accordance with their existing plans.</instruction><web>http://www.dhs.gov/dhspublic/display?theme=29</web><parameter><valueName>HSAS</valueName><value>ORANGE</value></parameter><resource><resourceDesc>Image file (GIF)</resourceDesc><mimeType>image/gif</mimeType><uri>http://www.dhs.gov/dhspublic/getAdvisoryImage</uri></resource><area><areaDesc>U.S. nationwide and interests worldwide</areaDesc></area></info></alert>
![Page 93: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/93.jpg)
Public Transportation: General Transit Feed Specification (GTFS)
� Proposed by Google– Started in 2006– 2009: moves from Google-specific to general– 2011: supports real-time update
� Purpose: public transportation schedules and associated geographic information
� Format is useful even if the data is not shared with Google
© 2010 IBM Corporation93
Source: https://developers.google.com/transit/gtfs/examples/gtfs-feed Source: Making Public Transportation Schedule Information Consumable for Improved Decision
Making, Raj Gupta, Biplav Srivastava, Srikanth Tamilselvam, In 15th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2012), Anchorage, USA
![Page 94: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/94.jpg)
General Transit Feed Specification (GTFS)
Filename Required Defines
agency.txt Required One or more transit agencies that provide the data in this feed.
stops.txt Required Individual locations where vehicles pick up or drop off passengers.
routes.txt Required Transit routes. A route is a group of trips that are displayed to riders as a single service.
trips.txt Required Trips for each route. A trip is a sequence of two or more stops that occurs at specific time.
stop_times.txt Required Times that a vehicle arrives at and departs from individual stops for each trip.
calendar.txt Required Dates for service IDs using a weekly schedule. Specify when service starts
© 2010 IBM Corporation94
and ends, as well as days of the week where service is available.
calendar_dates.txt
Optional Exceptions for the service IDs defined in the calendar.txt file. If calendar_dates.txt includes ALL dates of service, this file may be specified instead of calendar.txt.
fare_attributes.txt Optional Fare information for a transit organization's routes.
fare_rules.txt Optional Rules for applying fare information for a transit organization's routes.
shapes.txt Optional Rules for drawing lines on a map to represent a transit organization's routes.
frequencies.txt Optional Headway (time between trips) for routes with variable frequency of service.
transfers.txt Optional Rules for making connections at transfer points between routes.
feed_info.txt Optional Additional information about the feed itself, including publisher, version, and expiration information.
Source: https://developers.google.com/transit/gtfs/
![Page 95: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/95.jpg)
Recommendation on Standards
� DATEX2–Use terms from Datex 2
• Impact, Measured Data, Elaborated Data are directly applicable• Evaluate others
–Use traffic state from Impact • Can be rule based using flow and/or speed.
� NIEM
© 2010 IBM Corporation
� NIEM–Align time, data, organizations, address, …, based on NIEM, if in North America
–Consider adopting NIEM in other regions, depending on relevance• Address needs localization
� CAP–Support CAP format as actual data exchange
� GTFS–Adopt as much as relevant
![Page 96: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/96.jpg)
References
� Smarter city data model standards landscape, Part 2: Transportation– http://www.ibm.com/developerworks/industry/library/ind-smartercitydatamodel2/
� CAP: http://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.html
� Datex 2: www.datex2.eu/
© 2010 IBM Corporation
� NIEM: www.niem.gov
�GTFS: https://developers.google.com/transit/gtfs/
� SCRIBE: http://researcher.watson.ibm.com/researcher/view_project.php?id=2505
![Page 97: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/97.jpg)
Addendum: Road Conditions
© 2010 IBM Corporation
97
![Page 98: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/98.jpg)
Road Surface Conditions
� Surface Condition� Boggy Conditions
An unsealed road surface that is wet and soft, which may lead to a vehicle becoming bogged. Can also apply to loose dry conditions.� Bulldust
A very fine material with a flour like texture often covering surface irregularities or deep holes.� Changing Surface Conditions
The condition of the road surface may change along the length of the road, and following road maintenance activities. Wet weather may also cause a localised change in road condition.
� CorrugationsAn unsealed road surface having ripples or undulations along the road.
� FloodwayA localised section of road designed to accommodate temporary overtopping allowing water to flow over the surface of the road. Refer also “Stream Crossing”.
� Loose SurfaceA layer of unbound or coarse material on the pavement surface. Can be very fine material (bull dust or sand), or large material in
© 2010 IBM Corporation
A layer of unbound or coarse material on the pavement surface. Can be very fine material (bull dust or sand), or large material in coarse gravel pavements.
� PotholesBowl-shaped depressions in the road surface, resulting from the loss of pavement material.
� RoughThe consequence of irregularities, uneven in nature, in the longitudinal profile of a road with respect to the intended profile.Interpretation of conditions will not allow vehicles to use the road at customary speeds.
� RuttingA longitudinal deformation of a pavement surface formed by the wheels of vehicles. May be on a sealed or unsealed road.
� Slippery SurfaceSurface becomes slippery in wet conditions or from fuel or material spills.
� Stream CrossingNatural watercourses that cross the road alignment. On sealed roads the crossing would generally be a bridge, floodway or culvert. On unsealed roads there may be no structure.Water levels at floodways and stream crossings are likely to change rapidly and may present an extreme hazard. The ability to cross these floodways and streams will depend on depth of water, velocity of flow, driver experience and vehicle type. ALWAYS assess conditions prior to crossing or travelling along the affected section. See also “Water over Road”.
� WashoutsSteep, irregularly sided grooves in the road surface caused by erosion of the road surface by water.
�
98http://www.ntlis.nt.gov.au/roadreport/include.jsp?pageName=term_files/index
![Page 99: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/99.jpg)
Road Types
� Road Types
� FormedAn unsealed road that has been constructed to above the natural surface using local materials. Generally of a higher standard than unformed.
� GravelledAn unsealed road that has been formed and strengthened with a good quality gravel material. Generally of a higher standard than formed.
� SealedA road that is constructed with a bitumen surface.
� Unformed
© 2010 IBM Corporation
� UnformedAn unsealed road that is generally a flat track following the natural terrain. Often occurs as a rough track with two wheel paths, and close vegetation.
� UnsealedA road without a bituminous surface that could be gravelled, formed or unformed.
99
http://www.ntlis.nt.gov.au/roadreport/include.jsp?pageName=term_files/index
![Page 100: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/100.jpg)
Types of Restrictions
� DetourAn alternative route available for traffic during a temporary closure of a road, or part of that road. Detours may be sealed or unsealed.
� High ClearanceA light vehicle with clearance sufficient to travel over rough and uneven road surfaces. The clearance on these vehicles generally exceeds common passenger vehicles.
� High Clearance 4WDA light vehicle with two axles that is built for on and off road use and clearance sufficient to travel over rough and uneven road surfaces. The clearance on these vehicles generally exceeds common passenger vehicles.
� Impassable
© 2010 IBM Corporation
� ImpassableAccess along the section of road is affected by flooding or other obstructions. Road conditions are likely to change rapidly and may present an extreme hazard. Persons should not attempt to use/access the road.
� Lane ClosureTemporary closure of one or more traffic lanes. Traffic control devices are in place -to enable traffic to use unaffected lanes.
� Road ClosedTemporary closure of the road where passage of motor vehicles is not permitted. Barriers are in place and penalties shall be applied to drivers ignoring the road closure.
� Weight and Vehicle Type RestrictionA restriction placed on heavy vehicles to control axle group loads and/or vehicle size to protect the road during adverse conditions.
� With CautionThe condition of part of the road may have deteriorated so that due care must be exercised by the travelling public.
� 4WDA light vehicle with two axles that is built for on and off road use. All four wheels can be engaged to improve traction in adverse conditions.
100http://www.ntlis.nt.gov.au/roadreport/include.jsp?pageName=term_files/index
![Page 101: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/101.jpg)
Traffic Management and AI
Section: Intelligence
Speakers: Anand Ranganathan, Biplav Srivastava
AAAI 2012 Tutorial
July 2012
© 2012 IBM Corporation
Toronto, Canada
![Page 102: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/102.jpg)
Outline
1. Traffic Management Problem
2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data
3. Interconnection1. Middleware 2. Traffic standards
© 2010 IBM Corporation
2. Traffic standards
4. Intelligence
1. Path planning
1. Simple Illustration
2. Path Planning for Individual Vehicles
2. End-user analytics
1. Bus arrival prediction and journey planning, with state-of-art instrumentation
2. Multi-modal journey planning, without sensors
5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots
102
![Page 103: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/103.jpg)
Simple Illustration
© 2010 IBM Corporation103
![Page 104: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/104.jpg)
Description� City’s road network is a 5x5 grid
� Travelers in the city:– A, B, C, and D live as shown on the map– Their workplace is W [2,2]
� Objective:
1.We want to help all the travelers plan their trips to work in the morning, and back home daily using shortest time.
2. If the office can be moved, where should it be moved to reduce the commute time for all employees?
Commute constraints– Speed of all travelers, by default, is 1 hop per 10 minutes.
– Speed is reduced by half if sun on the face while commuting. Hence, in the morning, travelers going east are slowed by half, and in the evening, travelers going west are slowed by half. Some examples:
• Time taken from [0,0] to [0,1] is 20 minutes in the morning, and 10 minutes for the rest of the day
© 2010 IBM Corporation
for the rest of the day• Time taken from [0,1] to [0,0] is 20
minutes in the evening, and 10 minutes for the rest of the day
• Time taken from [0,0] to [ 1,0] remains 10 minutes throughout the day
– If two persons travel on the same road, their speeds reduce by half.
– The two slowing factors work independently. An example is that two vehicles going from [0,0] to [0,1] in the morning will take 40 minutes
Loosely Synchronized Multi-Modal Plans for Traffic Improvement and Commuter Convenience, Biplav Srivastava, Raj Gupta and Nirmit Desai, IBM Research Report 2012
![Page 105: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/105.jpg)
[4,4][4,0]
[2,2]
C (Chumki)D (Deb)
W (Workplace)
D (Car2) time- 60
C (Car2) time- 40
[Coordinated Planning] Total time for all: 200 minutes
© 2010 IBM Corporation
[0,0] [0,4]
A (Alice) B (Bob)
W (Workplace)
A (Car0) time- 60B (Car1) time- 40
![Page 106: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/106.jpg)
[4,4][4,0]
[2,2]
C (Chumki)D (Deb)
W (Workplace)
D (Car2) time- 80
C (Car2) time- 50
[Non Coordinated Planning] Total time for all: 260 minutes
© 2010 IBM Corporation
[0,0] [0,4]
A (Alice) B (Bob)
W (Workplace)
A (Car0) time- 80B (Car1) time- 50
East
North
![Page 107: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/107.jpg)
Path Planning for Individual Vehicles
© 2010 IBM Corporation107
![Page 108: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/108.jpg)
Traffic - Dependent Route Planning
� Definition: Find a path from a source to a destination on a road network with minimum travel time delay and that considers current and future traffic conditions on all road links.
� Needs dynamic time-dependent shortest path computations–Cost of each link is different at different intervals of time, and
© 2010 IBM Corporation
–These time-dependent costs get updated as new traffic information is obtained.
� Challenges related to performance–Need high throughput and low latency
• 100s of users, need response in less than a second
–Scale well as we increase size of road network• 10,000s or 100,000s nodes and links
–Handle real-time updates to road network conditions
108 7/22/2012
![Page 109: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/109.jpg)
Dynamic Time-Dependent Shortest Path
� Previous works have focused either on dynamism or on time-dependency, but not both
� A* adapted for time-dependent shortest path networks (Chabini and Lan)–Assumes First In – First Out rule on each link
• A traveler who travels on a link will reach the end of the link before anyone who departs later from the beginning of the link.
© 2010 IBM Corporation
� Our approach : Modify A* as follows :–As paths are extended with links during the execution of A*, time is advanced and therefore future delay values of links are used as link costs.
– To ensure the FIFO property in a time-dependent network,• If the time interval changes while traveling on a link, new delay value used for the rest of the link.
• A link can be travelled during many time intervals–Update link costs with new traffic information –Use heuristic function as (Euclidean Distance / max speed limit)
109 7/22/2012
![Page 110: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/110.jpg)
Implementing Shortest Path as a Stream Processing Application
� Shortest Path Algorithm implemented as a C++ user defined Operator
� One time input to Operator :–Road Network
� Continuous Updates to Operator:
© 2010 IBM Corporation
� Continuous Updates to Operator:–Stream with Delays on different links in road network–Stream with Queries (containing Origin, Destination, Departure Time)
� Result : Stream containing Links in Shortest Path and Expected Travel Time
� Can Parallelize effectively–By Destination Point–Allows scaling and caching of heuristic function
110 7/22/2012
![Page 111: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/111.jpg)
© 2010 IBM Corporation111 7/22/2012
![Page 112: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/112.jpg)
Experiments on Shortest Path
� Updates to Traffic Conditions– Generated continuously as GPS data is streamed in
– Divided day into 5 minute intervals
– 61422 updates on 11177 links in 1 day
© 2010 IBM Corporation112 7/22/2012
![Page 113: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/113.jpg)
Performance Experiments
� Measured delay and throughput on 10, 20 and 50 machines when there is a burst of 10,000 shortest path queries
� Scales almost linearly as number of machines increases
© 2010 IBM Corporation113 7/22/2012
![Page 114: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/114.jpg)
Evaluating the benefits of doing dynamic time-dependent shortest paths – quantifying the time savings
� How much do we gain from using traffic data in the shortest path calculation?– Compared to path obtained using static speed limits
� Ran shortest path algorithm for 50 different departure times for 500 origin-destination pairs
� Upto 62% decrease in travel time
© 2010 IBM Corporation114 7/22/2012
![Page 115: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/115.jpg)
Number of changes in Shortest Path
� Shortest Path for a certain origin-destination pair changed upto 37 times (out of the the 50 consecutive runs)–30 changes for more than half of the queries
© 2010 IBM Corporation115 7/22/2012
![Page 116: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/116.jpg)
Degree of change of shortest path
� Degree of change = (1 - (number of common links / total number of links in two paths) )*100
� Paths can change by upto 98% between consecutive runs
© 2010 IBM Corporation116 7/22/2012
![Page 117: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/117.jpg)
Visualizing the paths for origin-destination pair with max number of changes
© 2010 IBM Corporation117 7/22/2012
![Page 118: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/118.jpg)
End User Analytics
© 2010 IBM Corporation118
![Page 119: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/119.jpg)
Bus Arrival Prediction and Travel Planner
Business Challenge
�Meanwhile, public transportation represents a major cost for governments and is one of the most visible services available to citizens.
� However, cities mostly lack the tools, information and visibility to understand and optimize how public transportation is performing
© 2010 IBM Corporation
understand and optimize how public transportation is performing and meeting citizen’s needs.
![Page 120: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/120.jpg)
IBM’s Public Transportation Awareness tool
� Continuously analyses data from Automatic Vehicle Location (AVL) systems
� Collects, processes, and visualizes location data of all public transportation vehicles
� Analytic algorithms are optimized for high performance, on-demand access
� Automatically generated transportation routes and stop locations
© 2010 IBM Corporation120
![Page 121: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/121.jpg)
Map & Dashboard
© 2010 IBM Corporation
![Page 122: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/122.jpg)
Bus Delay Overview
© 2010 IBM Corporation
![Page 123: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/123.jpg)
Bus Delay Query http://www.youtube.com/watch?v=VBv5XRGQ7nA
© 2010 IBM Corporation
![Page 124: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/124.jpg)
Bus Stop, Bus Route Queries
© 2010 IBM Corporation
![Page 125: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/125.jpg)
Bus Arrival Prediction
© 2010 IBM Corporation
![Page 126: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/126.jpg)
Dublin Experience
© 2010 IBM Corporation
![Page 127: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/127.jpg)
Dublin Experience – Deployed Dublin Bus RTPI
150 bus routes / 5000 bus stops
600 buses contemporary on routes during the day
20 stops per bus
20*600 = 12,000 predictors per min
© 2010 IBM Corporation
![Page 128: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/128.jpg)
Use Case and Performance
� Deployed in a production environment at the Dublin City Council since Jan 2011
� Run on VMWare Linux RHEL 5.4, 4 cores Intel Xeon© 2.27GHz, 6GB RAM
� Has been running on the average 2-3 months between
© 2010 IBM Corporation
1
� Has been running on the average 2-3 months between updates
� Typically 1 – 3 user sessions running simultaneously between DCC and IBM
![Page 129: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/129.jpg)
Technical Challenges
1. Data diversity, heterogeneity
2. Data accuracy, sparsity
3. Data volume
Routes & maps
Parkingcapacity
SCATS
Induction loop
TimetablesCar
700 intersections
4,000 loop detectors
20,000 tuples / min
© 2010 IBM Corporation
Bus AVL (GPS)
Accessibility
CCTV
Bike
1,000 buses
3,000 GPS / min
200 CCTV cameras
![Page 130: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/130.jpg)
SIRI
VehicleMonitoring
Adapter
GIS
Map bus to route
Bus to route
tracking
Bus KPI: Average bus speed, traffic status, bus outside routes …
Bus to link spatiotemporal aggregations
Link KPI (speed, …)
LinkUpdateLinkUpdate
TMDD
Adapter
Real time Bus Information adapter
PTA Design – Streams Flow Graph
© 2010 IBM Corporation
to route tracking bus outside routes … aggregationsAdapterAdapter
Bus Estimated Arrival Times at Stops
SIRI StopMonitoring
Adapter(*)
(*) Returns StopMonitoring info for all stops
![Page 131: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/131.jpg)
Data
� GPS readings once every 20 seconds, with:
� Time of capture, GPS location, Bus line ID, Bus vehicle ID– Delay information => Calculated by Dublin bus with respect to timetables– Delay information embedded on the GPS readings and available for processing
� Approx 19 million GPS readings per month
© 2010 IBM Corporation
� Also, GPS coordinates of Bus Stops
131
![Page 132: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/132.jpg)
Pre-Processing
� Detect arrival/departure buses– Radius of 20m around the bus stop and track a bus from a bus line.
� Modeling of delays at every bus stop
� Recovering timetables
� Modeling of delay correlation between delay at departure bus stop and arrival bus stop
© 2010 IBM Corporation
� Modeling of delay correlation between delay at departure bus stop and arrival bus stop
� Calculate Travel Times
132
![Page 133: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/133.jpg)
Modeling delays at bus stops
� Delay: bus arrival/departure earlier/later than the expected on the timetables
� Detect when the bus is at a bus stop (within a 20m radius)
� Calculate Histograms– Per hour of the day, – Weekdays, Weekends
� Use of Kernel Density Estimators
© 2010 IBM Corporation133
![Page 134: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/134.jpg)
Modeling delays at bus stops
© 2010 IBM Corporation134
![Page 135: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/135.jpg)
Recovering timetables
© 2010 IBM Corporation135
![Page 136: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/136.jpg)
End to end trip times
� Estimate distributions of travel times from A to B on the network
� As a function of:
© 2010 IBM Corporation
� As a function of:
� Buses early/late/missing connections => how the trip changes and probabilities
136
![Page 137: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/137.jpg)
Scenario and Simulation
© 2010 IBM Corporation137
![Page 138: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/138.jpg)
Results
© 2010 IBM Corporation
� Expected value differs by 10 minutes of the deterministic trip time
� Variance is also large
� Travel time distribution mean and variance is function of the time of the day138
![Page 139: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/139.jpg)
Multi-modal journey planning, without sensors
Based on joint work by Raj Gupta, Srikanth
© 2010 IBM Corporation139
Based on joint work by Raj Gupta, Srikanth Tamilselvam, Biplav Srivastava
Making Public Transportation Schedule Information Consumable for Improved Decision Making, Raj Gupta, Biplav Srivastava, Srikanth Tamilselvam, In 15th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2012), Anchorage, USA, Sep 16-19, 2012.
![Page 140: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/140.jpg)
Problem
� Input: – A person wants to travel from place A to B
� Invariant Inputs: – Mode and mode categories for commuting
• The city has taxis, buses, metros, autos, rickshaws• Buses and metros have published routes, frequency and stops. No other instrumentation.• Autos and rickshaws can be available at stands, or opportunistically, on the road• Taxis can be ordered over the phone
© 2010 IBM Corporation
– Preferences over commuting modes and categories• The person can have a vehicle (e.g., car), • Can walk short distances• Has preferences over commuting modes based on cost, time, safety, physical effort.
� Output– Suggest to the person which mode or combination of modes to select
�Observation: there is no coordination assumed from the transport operators
![Page 141: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/141.jpg)
Multi-Mode Commuting Recommender in Delhi And Bangalore, India
© 2010 IBM Corporation141
Highlights
• Published data of multiple authorities used; repeatable process
•Multiple modes searched
• Preference over modes, time, hops and number of choices supported; more extensions, like fare possible
![Page 142: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/142.jpg)
Technical Challenges in Data Processing
� Factors which affect/ can improve accuracy– Quality of schedule published by public transportation operators (bus and metro for us)
• Names, spelling and conventions in stops by different agencies• We correct and can do more - If we correct too much, we remove the traceability to original published schedules
– Lack of co-relationship across stop names and location• Affects what the user sees when they select• We can include geo-spatial analysis when we offer choice of
locations
– Increase inter-operability across agencies
© 2010 IBM Corporation
– Increase inter-operability across agencies• Make traffic data into linked open data format
– Integrate with geo-spatial analysis of tools
142
![Page 143: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/143.jpg)
References
� Ugur Demiryurek, Farnoush Banaei-Kashani, Cyrus Shahabi and Anand Ranganathan. Online Computation of Fastest Path in Time-Dependent Spatial Networks. In 12th International Symposium on Spatial and Temporal Databases (SSTD 2011), Aug 24-26, 2011, Minneapolis, MN
� Baris Gu�c, Anand Ranganathan. Real-Time, Scalable Route Planning Using a Stream-Processing Infrastructure. In IEEE Intelligent Transportation Systems Conference, Sept 20-22, 2010, Madeira, Portugal
� A. Botea, M. Mueller, and J. Schaeffer. Near optimal hierarchical path-finding. Journal of Game Development, 1:7–28, 2004.
� I. Chabini. Discrete dynamic shortest path problems in transportation applications: Complexity and algorithms with optimal run time.Transportation Research Records, 1645:170–175, 1998.
� I. Chabini and S. Lan. Adaptations of the A* algorithm for the computation of fastest paths in deterministic discrete-time dynamic networks. IEEE Transactions on Intelligent Transportation Systems,3(1):60–74, 2002.
� R. Dechter and J. Pearl. Generalized best-first search strategies and the optimality of A*. JOURNAL OF THE ACM, 32(3):505–536, 1985.
� B. Ding, J. X. Yu, and L. Qin. Finding time-dependent shortest paths over large graphs. In EDBT ’08: Proceedings of the 11th international conference on Extending database technology, pages 205–216, New York, NY, USA, 2008. ACM.
© 2010 IBM Corporation
conference on Extending database technology, pages 205–216, New York, NY, USA, 2008. ACM.
� S. Ganugapati. Parallel algorithms for dynamic shortest path problems. International Transactions in Operational Research, 9:279–302(24), May 2002
� E. Kanoulas, Y. Du, T. Xia, and D. Zhang. Finding fastest paths on a road network with speed patterns. In ICDE ’06: Proceedings of the 22nd International Conference on Data Engineering, page 10, Washington, DC, USA, 2006. IEEE Computer Society.
� D. Kotzinos. A massively parallel time-dependent least-time-path algorithm for intelligent transportation systems applications. Computer Aided Civil and Infrastructure Engineering, 16:337–346(10), September 2001.
� C.-C. Lee, Y.-H. Wu, and A. L. P. Chen. Continuous evaluation of fastest path queries on road networks. In SSTD, pages 20–37, 2007.
� A. Orda and R. Rom. Shortest-path and minimum-delay algorithms in networks with time-dependent edge-length. J. ACM, 37(3):607–625,1990.
� Y. Tian, K. C. K. Lee, and W.-C. Lee. Monitoring minimum cost paths on road networks. In GIS ’09: Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 217–226, New York, NY, USA, 2009.ACM.
143
![Page 144: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/144.jpg)
References
� L. Gasparini, E. Bouillet, F. Calabrese, O. Verscheure, Brendan O’Brien, Maggie O’Donnell, "System and Analytics for Continuously Assessing Transport Systems from Sparse and Noisy Observations: Case Study in Dublin", ITSC 2011
� A. Baptista, E. Bouillet, F. Calabrese, O. Verscheure, "Towards Building an Uncertainty-aware Multi-Modal Journey Planner", ITSC 2011
� A. Ozal, A. Ranganathan, N. Tatbul, "Real-time Route Planning with Stream Processing Systems: A Case Study for the City of Lucerne", ACM SIGSPATIAL International Workshop on GeoStreaming (IWGS'11), Chicago, IL, USA, November 2011
� Making Public Transportation Schedule Information Consumable for Improved Decision Making, Raj Gupta, BiplavSrivastava, Srikanth Tamilselvam, In 15th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2012), Anchorage, USA, Sep 16-19, 2012.
© 2010 IBM Corporation
� Loosely Synchronized Multi-Modal Plans for Traffic Improvement and Commuter Convenience, Biplav Srivastava, Raj Gupta and Nirmit Desai, IBM Research Report, 2012.
144
![Page 145: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/145.jpg)
Tutorial: Traffic Management and AI
Section: Supporting Topics
Biplav Srivastava, Anand RanganathanIBM Research
AAAI 2012 Tutorial
22nd July 2011
© 2012 IBM Corporation
Toronto, Canada
![Page 146: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/146.jpg)
Outline
1. Traffic Management Problem
2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data
3. Interconnection1. Middleware 2. Traffic standards
© 2010 IBM Corporation
2. Traffic standards
4. Intelligence1. Path planning
1. Simple Illustration2. Path Planning for Individual Vehicles
2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors
5. Supporting topics
1. Traffic Simulators
2. Practical considerations for real-world pilots
146
![Page 147: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/147.jpg)
Why Use Traffic Simulators
� Use them to prepare for real-world experience
� Overcome– Data issues– Try what-ifs– Cost reasons
� No simulator is a replacement for actual deployment
© 2010 IBM Corporation
� No simulator is a replacement for actual deployment
147
![Page 148: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/148.jpg)
Traffic Simulators
Availability LicensingProgramming
language
Code
Extension
Development
Started
Current
status
Macro /
micro traffic
flow
Location
Dynamit Academiamit.edu/its/dyn
amit.html
Freesim Open Source GNU java Possible 2004last release 25-7-2011
bothhttp://www.freewaysimulator.com/index.html
Sumo Open Source SourceForge C++ Possible 2002 upto date microhttp://sumo.sourceforge.net/
Microsimulation
of Road Traffic
Flow
Academia java applet PossibleLast release June 2011
microhttp://www.traffic-simulation.de/
Credit: Table compiled by Raj Gupta, IBM Research
© 2010 IBM Corporation
TraNS Academia 2007 2008http://lca.epfl.ch/projects/trans
Megaffic IBM
http://www.research.ibm.com/trl/projects/socsim/project.htm
MITSIMlab Open Source Free to use Possible 1999No support since 2005
micromit.edu/its/mits
imlab.html
TransmodelerSingle User licence
10 K micro
http://www.caliper.com/TransModeler/default.htm
Matsim Open source GNU Java Possible 2003last release 28-3-2011
macrohttp://www.matsim.org/
148
![Page 149: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/149.jpg)
Matsim
� Agent-Based Transport Simulations
� Key Features of MATSim– Fast Dynamic and Agent-Based Traffic SimulationSimulate whole days within minutes It’s not time based.
– Supports Large ScenariosMATSim can simulate millions of agents or huge, detailed networks
© 2010 IBM Corporation
MATSim can simulate millions of agents or huge, detailed networks – Versatile Analyses and Simulation OutputE.g. compare simulated data to real-world counting stations
– Modular ApproachEasily extended with your own algorithms
– Sophisticated Interactive VisualizerSee what each agent is doing during the simulation
– Open SourceYou get the Java Source Code, which runs on all major operating systems
� Easy integration with Open Street Maps
![Page 150: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/150.jpg)
Matsim: Input and Output
Matsim
Network<node id="23" x="6900" y="0"/><link id="0" from="0" to="1" length="300" capacity="60" freespeed="30" permlanes="60" />
Plan<person id="1"> <plan> <act type="h" x="0" y="0" end_time="00:18" /><leg mode="car"> </leg><act type="h" x="6900" y="0" />
© 2010 IBM Corporation
Matsim<act type="h" x="6900" y="0" /></plan>
</person>
Network_Change<networkChangeEvent startTime="02:07:00" ><link refId="0"/><freespeed type="scaleFactor" value="0.99"/>
</networkChangeEvent>
Events<event time="1080.0" type="actend" person="1" link="0" actType="h" /><event time="1080.0" type="departure" person="1" link="0" legMode="car" /><event time="1080.0" type="wait2link" person="1" link="0" /><event time="1081.0" type="left link" person="1" link="0" /><event time="1081.0" type="entered link" person="1" link="2" /><event time="1092.0" type="left link" person="1" link="2" />
![Page 151: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/151.jpg)
Practical Considerations
© 2010 IBM Corporation151
![Page 152: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/152.jpg)
High-level Suggestions
� Follow and understand the data– Problem characteristic depends on it– Relevance of algorithms depends on it– Result quality depends on it
� Prototype early– Optionally, in a simulator, and
© 2010 IBM Corporation
– Optionally, in a simulator, and– Definitely, in real-world– No solution works unless millions are using it
� Think long term– Traffic systems last for decades, not 2-3 years (typical life of a computer)– Support inter-operability in IT as well as in users– Quantify benefits and costs using standard metrics– Always think of people issues
• Good to have analogies with computer networks, but remember that packets can be dropped while travelling, but not people
• If people adopt, an imperfect solution will still be successful
152
![Page 153: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/153.jpg)
Experience in Dublin, Ireland
� Close collaboration with the city– Jointly funded research center
� Access to a number of live data feeds– GPS from buses, loop, traffic signal cycles, CCTV
© 2010 IBM Corporation
� City was willing to try out research prototypes
� Goal was to improve different aspects of transportation
153
![Page 154: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/154.jpg)
Experience in Stockholm, Sweden
� Close collaboration with KTH and with Trafik Stockholm– Research funding
� Access to taxi GPS data and to “MCS” data (overhead microwave sensors)– Through a 3rd party collector– Privacy restrictions limited completeness of data
© 2010 IBM Corporation
– Privacy restrictions limited completeness of data
� Ongoing research collaboration
� Key goal is to improve travel time estimation to various points in the city
154
![Page 155: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/155.jpg)
Experience in New York, USA
� Research Collaboration with CUNY and NYU
� Access to traffic data (only start and stop information)
� Privacy restrictions limited identification information
� Very limited in terms of ability to extract traffic information
© 2010 IBM Corporation
� Very limited in terms of ability to extract traffic information
� However, can still extract human mobility patterns
155
![Page 156: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/156.jpg)
Experience in Boston, USA
� Smarter Cities Challenge (SCC), an IBM philanthropic initiative to bring IBM's experts to cities around a topic and get a recommendation on what the city needs to do. – SCC in Boston was on traffic in June 2012 for 3 weeks. – The project has been unique for two reasons: (a) academia has been involved and (b) the team does a running prototype along with a recommendation.
� Boston collects a significant amount of data from many sources that could be quite useful to researchers, developers, transportation engineers, urban planners, and above all, citizens.– Siloed, in multiple formats and not fully exploited.
© 2010 IBM Corporation
– Siloed, in multiple formats and not fully exploited. – Developed a prototype that created a standards-based common model, loaded > 1M data from 3 sensor types already in city, and showed the potential of insights from them; also recommendations for more sensor types and analytics
156
Press on the IBM SCC Boston team work:
1. Boston Globe, June 29, 2012http://www.boston.com/business/technology/articles/2012/06/29/ibm_gives_advice_on_how_to_fix_boston_traffic__first_get_an_app/
2. Popular Science, 2 July 2012http://www.popsci.com/technology/article/2012-07/bostons-ibm-built-traffic-app-merges-multiple-data-streams-predict-ease-congestion
3. Others: National Public Radio (USA), and a range of local TV stations on the work.
Team at work – Source: Boston Globe article
![Page 157: Tutorial on Taffic Management and AI](https://reader033.vdocuments.net/reader033/viewer/2022042607/554a3343b4c9051b578b537d/html5/thumbnails/157.jpg)
More Attempts
� Around the World– Level of instrumentation can vary drastically, or even be absent – Ideas needed to incentivize people to reduce demand– Funding for large-scale supply (infrastructure) increase is reducing
� Beyond city bodies– Collaboration with mobile companies
© 2010 IBM Corporation
– Collaboration with mobile companies– Collaboration with bus companies– …
� And by individual organizations. Any can take a lead to reduce traffic demand• Car-pooling. [Making Car Pooling Work – Myths and Where to Start, Biplav Srivastava, in 19th World Congress on Intelligent Transportation Systems, Vienna, Austria, 2012]
157