ITS and Telematics in Japan

Katsushi IkeuchiThe University of Tokyo

Outline

BackgroundIn-Vehicle terminals– Car navigation– ETC– AHS & ASV for safe driving

Road Traffic InfrastructureU Tokyo Research Program

Organization

Person

Vehicle

Advanced Information Society Dev. HQ(Director: Prime Minister) 8/1995

7/1995

4 Ministry Meeting(Police,Post&TelecomTransport,Commerce)

ITS Japan

1/1994

ITS World

Congress

Industries Universities

ISOCommit.

TC/204

9/1992

Navigation Systems

Emergency Vehicle Op.

PedestriansSafe Driving

Traffic Management

Road Management

Public Transport

Electronic Toll Collection

Commercial Vehicle Op.

ITS Key Development Fields in Japan

Characteristics of Japanese Market

Navigation

Driver Service (from 2003)

Safe Driving Assistance (from 2005)

Information for Drivers

Information for Drivers

StaticInformation

DynamicInformation

DynamicInformation

Package Media

Communication

- Road Map- City Map, etc.

- Traffic Congestion- Traffic Accident- Road Construction- E-mail, etc.

Information for Drivers

4th GenerationMobile Phone

ETC

10k 100k 1M 10M

Mobility

Data Transfer Rate (bps)

PDC

PHS

IMT-2000

Radio Beacon(VICS)

MMAC

Optical Beacon(VICS)

ETC

Digital Broadcasting

DSRC

Stationary

WalkingSpeed

High-SpeedDriving

Communication System in ITS

Communication Infrastructures

1980 1990 2000 2010

PSTN

High SpeedDriving

Movin

g Speed

1k 10k 100kData Transfer Rate (bps)

1st Generation 2nd Generation 3rd Generation 4th Generation

Analog MobilePhone

(High-CAP)

Digital MobilePhone(PDC)

IMT-2000(Japan W-CDMA)- Global Standard- Multi-media Capable(64K - 384k - 2Mbps)

IMT-2000(Japan W-CDMA)- Global Standard- Multi-media Capable(64K - 384k - 2Mbps)

FPLMTS ( )10Mbps

High Speed Move

CordlessPhone PHS

Wireless ATMMulti-media Capable(up to 12Mbps)

20-40GHz10 Mbps

ATM

IMT:International Mobile Telecommunications

WalkingSpeed

Stationary

i-Mode(PDC Packet)

Roadside-Vehicle Communication

Electro-magneticInduction

Milli-meterWave

1996 1997 1998 1999 2000 2001 2002 ---

Bus Locators

Radio Beacon(2.5 GHz:VICS) Electronic Toll Collection System

(5.8GHz:ETC)

Optical Beacon (800nm :VICS --> Nationwide expansion

MilliWave

RadioWave

Optical

DSRC

In-vehicle Terminals

Evolution of In-vehicle TerminalAdvances in Navigation System

DVD Car Navi

- Advances in Search Function

- Touch Panel

- Operation and Guide by Voice

- Handy Navigation, etc.

Mobile Communication- VICS- Interactive Communication via

Mobile Phone- Internet- Digital Satellite Broadcast- Information Contents Industry,

etc.

VICS Navi

Safety and Comfort

- Rear Monitor

- Blind Corner Monitor

- Zigzag Driving Detection Function- Shift Control Coordination

On-board Camera, HELP

A Multiplicity of Functions

AVN, ETC

- FM Multiplex with Subtitle- Coordination with TV and

Audio Systems- Display and Operation of Air

Conditioner- On-board Personal Computer

Car Navigation

In-vehicle Terminal“Car Navigation”

History of Car Navigation

A/M

OEM

Navigation Sold in Market Total 7.8Mil. VICS Sold in Market Total 2.8Mil.

Car Multimedia

Comm. NaviInternet

AVNDVD

D-GPS

VICSVoice Recognition

Detailed City MapVoice Guidance

Route Guidance

Road MapGPS

(Unit: 000)

2,500

2,000

1,500

1,000

500

091 92 93 94 95 96 97 98 99 2000 2001

1,300

861

627

337

1493012

1,010

43%

1,950

1,530

57%

DVD Qty(Est.)

(2,400)

1,400

35%

128314

577796

989

(1200)

VICS Qty(Est.)

GPS

GPS Antenna

1. Indication of Self-position2. Search of Destination (Destination Input)3. Calculation of Route4. Guidance to Destination

LCD Monitor

Navigation Computer

Speed Sensor

Map Disc

DVD-,CD-ROM

Architecture of Car Navigation System

3D Magnified Intersection Map 3D Satellite Map

•WIDE-VGA display•Gyro, D-GPS, Map-matching•Voice Control•Route Search (64 bit bus Risc CPU)•Destination Search (Address, Tel#)•Connectivity to VICS, i-mode, HELPNET, ETC

DVD Car Navigation Systems (e-navi:fine sped network)

VICS Center- Established in ’95 and Started in Apr., ’96.

- Road Traffic Info. In Real Time through Communication Media.

- Media: Radio Beacon, Optical Beacon,FM Multiplex Broadcast

Car Navigation- Road Traffic Info. Displayed by Maps,

Diagrams and Characters

- Less-congested Route Guidanceby a Triple Media Receiver in 1-DIN Body

Vehicle Information & Communication System (VICS)

Information Collection Information Providing

Ministry of ConstructionNational Police Agency

Japan Road TrafficInformation

Center

Japan Road TrafficInformation

Center

OtherInformation

Sources

OtherInformation

Sources

Radio BeaconRadio Beacon

Optical BeaconOptical Beacon

FM Multiplex Broadcast

Traffic Congestion Info.Map Display

Traffic Management

Center

VICS Center Communication Media

Car Navigation

Traffic Congestion Info.Diagram DisplayInformation Processing

!

!

First Phase Service Start Mar.1998

Tokyo, Chiba, Saitama, Kanagawa, Osaka , Aichi, Kyoto, Hyogo,Nagoya National Expressway

Second Phase Sep 2001

Fukuoka, Hiroshima, Miyagi, Hokkaido(around Sapporo)Shizuoka, Gunma, Okayama, Fukushima, Okinawa, Miyazaki, Gifu, Mie, Yamaguchi, Ibaragi, Hokkaido(around Asahikawa) Wakayama, Shiga, Nara, Tochigi,Yamanashi

Third Phase around 2005

Niigata, Ishikawa, Hokkaido( around Hakodate), Kumamoto, Oita, Kagawa, Ehime, Saga, Nagasaki, Kagoshima, Tokushima, Kochi, Fukui, Toyama, Yamagata, Akita Aomori, Iwate, Shimane, Tottori

9

20

VICS Service Coverage

Restaurant/Hotel/Gas Station, etc.

JapanRoad & TrafficInformation

Center

Kanagawa/Chiba/SaitamaTraffic Control

Centers

Japan HighwayPublic

Corporation

Police AgencyTraffic Control

Center

Tokyo MetropolitanExpresswayCorporation

In-vehicle Navigation System Personal Computer ATIS Terminal

Office/Center

Traffic Information Service ATIS/ITGS

MobileTelephone

TelephoneLine

Car

TelephoneLine

ATIS / ITGS Bi-directional Traffic Info. Service

Route guidance

I-mode functionsTransactionNews, Weather forecast, Town infoGame, KaraokeE-mail

i-mode Car Navigation

-Road traffic info. (Congestion, Travel time, etc.)-Weather Info. (Fog, Snow, Icy road, etc.)-Parking Info. Opening hours, Tariff, Availability-Reservation (Hotel, Restaurant, etc.)-Help (Accident, Illness, Theft, etc.-E-mail, Internet, Music download

IP (Information Provider)

Center

IMT-2000

- Voice- Data- Moving

Picture

64kbps

384kbps

Camerai-mode

HELP

H/F

GPS

I/F USB Bluetooth

ETC

DVD map

SD card

MPEG4

Voice Recognition

IMT-2000 Car Navigation

- IMT-2000 Terminal Equipped with GPS Unit.

- Download a Map and Indicate the Present Position.

- Search for POI.

- Route Guidance.

IMT-2000 (W-CDMA) Walking Navigation

Electronic Toll Collection

Electronic Toll Collection (ETC)

Vehicle Detector

CameraEquipment for Monitoring

Lane Control Equipment

Roadside WirelessEquipment

Toll Booth Roadside System ETC Vehicle Unit5.8GHz, 1Mbps

Bi-directional DataCommunication

- Reduction of Traffic Jams by Non-stop Automatic Toll Collection(35% of Total Jams are at Toll Booth)

- Started Around Chiba Area (Mar. 2000)- Service Upgrade by Cash-less System- Reduction of Management Cost of Toll Collection

Core Tech. :Toll Collection / Lane Management / Vehicle Detection /Licence Plate Recognition

Main Unit

Antenna

Core Tech.:Road to Vehicle CommunicationCoding / Vehicle Unit / Car Navi.

!

!

Operational Section(164 Tollgate)Planned Section Year 2001(600 Tollgate)

* 70% of Tollgate in Japan will be Coveredby the End of FT 2002 (900 Tollgate)

Development Plan

Optical Cable

Server

ETC Center

SA/PA, Michi-no-EkiManagement

ETC Terminal

ETC Terminal Connected with Navi.

Year 2000

ETC/DSRC Dual TerminalASK/QPSK)

ETC/DSRC Dual TerminalConnected with Car Navi.

Simplified Road-side Radio Equipment

ERP

Michi-no-Eki,SA/PA

Gas Station

Drive Through

Parking

Led by Government Agency

Led by Private Business

ERP Center

Charge Clearance, Provision of Information

Charge Clearance

Deployment of ETC and New ETC/DSRC System

Server

Server

Emergency Report System

Lung Resuscitation within Two Minutes of an Accident: 90% Chance of Saving the Life.

Accident Occurs Report Ambulance Arrivesat the Scene

Arrival at Hospital

Total:7.3 min.1

8.9

1

1.3

5.3

6.1Without HELP

With HELP

Total:16.3 min.

Emergency Reporting System (HELP)

HELP architecture

Rescue

Automatic Report

Manual ReportAccident Position

AccidentIllness

110

Police Dept

119

Fire Dept.

Emergency Report Center

Vehicle Unit

Road Service

Emergency Vehicle Support

Rescue

Cellular Telephone

GPS

Battery

OHM

NavigationSystem Air-

Bag Crash Information

Location Information

TEL AntennaManual Report

Accident

Power

PDC Module

Primary Battery

HELPTerminal

(Manual Button, Microphone, Indicator)

HELP Terminal

BrowserSoftware

(Smart Media)

EmergencyButton

LCD monitor

Speakerfor

Voice Guidance

Microphonefor

Voice Recognition

Hands-freeSet

MobilePhone

Modem

DedicatedMap DVD

Replace withStandard MapDVD

Car Navigation SystemCar Navigation System Mobile Communication UnitMobile Communication Unit

CD-ROMfor

HELP Function

CD-ROMfor MONET,

i-modeFunction

HELP Terminal (Option for Navigation System)

In-vehicle TerminalAssistance for Safe Driving

AHS: Advanced Cruise-Assist Highway System

2000 Proving Test Program “Smart Cruise 21”Oct.-Dec.: The Public Works Research Institute

of the Ministry of Construction

2003 Start Installation of “Smart Cruise 21”

2010 Establishment of 150,000 km long “Information Highway” Optical Fiber Network

2015 Realization of “Smart Way”

By AHS Research Association

Smart Cruise 21

Preventionof Lane Departure

Preventionof Overshooting on

Curve

Preventionof Collision with

Pedestrian

Maintaining Headway

Preventionof Right Turn Collisions

Preventionof Crossing Collisions

Preventionof Collisions with

Obstacles

- Blind Spot Assist (Parking Assist,Blind Corner Monitor Synthesize Several Images and Provide Wide Angle Image

- Visibility Enhancement (Night Vision) Obstacle Detection Using Infrared/Visible Camera and Warning System

- Reduction in Driving Operation(Look Off Detection, Stop & Go) Assist Using Camera Image, Radar, Lane-Marker

Image SynthesesImage Syntheses

Camera Image

Synthesized Image

Rear-viewCamera

Left Door Mirror Left Pillar

Image ClusteringImage Clustering

Real Image

Clustering

Infrared/VisibleCamera

Infrared/VisibleCamera

Detection of Face Direction

Lane-Marker

Detection of Face Direction

Lane-Marker

Night Vision

- Radar- Distance Measurement

by Stereo Image- Lane Marker

- Radar- Distance Measurement

by Stereo Image- Lane Marker

Sedan

Truck

Parking Assist

Look OffDrive

Detection

Blind Monitor

Stop & Go

ASV:Advanced Safety Vehicle

Right Pillar

Right Door Mirror

Rear view camera

108 wide angle back view 44 normal angle side view

Metal Housing CaseResin Housing Case

Front lateral view camera

From dealer’s option manualTaken on photo

On-board Monitoring Camera

Road Traffic Infrastructure

Mobile Communication

Diversity of Functions

Traffic Control SystemRoad Information System

- Beacon- FM Multiplex Broadcasting- Mobile Phone- Internet- Digital Broadcasting

- Collection of Traffic Info.- Collection of Weather Info.- Processing of Traffic Info.- Provision of Traffic Info.- Control of Traffic Signal

- ETC- Road Pricing- Dynamic Route Guidance- Priority Passing for Public Transport

- Support for EmergencyVehicle

- Decrease of Pollution

- Emergency Report- Anti-Theft, Trace- Traveling Support Road-Support for Pedestrian

Evolution of Road Traffic InfrastructureAdvances in Functions

Safety / Comfort

Information Collection Information Processing Traffic Control

Information Provision

AVI Sensor 2,200

Traffic Flow Measurement :2,050Traffic Control Center 170 Cities

Traffic Control Center(Processing System)

Traffic Signal

Multi-Pattern Display

Free-Pattern Display

Road-Side Communication

Ultrasonic Sensor:120,000Optical Beacon:36,000 165,000

2,700

810

Police Traffic Management System

90

(Inside Controlling Area:57,000)

Incident Detection System

Image Recording Device

Control Processing Device

Device for Detecting Unforeseen Occurrences

CCD Camera

CCD Camera

System is Controlled by Images

from Each Camera

CCD Camera

Display Board

Provide Information to the Following Vehicle

Travel Time Information Board

15

AVI(Automatic Vehicle Identifier) System

Measured Travel Time

License PlateLicense Plate

Image RecognitionImage Recognition

AVI: Automatic Vehicle Identifier

Recognition Unit

Sensor CameraCameraFlash

AVI Central Unit

AVI Terminal Unit(Point B)

AVI Terminal Unit(Point A)

- Prepare Travel Time Information

- Prepare Clustering Information

- Prepare OD Information

Bus Locating System

Electromagnetic InductionEffect Creates a SignalEach Time Buses Pass by.

○○

““

○○

Bus Approach Indicator Shows the Bus Location with a Flashing Light.

Data Collecting Station

Data Providing Station

Voice Command Station

Location Monitoring Servers

Operation Status Displays

Host Computers

Data AuthoringSystems

Center System

Monitor Bus Location Via Two-way Communication System Using Electromagnetic Induction Antennas at Bus Stops.Central Data Collecting Station Analyzes Bus Transit Information Supplied through MCA Control Station.Central Data Providing Station Delivers Bus Operation Status/Approaching Information to Each Bus Stop through MCA Control Station.

Bus ApproachIndicator

““

Social BenefitsSocial Benefits

Optimum DispatchingOptimum Dispatching

Character/VoiceMap display

Character/VoiceMap display

ManagementEfficiency

ManagementEfficiency

CustomerManagementCustomer

ManagementAutomatic

DispatchingAutomatic

Dispatching

Customer ServiceImprovement

Customer ServiceImprovement

Reduction of Driver’s StressReduction of

Driver’s Stress

OperationRecord

Management

OperationRecord

Management

Closest TaxiSearching

Closest TaxiSearching

IncidentInformationIncident

Information

Taxi Dispatching System

Taxi Radio

DestinationRoute Searching

DestinationRoute Searching

Fleet Operation Management System

GPS

Radio communication Controller

Control StationAC Power Supply

Controllerand

Microphone CPU

Office-side System

● System Specifications (Capability)Connectable Stations 2 Control Stations/systemMax. Vehicles 200 Vehicles/SystemNumber of Zone 16 Zones/SystemCustomer Listing 20,000 Customers/SystemNumber of Registered Conditions 99 Items /SystemPre-registered messages 15 Messages/System

10 Messages/Control StationRegistered Condition Selector

Option

GPS Receiver + GPS Antenna

Mobile Radio + Communication ControlUnit

Mobile System

- By Using MCA Radio Communication, an Office Display Indicates the Fleet Location and Status.- Computer Finds the Customer’s Location from Their Name or Telephone Number and Indicates on the Display.- Customer Information is Provided to a Delivery Truck Closest to the Customer.

(Note)

(Note) Car Navigation Unit can be used as the Mobile Radio Unit.

- Customer Location- Vehicle Location- Vehicle Direction/Speed- Trace of the Fleet

Digital MCA Control Station

Microphone

U Tokyo Research Program

Information gathering– Recognition of vehicle in infrared images– Land mark recognition– Event recognition at intersection

Information display– Modeling environmental map through vision– Visualization of arbitrary views for driver

Recognition of vehicles(in Infrared Images)

Eigen-window method•Recognize vehicle without reading number plates•Possible under occlusion•No need to segmentation

Previous methods•Detection; not recognition•Vehicle class by size

Eigen-window methodTraining images

Input images

votingEigenspace

Recognition results

Car-A

Voting space : Pos x Model#

Recognition results (outdoor)

daynight

daynight

Event recognition

TV camera

Surugadai Local analysis

Sakauchi LabNTT Optical fiber line

Event DBapplication

Event recognition

StoppingIllegal parking

Tracking

Delete

Extend

Move Blocks based on

Estimated motion vector

Adapting to

new formation

Next frame

Current frame Next frame(Middle situation)

Adaptive algorithm

Feature Extraction

1

2

Normalization Indexing

8

Motion vectors

HMM

State

0

State

1

State

n

a01

b01

(k)

a12

b12

(k)

an-1n

bn-1n

(k)

a00

b00

(k)

ann

bnn

(k)

a11

b11

(k)

Left - to - Right HMM

♦ BumpingClose in Impact Repulsion

Close in

Close in

Stop Restart

Overlap Pass

♦ Tandem

♦ Passing

Accident Models in HMM

0 1 2 3 0 8 7 0

0 1 2 0 8 7 0

0 1 2 3 2

Yokohama-harajuku AHS

Accident recognition

Accident recognition

Modeling Urban Scene

Omni camera

Moving Direction

1. Image base rendering2. Model base rendering3. Augmentation

Data Acquisition Vehicle

Outline

Acquiring omni-images along a path

Generate another images along a different path

Panoramic image

Perspective images

Image capturing

Basic idea

P

C

Synthesize image

C

B

A

B

A

2D image from light field

Synthesize Sequence(Changing lanes)

Synthesize Sequence(Rotation)

Merits

Image sequence different from acquired one

Acquired Synthesized

Constructing 3D Map

Omni image acquisition

3D digital map by Omni analysis

Texture mapping

Omni Spatio-temporal Image

Cross-section (an elliptic curve)D

epth Info

Digital Map

Depth info +

Correspondence between map and image

Cross-section (a radius line)

Panorama image

3D Map

Virtual Surugadai intersectionCurrent situation into virtual worldWarning to drivers

Outline

Tracking vehicles

Combine VR vehicles with the VR Surugadai

Generate arbitrary views with transparent building

Tracking

Making paths to run

Projecting detected coordinates onto the virtual world coordinateMaking paths to run by connecting them

Tracked images The virtual world coordinate system

Transparent buildings with vehicles

Modeling from Reality

Real Object

VR Image

Modeling from Reality

U Tokyo Program

Information gathering– Vehicle recognition based on Eigen window– Event recognition based HMM

Information display– Virtual Surugadai intersection – Modeling urban scene using Omni images

Summary

BackgroundIn-Vehicle terminals– Car navigation– ETC– AHS & ASV for safe driving

Road Traffic InfrastructureU Tokyo Program