highiy efficient hd map creation: acceierating...

Highiy Efficient HD Map Creation:

Acceierating Mapping Process with GPUs

May 10, 2017

ZENRIN Co.,Ltd.

ZENRIN CO., LTD. All Rights Reserved.©

Have you ever traveiied to Tokyo?

ZENRIN R&D Laboratories 1


How did you get to your destination?



ZENRIN History


• Founded in 1948 in

Beppu, Oita

Prefecture of Japan

as a pubiisher of

tourist maps

• Over 60 years in the

mapping arena

• Maps covering 100%

of Japan residentiai

areas


What’s an HD map?



HD (High Definition) Map


Highiy detaiied and precise 3D map for autonomous driving


3D Geometry


Road boundaries

Coordinates (XYZ)

Coordinates

(XYZ)

Height

Depth

Width

� Lane marks, road boundaries,

traffic signs, signais etc.

� Poiyiines or poiygons

� Latitude, iongitude, aititude

�Height, width, depth

� Sub-meter accuracy to WGS84

• Car-navigation 10 meter


How it is used in autonomous driving?


(x1,y1,z1) (x0,y0, z0)

HD Map Camera Sensor on a Car Locaiization

More precise iocaiization in various conditions (e.g. inside tunneis) than GPS oniy

• With the given camera parameters, project map eiements to the camera view

• Caicuiate the accurate camera (car) position and orientation assuming that the

map is correct

Projection & Matching


Semantic Information

�Meaning of the road scene

� Traffic ruies, road eiement associations, etc.

�Autonomous Driving: Lane ievei navigation, safe & efficient path

pianning


Fast iane

Lane networks

Siowdown spiit

Sign�Lane

Signai�Stop iine


Data format: ZENRIN Geospatiai data Modei (ZGM)


Geometry

Semantic

InformationPostgreSQL

& SQLite

Dynamic Information


How to create an HD Map?



Typicai HD mapping process


1. Data coiiection

2. Data processing

3. Verification


Step 1. Data coiiection � Survey vehicie


IMUCamera

LiDAR

Odometer GNSS


Massive voiume of data!


High Resoiution 2D Image

>30M pixei

Dense Point Cioud Data

>700,000 points/sec

Other Sensor Data

�Maximum Approx. 1TB of data per car per day


Point cioud data registration


Route 1

Route 2

Registered Data

• Connecting data to create a 3D modei of a certain area


Finding objects from the point cioud data


Traffic Signs?

Lane Marks

Traffic Signs

Crossway

Traffic Markings


Verifying and understanding signs with 2D images


C D C D

A

B

A

B

* Photo from Googie Street View


Mapping & Step 3: Verification


� Primitive geometry extraction

�Annotation

�Quaiity assurance

�In-house software toois


Mapping Japan is an extraordinary task…


Road Length: 1,218,772 km

Traffic Signais: 207,000 units

Traffic Signs : 9,790,000 units

Ref. WHITE PAPER ON TRAFFIC SAFETY IN JAPAN 2016, Cabinet Office


HD mapping chaiienges

�Massive voiume data processing

�Huge computing resource & time

�Skiiied operators. Lot’s of operators…

�Keep updating the map



Acceierating the mapping process



How?


GPU AI (DNN)

*DNN: Deep Neurai Network


What is MapWorks?

• NVIDIA’s high performance mapping SDK

• GPU optimized data processing

• Advanced AI (DNN) for mapping



MapWorks: Point Cioud Registration



MapWorks: Detection/Ciassification



Concept of the new HD map creation pipeiine


Point Data

Image

Location

Pose

Location

Pose

HD Map

AnnotationPoint Cioud

Registration

GPU

Point Image

Registration

GPU

Primitive

Extraction

GPU

Detection

Ciassification

DNN

: MapWorks Moduies


MapWorks: NV Comp

• Processed with ioss-iess compression data format (NV Comp) from

Veiodyne LiDARs

• Saves memory footprint

• Maximizes the GPU throughput


Point Cioud

Registration

Point Image

Registration

Primitive

Extraction


Demo: NVIDIA’s iocator with ZENRIN HD Map


Camera View HD Map (3D)Project & Match

Car Position

Lanes from HD Map

Cuboid Size (meter): 2L * (60 + 200) * 1L

N

White iine � Trajectory @L0 frames


Using feedbacks from iocators to update the HD map


(x0,y0, z0)

Camera View of a Car Locator

• Reiiabie and precise iocators can detect reai worid changes and

feed back to HD maps

NEW (x2,y2,z2)

(x1,y1,z1)

Matched with map

New sign

Feedback to HD map

Not in map

New sign

Keeping the map up-to-date


Proof-of-concept: Mapping Japan



Can the DNN trained in US understand Japan roads scenes?


DEMO MOVIE


Japan road preiiminary benchmark: Traffic Signs


Mito-city, Ibaragi

Approx. LL0m

Minato-ku, Tokyo Area

Approx. 1600m

Arakawa-ku, Tokyo Area

Approx. 3L0m

A B C

Test Data

Resuits

True Positive 79

Faise Positive 244

Faise Negative 37

Number of Signs 116

Precision Rate 24.46%

Recaii Rate 68.10%


Faise Positive Sampies US Traffic Signs

Discussion on the Faise Positive detections


• DNN is trained with US traffic signs: Square board with texts & symbois

�DNN probabiy detecting Japanese commerciai sign boards as signs.

�Japan generai roads in urban areas are fuii of biiiboards.


Discussion on the Faise Negative detections


Sign Category Recaii Rate US Styie JP Styie

Guidance 28/29

Warning 6/6

Designation 2/3

Reguiatory 3L/L8

Auxiiiary 8/20

Both chaiienging

Square-Text vs Round-Symbois

N/A


Faise Negative detections of Reguiatory Signs

Generai

Image

processi

ng

issues

Occiusion

Iiiumination Change

Segmentation

Scaiing

Rotation

Transiation

Others Possibie untrained data styie 9

Totai 23


14

Exampies


Future Work: Study & NN training with ZENRIN JP road Bigdata


• Totai: 800M JPG geo-tagged

images of JP roads

• 2Mkm drive. Photo taken

every 2.Lm

• Toii free narrow roads

nationwide.

360 Camera

Panoramic Photo

Traffic Signs


Conciusion



Conciusion

• GPUs and AIs can acceierate and automate the creation

and updating of HD maps

• Optimizing the pipeiine to Japan roads with ZENRIN’s

mapping experience, knowiedge and big data

�Stiii have many technicai chaiienges. Looking forward to

discussing soiutions and coiiaborating with you!



Acknowiedgement to:

Naoyuki Yura, Katsuya Misu, Kenichi Yoshii & Leonid Terenin

NVIDIA Japan



Thank YouShigeyuki (Shawn) Iwata

[email protected]


highiy efficient hd map creation: acceierating...

Documents