akhil khemani akhemani@usc

13
GOOGLE STREET VIEW: Capturing The World At Street Level- Dragomir Anguelov, Carole Dul Daniel Filip, Christian Frueh, Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver, Google . 2010 Akhil Khemani [email protected]

Upload: tamira

Post on 19-Jan-2016

59 views

Category:

Documents


1 download

DESCRIPTION

GOOGLE STREET VIEW: Capturing The World At Street Level- Dragomir Anguelov, Carole Dulong, Daniel Filip, Christian Frueh, Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver, Google . 2010. Akhil Khemani [email protected]. Introduction. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Akhil Khemani akhemani@usc

GOOGLE STREET VIEW: Capturing The World At Street Level-Dragomir Anguelov, Carole Dulong,

Daniel Filip, Christian Frueh, Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent,

and Josh Weaver, Google .

2010

 

Akhil [email protected]

Page 2: Akhil Khemani akhemani@usc

Introduction

• What this paper discusses ?Street View serves millions of Google users daily with panoramic imagery captured in hundreds of cities in 20 countries across four continents. A team of Google researchers describes the technical challenges involved in capturing, processing, and serving street-level imagery on a global scale. [Link]

• How it all started?•Google cofounder Larry Page drove around the San Francisco Bay Area and recorded several hours of video footage using a camcorder pointed at building facades. •Research collaboration with Stanford University called CityBlock that soon thereafter became Google Street View.

• Issues?•CIA’s The World Factbook:

• the world contains roughly 50 million miles of roads, paved and un- paved.• 219 countries. • Driving would be equivalent to circumnavigating the globe 1,250 times .

2

Page 3: Akhil Khemani akhemani@usc

History

3

• Garage Phase - SUVs • cameras, lasers, and a GPS on the roof & several computers in its trunk.

• 2nd Generation - Chevy Van.• Laser scanners• 2 high-speed video cameras, 8 high-resolution cameras (rosette (R) configuration), • recording data to 20 hard drives at 500 Mbps.

• 3rd Generation - “lite” cars. • had a low-resolution camera • One standard desktop PC with a single hard drive. • enabled international expansion to Australia, New Zealand, and Japan.

Page 4: Akhil Khemani akhemani@usc

History – ctnd..

4

• Fourth Generation : R5-custom panoramic camera system.• allowing the camera to be retracted when the vehicle passed under low bridges. • capture of coarse 3D data alongside the imagery. • has captured the majority of imagery live in Street View today.

• Others:• Trike, Snowmobiles and Trolleys to quickly record pedestrian routes in both urban and rural environments.

• Links: The White House Tour.

SNOWMOBILE TRIKE TROLLEY

Page 5: Akhil Khemani akhemani@usc

Challenges

• Hard drives are sensitive to shock, vibration, and temperature extremes, both while the vehicle is in operation and, to a lesser degree, while being shipped.

• We use various techniques to minimize data loss :•shock-mounted disk enclosures •Custom-shipping packaging with extra-thick foam•Solid-state disk drives are also used when temperature and vibrations

5

Page 6: Akhil Khemani akhemani@usc

Camera

6R7 Street View camera system.

• The R2 System• a ring of eight 11-megapixel, interline-transfer, charge-coupled device

(CCD) sensors with commercial photographic wide-angle lenses.

• The R5 system • uses a ring of eight cameras, like R2, • plus a fish-eye lens on top to capture upper levels of buildings.

• The R7 system• uses 15 of these same sensors and lenses, but no fish-eye, to get high-

resolution images over an increased field of view

• Reliability addressed through a ruggedized design, well-tested firmware and software, substantial vibration and temperature testing, burn-in tests of every rosette.

Page 7: Akhil Khemani akhemani@usc

POSE OPTIMIZATION

• Goal: Accurate position estimates of Street View vehicles are essential for associating our high-resolution panoramas with a street map and for enabling an intuitive navigation experience.

• Approach :• GPS, wheel encoder, and inertial navigation sensor data logged by the

vehicles to obtain these estimates. • An online Kalman-filter-based algorithm is deployed on the vehicles to provide

real-time navigation information to the drivers.• A model which includes detailed knowledge about one-way streets and turn

restrictions is used to optimally transform the sensor pose into accurate road-positioning data.

7

Page 8: Akhil Khemani akhemani@usc

NAVIGATING STREET VIEW IMAGERY

• Amongst various ways in which Google surfaces Street View Images, 360-degree panorama is most popular. user can virtually explore streets & cities using many mobile and desktop clients.

• Combination of Street View imagery with other data sources:• finding local businesses, getting driving directions,• real estate search.

• User contributions • correct the location of businesses by directly dragging markers in Street

View and automatically snapping it to facades. (Picture)• photos from Flickr, Panoramio, and Picasa in Street View.

8

Page 9: Akhil Khemani akhemani@usc

LEVERAGING 3D DATA FOR SMART NAVIGATION

•CLICK-TO-GO :• Street View supports a unique 3D navigation mode known as “click-to-go,”

which lets users click their mouse on a point in the scene and be transported to the image nearest to that point’s 3D location.

9

Page 10: Akhil Khemani akhemani@usc

DEPTH MAP

• DEPTH MAP : • that stores the distance and orientation of every point in the scene .• using laser range scans or image motion (optical flow) when laser data isn’t

available. • It is of lower resolution.• For 3D navigation, depth map only encodes the scene’s dominant surfaces,

such as building facades and roads, while ignoring smaller entities such as cars and people.

10

Page 11: Akhil Khemani akhemani@usc

DEPTH MAP – ctnd..

• DEPTH MAP : • Panoramic Depth Map:

• by tracing rays from each panorama position. • Each pixel in the depth map represents a lookup into a table of 3D plane equations,

which enables the client code to reconstruct the real depth values at runtime.

• Lossless Compression: The encoded depth map is just a few KB in size. • Panoramic 3D anaglyphs.

11

Page 12: Akhil Khemani akhemani@usc

COMPUTING 3D MODELS FROM LASER DATA

• Street View data to create photorealistic 3D models for Google Earth.• Google Earth created 3D city models using airborne imagery, resulting

in low-resolution facades with little detail• Suitable for fly-through • Unsuitable for walk-through experience.

• In contrast, 3D facade models reconstructed from Street View’s laser scans and imagery are high resolution.

• Final 3D Facade Models : existing airborne models are fused into a single model that has high-resolution facades as well as rooftops and back sides from an airborne view.

12

Original 3D models of a New York (airborne data) only.

(b) Fused 3D model with high-resolution facades.

Page 13: Akhil Khemani akhemani@usc

SERVING IMAGERY TO USERS

13

• Stitch the images into panoramas .• Tile them at different zoom levels• face detection and license-plate detection on all published imagery and

blur images accordingly to protect privacy.