1

1Web3D 2006

Using Expressive Rendering for Remote Visualization of Large City Models

Wednesday, April 19

Jean-Charles QuilletGwenola ThomasXavier GranierPascal GuittonJean-Eudes Marvies

2

2Web3D 2006

Introduction

Remote visualization is concerned with improving realistic rendering• Complex 3D models

• Complex lighting system

• Progressive texture transmission

For trip or assisted navigation• The solutions mostly rely on 2D information

Available on PDA and mobile devices combined with GPS• Powerful navigation tool

Remote visualization city models on mobile devices

3

3Web3D 2006

Introduction

Remote visualization context• The server disposes of the entire geometry of the virtual city

• It sends a subset of the city model to the client on demand

Virtual cities• Simplified geometry

• Textured by photographs of real building facades

Issues• Limited network bandwidth

• Not adapted to the small size of the display

4

4Web3D 2006

Our approach

Expressive or non-photorealistic rendering (NPR) with feature lines

• Making more legible image

• More rapid transmission for data

We present a pipeline able

• To extract the feature lines from photographs of building facades

• To transmit and render them on clients

5

5Web3D 2006

Previous work

NPR for remote rendering• [Hekmatza et al. 2002] and

[Diepstraten et al. 2004]

• Lines are computed for each new viewpoint

• Adapted to smooth 3D models

For cities, feature lines are related to the façades

3D model

Server ClientViewpoint

Extracted lines

2D Line-based rendering

6

6Web3D 2006

System overview

Buildings extrusion

Extraction of feature lines

Spatial subdivision

Visibility computation

Model streaming

Line rendering

City footprints+ Textures

NPRdatabase

Optimizeddatabase

NPRmodeling

Databaseoptimization

Client-servervisualization

Offl

ine

Onl

ine

7

7Web3D 2006

Scene subdivision

Goal: produce a 3D database ready to be streamed using visibility information

• VRML97 extension for visibility streaming [Marvie et al. 2004]

Scene subdivision in cells to produce the navigation space

• Constrained BSP subdivision [Marvie PhD 2003]

Computation of Potentially Visible Object Set (PVOS) for each cell• Compute cell-to-objects visibility relationships

• OpenGL shooting algorithm [Marvie PhD 2003]

8

8Web3D 2006

Visibility streaming

The process is initiated by the client

• Buffering step:

– the starting cell is downloaded in a synchronous way together with its PVOS

• During navigation:

– Prediction is made about the next visited cell

– This cell and its adjacent cells are downloaded together with their PVOS in an asynchronous way

9

9Web3D 2006

Feature extraction of the facades

Edge detection Vectorization Rendering

10

10Web3D 2006

Edge detection

Classical approach• Gradient approximation using a convolution kernel: Sobel [Pingle 1969],

Roberts [1965], Prewitt [1970]

• Simple thresholding

• The contours are not 1 pixel wide, needed for vectorization

Canny edge detector• Gaussian blur

• Gradient computation

• Non-maximal suppression: insures 1 pixel wide contours

• Hysteresis thresholding

11

11Web3D 2006

Pixel chaining

1. Contours

2. Junction and end-points detection

3. Graph initialization

4. Graph filling

5. Loop processing

6. Vectorization

12

12Web3D 2006

Vectorization

Vectorization of the pixel chains• Minimization of the distance between the approximation and the curve

[D.H. Douglas 1973], [de Figueiredo 1995]

• Best approximation of the angular [Dunham 1986] or the curvature [Asada and Brady 1986]

• Minimization of the area between the approximation and the curve [Wall and Danielsson 1984]

Post processing and cleaning: simple heuristics• Junction correction

• Noise removing

• Line straightening

13

13Web3D 2006

Rendering of lines

Lines are transmitted as IndexedLineSet

Lines are rendered with a constant width of one pixel

At some point the screen is saturated with lines

• Need to remove some lines

14

14Web3D 2006

Rendering of lines

Level of details• Depending on the distance the

façade is viewed

• We draw a subset of the lines

Advantages• Reducing the line density on

screen

• Less geometric primitives are drawn, rendering time is improved

Line selection criterion• Based on line length

• Same lines are used between each level

• Visual continuity between the different levels

15

15Web3D 2006

Results

We compare our approach to textured rendering• On data size

• Rendering speed

Hardware: DELL Axim X50V• 624 MHz ARM processor

• 64 MB of RAM memory

• Intel 2700 GPU

Software based on OpenGL-ES standard• Magellan framework for WinMobile (VRML/X3D based streaming & rendering)

• Intel SDK for hardware acceleration

• Software-based library developed by Hybrid Graphics

16

16Web3D 2006

Results

Initial data set

• Fixed viewpoint

– Low resolution pictures (250x320): French yellow pages website

– High resolution pictures (3008x2000): digital camera

• City walkthrough

– Model and textures maps of Bordeaux [Hachet Guitton 2001]

– Small texture map sizes (~64x64)

17

17Web3D 2006

Fixed viewpoint

Data size

• Vector data smaller than original pictures

• Drawn images smaller than original pictures

60.56 KB37.55 KB387.76 KBHigh resolution

3.13 KB4.16 KB8.55 KBLow resolution

Drawn images PNGVector dataJPEG

18

18Web3D 2006

Fixed viewpoint

Rendering speed

• Using low resolution images– On software-based rendering

– texture ~ lines + LOD: 6 fps– Using hardware acceleration

– Texture: 39.6 fps – Lines + LOD: 17.3 fps (because no display lists on GL|ES)

• Using high resolution images– Still practicable using lines + LOD: 3 fps– Results for lines are better using software-based rendering

– no transfer on the bus for SW based

• Significant better performances using LOD than full line rendering

19

19Web3D 2006

Moving viewpoint: the complete city model

Record statistics during a city walkthrough• Memory used

• Frame rate

• Rendering time

• Scene graph traversal

• Download bitrate

Using different models• Geometry only as reference

• Textured based model

• Model together with its full line appearance description

• Model that takes advantage of the presented LOD technique

20

20Web3D 2006

Moving viewpoint: the complete city model

Average size of downloaded files

• Textures: 35.61 KB

• Lines: 7.55 KB

Memory occupation 2 MB smaller for the lines than for textures

21

21Web3D 2006

Moving viewpoint: the complete city model

Rendering speed (SW vs HW)

• Textures: 2.18 fps / 11.97 fps• Lines: 1.17 fps / 1.25 fps• Lines + LOD: 2.28 fps / 5.45 fps

In software

• Lines + LOD ~ textures (because of their small resolution)

In hardware

• Texture gets better result (because of bus transfers)

22

22Web3D 2006

Moving viewpoint: the complete city model

The rendering is divided in two steps• Scene graph traversal time• Rendering time

Scene graph traversal time• Texture: 12 ms• Lines + LOD: 61 ms• The scene graph is simpler for the texture-based rendering

Rendering time• With hardware acceleration:

– Texture: 90 ms – Lines + LOD: 258 ms

• With software library:– Texture: 388 ms– Lines + LOD: 347 ms

Improve the scene graph traversal time by implementing a dedicated node

23

23Web3D 2006

Moving viewpoint: the complete city model

Bitrate measurement: the rate the data are sent by the server to the moment they are available in the client scene graph.1. Data transmission over the network

2. Data decompression

3. VRML97 parsing and conversion to fixed point

4. Node initialization

Resulting bitrate• for the lines: 17306 bps

• for the textures: 26902 bps

• Lines bitrate lower than map’ ones due to conversion to fixed point

• Might need fixed point fields types in VRML/X3D

24

24Web3D 2006

Conclusion

Our approach• Based on expressive rendering

• Features lines are extracted from original texture

• The resulting data set is smaller than the original one

Experiments• On PDA

• The result conveys the required information

• The amount of data to transmit is greatly reduced

• We can obtain interactive rendering even using software 3D rendering

25

25Web3D 2006

Future work

Feature lines extraction

• Improve its robustness and efficiency

• Perform a cognitive study on scene recognition

Rendering system

• Use up-coming vector processor based on OpenVG standard

• Allow integration of larger range of NPR styles

26

26Web3D 2006

Thank you