web3d 2012 - los angeles, august 2012

www.crs4.it/vic/

Web3D 2012 - Los Angeles, August 2012

Adaptive Quad Patches: an Adaptive Regular Structure for Adaptive Quad Patches: an Adaptive Regular Structure for Web Distribution and Adaptive Rendering of 3D ModelsWeb Distribution and Adaptive Rendering of 3D Models

Enrico Gobbetti1, Fabio Marton1, Marcos Balsa Rodríguez1, Fabio Ganovelli2 and Marco Di Benedetto2

1 CRS4 Visual Computing Group, Sardinia Italy2 ISTI-CNR Visual Computing Laboratory, Pisa Italy


Marcos Balsa, Adaptive Quad Patches

Context• 3D scanning systems has

become commodity hardware– Easy to use– Fast– Portable

• Increasing virtual world creation– Museums– Virtual shops and galleries– Games

• WebGL– Mobile devices– Web integration– Internet– More users



Motivation• 3D models

– Heavy– Non-trivial to render– Non-linear inspection

• Constrained environments– Lightweight– Interpreted– Scripted

Reuse components already optimized!

Javascript



Pipeline overview

• Unattended software pipeline• Compact multiresolution representation

4



Adaptive Quad Patches – Approach• Remote rendering

– Compact multiresolution representations• Storage• Distribution• Real-time rendering

• Mesh reparametrization– Point cloud -> quad mesh– Multiresolution– Tightly packed texture atlas

• Low CPU overhead– GPU accelerated adaptive tessellation algorithm

• WebGL based– Portable platforms



Related work

• General solutions available for Desktop environments [Yoon et al. 2008]• Mesh compression – MPEG-4 [Jovanova et al. 2008]• View dependent LOD and progressive streaming [Hoppe 1997]

– Compute view dependent triangulation each frame -> CPU bound

• Surface patches [Cignoni et al. 2004]– Effective in terms of speed -> require non-trivial data structures and techniques for

decompression

• Geometry images [Gu et al. 2002]– Exploit current GPU capabilities / optimized libraries for compression and streaming of images

6



Related work• Quad remeshing

– Single-disk parametrization [Floater and Hormann 2005]• Simplest -> only genus-0 almost zero Gaussian curvature

– Base mesh to parametrize the model [Petroni et al. 2010]• Quad-based domain -> tight packing and simpler handling of chart boundaries

• Detail rendering– GPU raycasting [Oliveira et al. 2000]– Displacement mapping in GPU [Shiue et al. 2005]– Semi-uniform adaptive patch tessellation [Dyken et al. 2009]

7



Pipeline overview

• Unattended software pipeline• Compact multiresolution representation

8



• Generate clean manifold triangle mesh– Poisson reconstruction [Kazhdan et al. 2006]– Remove topological noise

• Discard connected components with too few triangles

• Parameterize the mesh on a quad-based domain

– Isometric triangle mesh parameterization • Abstract domains [Pietroni et al. 2010]

– Remap into a collection of 2D square regions

• Resample each quad from original geometry

– Associates to each quad a regular grid of samples (position, color and normal)

Pre-processing (Reparameterization)

9



Pre-processing (geometry representation)

• Coarse geometry– Quads (4 corners)

• Detailed geometry– Displacement Texture

• With respect to corner interpolation at (u,v) coordinates

10



• Collection of variable resolution quad patches

– Coarse representation of the original model• Multiresolution pyramids

– Detail geometry– Color – Normals

• Shared border information– Ensure connectivity

Pre-processing (Multiresolution)



• Ensure continuity (detail geometry, color, normals)

– Special filtering maintains shared information on borders

• Per patch quantization (quad ranges)– Max/min at patch corners -> Avoid discontinuities

• PNG or S3TC compression– Geometry -> lossless PNG– Color and normals could be lossy

• Tiled storage– N pyramids– Separated mipmap levels (per pyramid)– Mipmap levels splitted into tiles (e.g. 512x512)

Pre-processing (Multiresolution - filtering)



• 4 3D corners with 16 bit / component• 4 Corners quantization ranges

– Bilinear interpolation of quantization ranges ensure continuity among quads.

• Patches of quantized 3D displacements– With respect to base quad

Adaptive rendering (Quad geometry information)



Adaptive rendering (LOD selection)

• For each quad– For each edge

• Evaluate edge LOD

– Quad LOD = max edge LOD (available)– Edge LOD = Min(edge LOD, neighbor

patches available LOD)– If LOD not available

• Post request• Use finest available LOD• Priority = Required LOD - Available LOD



Adaptive rendering (GPU Storage)

• 1 VBO per resolution level reused• 1 stream of (u,v,e) information per vertex• (e) determines the vertex (0,1,2,3, or 4 for inner vertices)

• K preallocated incomplete (filled on demand) texture mipmaps of– Displacements – Normals– Colors

• Texture with quad ranges (dequantization)• Texture with edge LODs



Adaptive rendering (Pyramid Rendering)

• Bind pyramid textures• For each quad

– Draw VBO with proper level triangulation– Shaders

• Vertex: • Dequantize using (u,v) coordinates• Snap edge vertices (u,v,e)• Displacement mapping

• Fragment• Color• Normal• Shading.



Adaptive rendering (Shader Geometry Dequantization)

• Base offset = quad corners Interpolation• Interpolate dequantization ranges• Dequantize displacement• Sum displacement to base offset



Rendering example

Patches Levels Shading



Adaptive streaming• Server

– Tile repositories– No processing– Berkeley DB + Apache2

• Client (WebGL)– Priority = desired LOD – available LOD– Separate thread for receiving data

• PNG decompression

– Insertion into pyramid when ready

19



Results (preprocessing)

20

• Adaptive rendering– 1 pixel accuracy – 37 fps (min 13 fps)

• Network streaming– Required 312Kbps (max 2.8Mbps) for no delay– ADSL 8Mbps – 2 s fully refined model from scratch



Results

21

• Tests configuration– 1.6Ghz laptop– 4GB RAM– Nvidia Geforce GTX 260M with 1GB VM– Linux Gentoo 2.6.39– Chromium browser 19.0.1084.24b– ADSL 8MBps



Conclusions• Effective creation and

distribution system

– Fully automatic– Compact, streamable and

renderable 3D model representations

– Low CPU overhead– WebGL

• Desktop• Mobile

• Limitations and future work

– Closed objects with large components

– Visual approximation (lossy)– Explore more aggressive

compression techniques– Occlussion culling– More sophisticated

shading/shadowing techniques

22



Questions and contacts• CRS4 - Visual Computing Group

http://vic.crs4.it

• ISTI-CNR - Visual Computing Lab http://vcg.isti.cnr.it

• Speaker: Marcos Balsa [email protected]

• The DIVA project http://diva-itn.ifi.uzh.ch/

This research is partially supported by the EU FP7 Program under the DIVA project (290277).

23

web3d 2012 - los angeles, august 2012

Documents