surface light fields for 3d photography
Post on 06-Jan-2016
24 Views
Preview:
DESCRIPTION
TRANSCRIPT
-
Surface Light Fieldsfor 3D PhotographyDaniel Wood Daniel Azuma Wyvern AldingerBrian Curless Tom DuchampDavid Salesin Werner Stuetzle
-
3D PhotographyGoalsRendering and editing
InputsPhotographs and geometry
RequirementsEstimation and compression
-
View-dependent texture mappingDebevec et al. 1996, 1998Pulli et al. 1997
-
View-dependent texture mappingDebevec et al. 1996, 1998Pulli et al. 1997
-
Two-plane light fieldLevoy and Hanrahan 1996Gortler et al. 1996
-
Surface light fieldsWalter et al. 1997Miller et al. 1998Nishino et al. 1999
-
Lumisphere-valued texture mapsLumisphere
-
OverviewDataacquisitionEstimationandcompressionRenderingEditing
-
OverviewDataacquisitionEstimationandcompressionRenderingEditing
-
Scan and reconstruct geometryReconstructed geometryRange scans(only a few shown . . .)
-
Take photographsCamera positionsPhotographs
-
Register photographs to geometryGeometryPhotographs
-
Register photographs to geometryUser selected correspondences (rays)
-
Parameterizing the geometryBase meshScanned geometryMap
-
Sample base mesh facesBase meshDetailed geometry
-
Assembling data lumispheresData lumisphere
-
OverviewDataacquisitionEstimationandcompressionRenderingEditing
-
Pointwise fairingFaired lumisphereData lumisphere
-
Pointwise fairing resultsInput photographPointwise faired(177 MB)
-
Pointwise fairingMany input data lumispheresMany faired lumispheres
-
CompressionSmall set of prototypes
-
Compression / EstimationSmall set of prototypesMany input data lumispheres
-
Reflected reparameterization
-
Reflected reparameterization
-
Reflected reparameterization
-
Reflected reparameterizationBeforeAfter
-
Median removal+ReflectedMedian(diffuse)Median-removed(specular)+
-
Median removalMedian valuesSpecularResult
-
Function quantizationCodebook of lumispheresInput data lumisphere
-
Lloyd iterationInput data lumispheres
-
Lloyd iterationCodeword
-
Lloyd iterationPerturb codewords to create larger codebook
-
Lloyd iterationForm clusters around each codeword
-
Lloyd iterationOptimize codewords based on clusters
-
Lloyd iterationCreate new clusters
-
Function quantization resultsInput photographFunction quantized(1010 codewords, 2.6 MB)
-
Principal function analysisSubspace of lumispheresInput data lumispherePrototype lumisphere
-
Principal function analysisApproximating subspacePrototype lumisphere
-
Principal function analysis
-
Principal function analysis
-
Principal function analysis resultsInput photographPFA compressed(Order 5 - 2.5 MB)
-
Compression comparisonPointwise fairing(177 MB)Function quantization(2.6 MB)Principal functionanalysis (2.5 MB)
-
Comparison with 2-plane light field(uncompressed)Pointwise-fairedsurface light field (177 MB)Uncompressedlumigraph / light field (177 MB)
-
Comparison with 2-plane light field(compressed)Compressed (PFA)surface light field (2.5 MB)Vector-quantizedlumigraph / light field (8.1 MB)
-
OverviewDataacquisitionEstimationandcompressionRenderingEditing
-
View-dependent level-of-detail
-
Render texture domain and coordinates in false color
-
Evaluate surface light field
-
Interactive rendererscreen capture
-
OverviewDataacquisitionEstimationandcompressionRenderingEditing
-
Lumisphere filteringOriginal surface light fieldGlossier coat
-
Lumisphere filtering
-
Rotating the environmentOriginal surface light fieldRotated environment
-
DeformationOriginalDeformed
-
Deformation
-
SummaryEstimation and compressionFunction quantizationPrincipal function analysis
RenderingFrom compressed representationWith view-dependent level-of-detail
EditingLumisphere filteringGeometric deformations and transformations
-
Future workBetter geometry-to-image registration
More complex surfaces (mirrored, refractive, fuzzy) under more complex illumination
Derive geometry from images
Combining FQ and PFA
-
AcknowledgementsMarc Levoy and Pat Hanrahan(Thanks for the use of the Stanford Spherical Gantry)
Michael Cohen and Richard Szeliski
National Science Foundation
-
The end
-
Geometry (fish)Reconstruction: 129,000 facesMemory for reconstruction: 2.5 MBBase mesh: 199 facesRe-mesh (4x subdivided): 51,000 facesMemory for re-mesh: 1 MBMemory with view-dependence: 7.5 MB
-
Light field data and rep (fish)Time to acquire: 1 hourInput images: 661Raw data size: ~500 MBLumisphere representation: 3 times subdivided octehedronLumisphere size: 258 directions
-
Lumigraph (fish)Lumigraph images: 400x400Lumigraph viewpoints per slab: 8x8Number of slabs: 6Lumigraph size (w/o geom): 184 MBLumigraph VQ dimension: 16384Lumigraph VQ codewords: 2x2x2x2x3Compressed size (w/o geom): 8.1 MB
-
Compression (fish)Pointwise faired: Memory = 177 MBRMS error = 9FQ (2000 codewords)Memory = 3.4 MBRMS error = 23PFA (dimension 3)Memory = 2.5 MBRMS error = 24PFA (dimension 5)Memory = 2.9 MBRMS error = ?
-
Pre-processing times (fish)Compute times on ~450 MHz P-IIIRange scanning time: 3 hoursGeometry registration: 2 hoursImage to geometry alignment: 6 hoursMAPS (sub-optimal): 5 hoursAssembling data lumispheres: 24 hoursPointwise fairing: 30 minutesFQ codebook construction (10%): 30 hoursFQ encoding: 4 hoursPFA codebook construction (0.1%): 20 hoursPFA encoding: 2 hours
- Breakdown and rendering (fish)For PFA dimension 3Direction mesh: 11 KBNormal maps: 680 KBMedian maps: 680 KBIndex maps: 455 KBWeight maps: 680 KBCodebook: 3 KBGeometry w/o view dependence:
-
Construct codebook using Lloyd iterationIterate until convergence:
Assign all data lumispheres to closest codeword, forming clusters.
Compute new codeword for each cluster by cluster-wise fairing.
Then split all codewords and start over.
-
Data extrapolationPhotographSurface light field
-
Comparison with 2-plane light field(uncompressed)Pointwise-fairedsurface light field (177 MB)Uncompressed2-plane light field (177 MB)
-
Comparison with 2-plane light field(compressed)Principal function analysissurface light field (2.5 MB)Vector-quantized2-plane light field (8.1 MB)
-
DetailsInput photographPointwise fairing(177 MB)Function quantization (3.4 MB)Principal function analysis (2.5 MB)
top related