texture synthesis on [arbitrary manifold] surfaces presented by: sam z. glassenberg* * several...
TRANSCRIPT
![Page 1: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/1.jpg)
Texture Synthesis on [Arbitrary Manifold]
SurfacesPresented by:Presented by:
Sam Z. Glassenberg*Sam Z. Glassenberg*
* Several slides borrowed from Wei/Levoy presentation
![Page 2: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/2.jpg)
Topics Investigated Thus Far
• 2D Texture Mapping (CS318)• 3D Textures (Hypertexture)• 2D Texture Synthesis
![Page 3: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/3.jpg)
+
Synthesize a texture on a surface by coloring mesh vertices
Input Texture Input Mesh Result
Today’s Goal
![Page 4: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/4.jpg)
Vertex Painting
• Advantages– Speed
• No additional transformation required
– Simplicity– Uniformly
distributed across geometry
• Disadvantages– Texture resolution
dependent on mesh complexity
![Page 5: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/5.jpg)
2 SIGGRAPH Papers:
1. Texture Synthesis on Surfaces, by Greg Turk
2. Texture Synthesis over Arbitrary Manifold Surfaces, by Li-Yi Wei and Marc Levoy
![Page 6: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/6.jpg)
![Page 7: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/7.jpg)
Desirable Properties
• Share advantages of 2D algorithm– Quality– Efficient– General– Easy to use
• Minimum distortion• Minimum discontinuity
![Page 8: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/8.jpg)
Differences betweenImages and Meshes
imageimage meshmesh
Synthesis Order
Pixels/Vertices
Local Orientation(Vector field)
?
?scanline
u
v
grid
Neighborhood
normal
? ?tangent bitangent
![Page 9: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/9.jpg)
Wei/Levoy Solution
imageimage meshmesh
Synthesis Order
Pixels/Vertices
Local Orientation(Vector field)
?
?scanline
u
v
grid
Neighborhood
normal
? ?tangent bitangent
Wei and LevoyWei and Levoy
mesh re-tiling[Turk’92]
user-specifiedrelaxationrandom
random
flattening/resampling
![Page 10: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/10.jpg)
Turk Solution
TurkTurk
repulsion[Turk’91]
User specified/interpolated
sweeping
Surface marching
imageimage meshmesh
Synthesis Order
Pixels/Vertices
Local Orientation(Vector field)
?
?scanline
u
v
grid
Neighborhood
normal
? ?tangent bitangent
![Page 11: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/11.jpg)
Both Papers ExtendFast Texture Synthesis using Tree-structured Vector Quantization
Specifically by generalizing their definition of “search neighborhoods”
to apply to meshes
![Page 12: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/12.jpg)
Texture Synthesis byNeighborhood Search
noise
Input pyramid
noise
Output pyramid
Search
Copy
![Page 13: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/13.jpg)
Surface Texture Synthesis byNeighborhood Search
(Wei/Levoy)
Input pyramid Output pyramid
Search
![Page 14: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/14.jpg)
Process
1. Build image/mesh pyramids
2. Assign texture orientation/Computation order
3. Generate texture
![Page 15: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/15.jpg)
What aspects of image pyramids must we maintain in
mesh pyramids?
• Uniform density• Power-of-two complexity
differences between levels
![Page 16: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/16.jpg)
Image & Mesh Pyramids
Mesh Retiling [Turk’92]
![Page 17: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/17.jpg)
2 Turk-ish Methods for Mesh Retiling
• Turk ’92, used by Wei/Levoy– Uniformly
distributes mesh vertices
– Requires “shooting normals” to move between levels
• Turk ’91, used by Turk– Uniformly
distributes mesh vertices
– Maintains parent/child relationship between levels
![Page 18: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/18.jpg)
Retiling Density
24576 vertices 73728 vertices
![Page 19: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/19.jpg)
Turk ‘91• Create a mesh hierarchy in which mesh
Mk = (Vk, Tk) is defined by its Vertices and Triangles– For the lowest mesh in the hierarchy, place n
points on the surface– Use repulsion to distribute points easily– Add 3n points to make the next level– Repeat
• Connect points by projecting nearby points onto a tangent plane
• Perform Delaunay triangulation to reduce triangles
![Page 20: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/20.jpg)
Now, we need to determine orientation…
In Turk’s method, we use this orientation to determine the computation order (Surface
Sweeping)
In Wei/Levoy’s method, orientation is needed to “flatten” the neighborhoods.
![Page 21: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/21.jpg)
Texture Orientation
• Generate a coordinate frame• Three orthogonal axes
– s (texture right)– t (texture up)– n (surface normal)
![Page 22: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/22.jpg)
Texture Orientation
• Methods for orienting textures – user-specified (Turk)– random (Wei/Levoy)– smooth or symmetric (Wei/Levoy)
![Page 23: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/23.jpg)
Texture Orientation (User-specified)
![Page 24: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/24.jpg)
4-way symmetric texture 4-way symmetric vector field
Texture Orientation (Symmetry)
![Page 25: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/25.jpg)
Texture Orientation
random 2-way symmetry4-way symmetry
![Page 26: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/26.jpg)
Texture Orientation (Relaxation)
• Minimize an error function
![Page 27: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/27.jpg)
Results (Wei/Levoy):Random Orientation
![Page 28: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/28.jpg)
Results (Wei/Levoy):Other Orientations
Random User-specified Relaxation
2-way symmetry
4-way symmetry
![Page 29: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/29.jpg)
Texture Synthesis on Surfaces:Wei/Levoy Style
![Page 30: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/30.jpg)
Synthesis
![Page 31: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/31.jpg)
Synthesis : 2 Lowest Levels
![Page 32: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/32.jpg)
Synthesis : Lowest Level
Random copy
![Page 33: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/33.jpg)
Synthesis Pass1 : Extrapolation
Search
Copy
Shooting normal
![Page 34: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/34.jpg)
Mesh Neighborhood
Resample
Compare
Resampled Grid
2D Patch
3D Patch
Flatten (Maillot’93)
![Page 35: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/35.jpg)
Neighborhood Flattening
• Project the triangles adjacent to p onto p’s local texture coordinate system
• Add triangles one-at-a-time until neighborhood template is covered
![Page 36: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/36.jpg)
Neighborhood Comparison
compare ?
![Page 37: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/37.jpg)
Synthesis Pass 2 :Full Neighborhood
Search
Copy
![Page 38: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/38.jpg)
Multiresolution Synthesis
![Page 40: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/40.jpg)
Turk’s Surface Sweeping
• Select an anchor vertex A• Assign s(v) = sweep distance to vertex v along
the vector field from A (for all v)• Consensus orientation b/w 2 vertices v and w:
Ovw = (O(v) + O(w))/2
• Calculate a new s(v) as a weighted average that its neighboring values dictate it should have.
• Visit vertices in order by sweep distance
![Page 41: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/41.jpg)
Turk’s Texture Synthesis (Pseudocode)
I = Input textureN(v) = Neighborhood around vM(a,b) = Neighborhood around (a,b)D(M,N) = Match Value (sum of squared differences)
![Page 42: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/42.jpg)
Turk’s Mesh Neighborhoods
• r = average distance between mesh vertices• O(v) = Surface tangent vector• P(v) = O(v) rotated 90o about surface normal• Together, O and P make a coordinate frame !
u
vnormal
P O• Now, we can traverse the surface by
point repelling
![Page 43: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/43.jpg)
Point Repelling
• Use color interpolation to determine color at current surface point
• Move r in the direction of O or P• When an edge is reached, fold the
path over the next polygon
![Page 44: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/44.jpg)
Results
smooth 4-way symmetry random
![Page 45: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/45.jpg)
Results
Surface displacement
smooth 2-way symmetry 2-way symmetry
![Page 47: Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation](https://reader030.vdocuments.net/reader030/viewer/2022032701/56649c815503460f94939d36/html5/thumbnails/47.jpg)
Summary of Differences
Turk’s approach Our approach
Vector field
Traversal order
Neighborhood
Mesh hierarchy
smooth random, symmetric
sweeping random
surface marchingflattening/resampling
explicit parent/child shooting normal