voxel-based surface flattening nahum kiryati* dept. of electrical engineering – systems tel aviv...
TRANSCRIPT
![Page 1: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/1.jpg)
VOXEL-BASED SURFACE FLATTENING
Nahum Kiryati*Dept. of Electrical Engineering – Systems
Tel Aviv University
*Joint work with Ruth Grossmann and Ron Kimmel.
![Page 2: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/2.jpg)
SURFACE FLATTENING
• Some distortion is usually unavoidable
• Need to define the most important property to preserve (area, angles,...)
• Use cuts (interruptions) to selectively reduce distortion
Transform a surface from 3D space to 2D plane.
Peters Projection: area accuarte Cuts reduce distortion
![Page 3: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/3.jpg)
Applications: Medical
• Cortical surface flattening for visualization, registration, etc.
• Virtual colonoscopy
Gray-white matter interface
Courtesy of Gabriele Lohmann, MPG Leipzig
![Page 4: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/4.jpg)
Applications: Computer Graphics
• Flatten the surface• Map the 2D pattern onto the flattened surface• Map the flattened surface back to 3D
Texture Mapping
From www.okino.com
Texture Mapping: Projecting a planar pattern onto a 3D surface
![Page 5: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/5.jpg)
Some previous works
• E.L. Schwartz, A. Shaw and E. Wolfson, “A Numerical Solution to the Generalized Mapmaker’s Problem: Flattening Nonconvex Polyhedral Surfaces”, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 11, pp. 1005-1008, 1989.
• C. Bennis, J.M. Vezien and G. Iglesias, “Piecewise Surface Flattening for Non-Distorted Texture Mapping”, Computer Graphics, Vol. 25, pp. 237-247, 1991.
• H.A. Drury, D.C. Van Essen, C.A. Anderson, C.W. Lee, T.A. Coogan and J.W. Lewis, “Computerized Mappings of the Cerebral Cortex: A Multiresolution Flattening Method and Surface-Based Coordinate System”, J. of Cognitive Neuroscience, Vol. 8, pp. 1-28, 1996.
• S. Angenent, S. Haker, A. Tanenbaum and R. Kikinis, “Conformal Geometry and Brain Flattening”, Proc. 2nd Intl. Conf. on Medical Image Computing and Computer Assisted Intervention (MICCAI), Cambridge, England, pp. 269-278, 1999.
• A.M. Dale, B. Fischl and M.I. Sereno, “Cortical Surface-Based Analysis. II: Inflation, Flattening and a Surface-Based Coordinate System”, NeuroImage, Vol. 9, pp. 195-207, 1999.
• S. Haker, S. Angenent, A. Tanenbaum, R. Kikinis, G. Sapiro and M. Halle, “Conformal Surface Parameterization for Texture Mapping”, IEEE Trans. on Visualization and Computer Graphics, Vol. 6, No. 2, 2000.
![Page 6: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/6.jpg)
New Method: Unique Features
Voxel data
From www.chez.com
• Operates on voxel data: Triangulation not necessary
• Optimal (global preservation of geodesic distances)
• Solid mathematical basis. Analytic solution (in principle)
• Computationally efficient
![Page 7: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/7.jpg)
Our Approach to Surface Flattening
• Mark N (very many) points on the 3D surface.
• Estimate the geodesic distances between each point and all others (how?)
• Optimally map the N points onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding 3D geodesic distances (how?)
• Regard the N points as control points, and map the rest by interpolation.
![Page 8: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/8.jpg)
Crucial problem #1:
• Represent the surface as a weighted graph:
• The weight of an edge depends on the link type
• Minimal geodesic distance ~ shortest path in graph
Graph is sparse: very fast algorithm
Kiryati & Szekely, 1993 :
Estimate the minimal (geodesic) distance between points on a 3D surface.
Accurate and efficient solution for surface represented by voxels.
edgelinkvertexvoxel ,
direct, minor diagonal, major diagonal
Question: what weights should we use?
![Page 9: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/9.jpg)
Crucial problem #1:
• Represent the surface as a weighted graph:
• The weight of an edge depends on the link type
• Minimal geodesic distance ~ shortest path in graph
Graph is sparse: very fast algorithm
Kiryati & Székely, 1993 :
Estimate the minimal (geodesic) distance between points on a 3D surface.
Accurate and efficient solution for surface represented by voxels.
edgelinkvertexvoxel ,
direct, minor diagonal, major diagonal
Question: what weights should we use?
Wrong answer: 3,2,1
![Page 10: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/10.jpg)
Bias in length estimation (2D example)
Unbiased, minimum MSE* length estimation:
• Continuous curve (length L) Digital curve
• Digital curve Chain code (8-connected)
• Estimate length:
GIQ
links) diagonal(#2links)direct (#1ˆ L
links) diagonal(#links)direct (#ˆ21 L
*Unbiased and minimum MSE for straight lines.
Unbiased for lines wonderful for curves (errors cancel out).
![Page 11: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/11.jpg)
Kiryati & Kübler, 1992:
Unbiased, minimum MSE length estimation for 3D curves
• RMS error for straight lines: 2.88%
• Straight lines are the worst case
(much better with curves)
diagonalsmajor #1.615 diagonalsminor #289.1linksdirect #9016.0ˆ L
Use the weights (0.9016, 1.289, 1.615) in the algorithm for finding shortest paths (geodesics) and minimal distances on digitized surfaces.
Verwer, Beckers & Smeulders
![Page 12: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/12.jpg)
Crucial problem #1:
Estimate the minimal distance between points on a digitized 3D surface.
• Solved (Kiryati & Székely, 1993)
![Page 13: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/13.jpg)
Our Approach to Surface Flattening
Mark N (very many) points on the 3D surface.
Estimate the geodesic distances between each point and all others (how?)
Optimally map the N points onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding 3D geodesic distances (how?)
Regard the N points as control points, and map the rest by interpolation.
Reminder:
![Page 14: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/14.jpg)
Crucial problem #2:
Given N points on a surface in 3D, map them onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding geodesic distances on the 3D surface.
![Page 15: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/15.jpg)
Crucial problem #2:
Given N points on a surface in 3D, map them onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding geodesic distances on the 3D surface.
Special case of Multi-Dimensional Scaling (MDS)
![Page 16: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/16.jpg)
Goal:
Place N points in the 2D plane, such that the sets and are as similar as possible.
ijD ijd
Objective functions
NameDefinitionAlgorithm
Stressiterative
SStressiterative
Strainsomething elseanalytic solution!
ij
ijij dD 2)(
ij
ijij dD 222 )(
The analytic solution for minimal strain is called classical scaling (Borg & Groenen, 1997).
![Page 17: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/17.jpg)
Classical Scaling
X Coordinate Matrix: (N x 2) matrix with the (x,y) coordinates of N points in the plane1 (unknown)
B Scalar Product Matrix: (N x N) matrix (unknown)
d* Squared Distances Matrix: (N x N) matrix containing the squared
Euclidean distances between points in the plane (unknown)
TXXB
2ijd)(* Xd
Suppose that d* is known. It can be shown that:
where and 1 is a vector of 1’s.
JJdB *
2
1
TIJ 11N
1
Having B, X can be found via eigendecomposition:
)()( TT QQQQB 2
1
2
1
holds exactly two (positive) non-zero eigenvalues. So 2
1
22QX
where and are (2 x 2) and (N x 2) submatrices.2 2Q
1Column centered: coordinate system origin at the point-set centroid.
![Page 18: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/18.jpg)
Classical Scaling (cont’d)
JJDB *ˆ2
1
2
1
22ˆ QX
But: we don’t have the Squared Distances Matrix d*, since interpoint distances in the plane are unknown.
We do have the interpoint geodesic distances on the 3D surface.
ijD
Algorithm
1. Create D* (Squared Geodesic Distance Matrix)
2. Compute the estimated Scalar Product Matrix:
3. Compute the eigendecomposition of up to rank 2:
4. The flattened Coordinate Matrix is:
TQQB 222ˆ B̂
Classical scaling minimizes the error metric (strain).
Zero for a true Scalar Product Matrix obtained from a true Squared Distance Matrix d* containing
squared planar Euclidean distances.
Partial eigendecomposition (up to rank 2) is easy and cheap (power method).
2ˆˆF
T BXX TXXB
![Page 19: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/19.jpg)
Our Approach to Surface Flattening
Mark N (very many) points on the 3D surface.
Estimate the geodesic distances between each point and all others.
Optimally map the N points onto the 2D plane, such that the 2D Euclidean interpoint distances will be as close as possible to the corresponding 3D geodesic distances.
Regard the N points as control points, and map the rest by interpolation.
Reminder:
We used N ~ 1000 control points, and mapped the rest using radial basis function interpolation.
Any reasonable interpolation method can be used.
![Page 20: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/20.jpg)
Curled planar surface (synthetic)
Unfolded
Euclidean distance (2D) vs. geodesic distance (3D)
![Page 21: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/21.jpg)
Curled planar surface (synthetic)
Unfolded
Euclidean distance (2D) vs. geodesic distance (3D)
Can you figure out why errors over small distances are larger than errors over large distances in this case?
![Page 22: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/22.jpg)
Curled planar surface (synthetic)
Unfolded
Euclidean distance (2D) vs. geodesic distance (3D)
Can you figure out why errors over small distances are larger than errors over large distances in this case?
On this surface, many short geodesic paths are straight, or nearly straight. Straight lines are the worst case for the geodesic distance estimator. Over long distances, paths are not straight, and errors cancel out.
![Page 23: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/23.jpg)
A depth image of a human face. Euclidean distance (2D) vs. geodesic distance (3D)
![Page 24: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/24.jpg)
A depth image of a human face. Euclidean distance (2D) vs. geodesic distance (3D)
The flattened surface is not shown, because the depth image has no texture.
![Page 25: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/25.jpg)
A depth image of a human face. Euclidean distance (2D) vs. geodesic distance (3D)
Errors are larger than in the previous example, because this surface is not developable – it cannot be flattened without some distortion.
![Page 26: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/26.jpg)
Textures
![Page 27: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/27.jpg)
Texture Mapping via Surface Flattening
![Page 28: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/28.jpg)
Texture Mapping via Surface Flattening
![Page 29: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/29.jpg)
Highlights
• New surface flattening algorithm
• Operates on voxel data. Triangulation is not necessary.
• Distance measurements on 3D surface + classical scaling.
• Global distance preservation
• Essentially analytic solution: Global optimum guaranteed
• Surface flattening, texture mapping
![Page 30: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/30.jpg)
Appeared
![Page 31: VOXEL-BASED SURFACE FLATTENING Nahum Kiryati* Dept. of Electrical Engineering – Systems Tel Aviv University * Joint work with Ruth Grossmann and Ron Kimmel](https://reader031.vdocuments.net/reader031/viewer/2022032517/56649c815503460f94939ce2/html5/thumbnails/31.jpg)
Companion paper
• Flattening with triangulated data
• Geodesic distance measurements via Fast Marching on Triangulated Domains + classical scaling
• State of the art texture mapping quality