siggraph asia 2011 preview seminar - material editing - yoshihiro kanamori 2011 nov. 25
TRANSCRIPT
![Page 1: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/1.jpg)
SIGGRAPH Asia 2011 Preview Seminar- Material Editing -
Yoshihiro Kanamori2011 Nov. 25
![Page 2: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/2.jpg)
Material Editing
• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)
• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)
• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),
Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)
• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),
Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)
![Page 3: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/3.jpg)
Material Editing
• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)
• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)
• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),
Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)
• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),
Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)
![Page 4: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/4.jpg)
Material Matting
• Given Spatially-Varying BRDFs (SVBRDFs), separates them into foreground (F) / background (B) layers, and allows editing
Input SVBRDF and separation Edited components and recomposited SVBRDF
![Page 5: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/5.jpg)
Matting?
• Standard matting equation
– Many methods exist for image matting
• This method peels apart SVBRDF V as layers Li one by one
– Extracted regions are completed by texture synthesis
![Page 6: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/6.jpg)
Overview
• Interactive system– Users draws scribbles to support separation,
like recent image matting methods
![Page 7: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/7.jpg)
Formulation
• Based on a Bayesian approach, i.e., Maximum A Posteriori (MAP) estimate
• Take –log( ), and for α minimize
– Likewise, for F and B
![Page 8: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/8.jpg)
![Page 9: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/9.jpg)
Material Editing
• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)
• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)
• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),
Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)
• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),
Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)
![Page 10: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/10.jpg)
Physically-Based Interactive Bi-Scale Material Design
• Not available now (coming soon)
![Page 11: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/11.jpg)
Material Editing
• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)
• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)
• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),
Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)
• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),
Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)
![Page 12: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/12.jpg)
AppGen: Interactive Material Modeling from a Single Image
• Extracts {specular/diffuse/roughness components, height field, and normal map} from a single image, for relighting applications
• Contribution: proposal of the whole system– Technically, diffuse extraction and normal map
construction are improved
![Page 13: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/13.jpg)
FYI: Bitmap2Material
• Similar commercial tool by allegorithmic
![Page 14: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/14.jpg)
Overview
1. Highlight/shadow removal– Simple thresholding for image intensities
2. Diffuse shading separation– Just like Intrinsic Images [Bousseau et al. 2009] – but different formulation, better results
3. Normal reconstruction– Shape from shading→height field→normal estimate
4. Specular assignment– User assigns preset BRDFs (Ward model)
![Page 15: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/15.jpg)
Comparisons
• Diffuse shading separation– Fewer user inputs than
[Bousseau et al. 2009]
• Normal reconstruction– Better results
than others
![Page 16: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/16.jpg)
![Page 17: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/17.jpg)
Failure Cases
• See Figure 13
W/ strong geometric structures
W/ large regions of highlights
Grayscale image, where the albedo only has grayscale variations
fails to separate the black text from shading variations and generates artifacts in the normal map
![Page 18: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/18.jpg)
Material Editing
• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)
• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)
• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),
Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)
• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),
Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)
![Page 19: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/19.jpg)
AppWarp: Retargeting Measured Materials by Appearance-space Warping
• Given measured materials (w/ SVBRDFs), the appearance of a source material is retargeted to that of a template material – Just like color transfer
![Page 20: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/20.jpg)
Overview
1. Calculates the distance between each pair of BRDF samples [Pellacini and Lawrence 2007]
2. Embeds the samples into low-dimensional (two or three) appearance space using multidimensional scaling (MDS)
3. Warp the appearance space using Moving Least Squares (MLS) Warping [Schaefer et al. 2006]– Affine transformation only, w/ 2k-10k control points
![Page 21: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/21.jpg)
Multidimensional Scaling (MDS)?
• Given pair-wise distances between samples(dissimilarity matrix)
• Calculate positions of N-dimensional samples {xi} so that the distances are preserved
![Page 22: SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25](https://reader035.vdocuments.net/reader035/viewer/2022062407/56649e8a5503460f94b8fd9e/html5/thumbnails/22.jpg)