inferring differential structure from defocused imagesfmannan/766/shape_inference_dfd.pdf ·...
TRANSCRIPT
![Page 1: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/1.jpg)
Inferring Differential Structure from Defocused Images
Fahim Mannan
School of Computer ScienceMcGill University
April 8, 2014
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 1 / 26
![Page 2: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/2.jpg)
Overview
1 Introduction
2 Overview of ApproachDepth From DefocusQuadric Surface FittingVariational RelaxationSurface Re-estimation
3 Conclusion
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 2 / 26
![Page 3: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/3.jpg)
Objective
Goal?
Estimate differential structure from a pair of defocused images.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 3 / 26
![Page 4: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/4.jpg)
Input
(a) Near Focused (b) Far Focused
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 4 / 26
![Page 5: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/5.jpg)
Output
(c) Depthmap (d) 3D
along with Normal and Principal Curvatures at every pixel.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 5 / 26
![Page 6: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/6.jpg)
Darboux Frame
Local representation of a surface.
N: Normal
M: Principle curvature(max direction)
m: Principle curvature(min direction)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 6 / 26
![Page 7: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/7.jpg)
Darboux Frame
Local representation of a surface.
N: Normal
M: Principle curvature(max direction)
m: Principle curvature(min direction)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 6 / 26
![Page 8: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/8.jpg)
Overview of the Algorithm
1 Initial depth estimate using Depth from Defocus (DFD)
2 Quadric surface fitting at every pixel and compute initial Darbouxframe
3 Refine initial estimate using variational relaxation
4 Use the refined Darboux frame estimates to re-estimating the surface
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 7 / 26
![Page 9: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/9.jpg)
Overview of the Algorithm
1 Initial depth estimate using Depth from Defocus (DFD)
2 Quadric surface fitting at every pixel and compute initial Darbouxframe
3 Refine initial estimate using variational relaxation
4 Use the refined Darboux frame estimates to re-estimating the surface
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 7 / 26
![Page 10: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/10.jpg)
Overview of the Algorithm
1 Initial depth estimate using Depth from Defocus (DFD)
2 Quadric surface fitting at every pixel and compute initial Darbouxframe
3 Refine initial estimate using variational relaxation
4 Use the refined Darboux frame estimates to re-estimating the surface
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 7 / 26
![Page 11: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/11.jpg)
Overview of the Algorithm
1 Initial depth estimate using Depth from Defocus (DFD)
2 Quadric surface fitting at every pixel and compute initial Darbouxframe
3 Refine initial estimate using variational relaxation
4 Use the refined Darboux frame estimates to re-estimating the surface
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 7 / 26
![Page 12: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/12.jpg)
Related Work
1 Based on Sander and Zucker’s “Inferring Surface Trace andDifferential Structure” [1].
2 Ferrie et al. “Darboux Frames, Snakes and Super-Quadrics” [2]
3 Mathur et al. Curvature Consistency using Optimal EstimationTheory [3]
4 Savadjiev et al. Surface Recovery and Geometric Flow [4]
Note1 [1] considers volumetric images.
2 [2, 3, 4] 3D point cloud acquired using laser scanner.
3 This project uses two defocused images, and
4 Depth (noisy and low resolution) estimated using DFD.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 8 / 26
![Page 13: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/13.jpg)
Related Work
1 Based on Sander and Zucker’s “Inferring Surface Trace andDifferential Structure” [1].
2 Ferrie et al. “Darboux Frames, Snakes and Super-Quadrics” [2]
3 Mathur et al. Curvature Consistency using Optimal EstimationTheory [3]
4 Savadjiev et al. Surface Recovery and Geometric Flow [4]
Note1 [1] considers volumetric images.
2 [2, 3, 4] 3D point cloud acquired using laser scanner.
3 This project uses two defocused images, and
4 Depth (noisy and low resolution) estimated using DFD.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 8 / 26
![Page 14: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/14.jpg)
Related Work
1 Based on Sander and Zucker’s “Inferring Surface Trace andDifferential Structure” [1].
2 Ferrie et al. “Darboux Frames, Snakes and Super-Quadrics” [2]
3 Mathur et al. Curvature Consistency using Optimal EstimationTheory [3]
4 Savadjiev et al. Surface Recovery and Geometric Flow [4]
Note1 [1] considers volumetric images.
2 [2, 3, 4] 3D point cloud acquired using laser scanner.
3 This project uses two defocused images, and
4 Depth (noisy and low resolution) estimated using DFD.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 8 / 26
![Page 15: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/15.jpg)
Overview of Depth from DefocusModeling Defocus Blur
f : Focal length.
A: Aperture.
N: f-number f /A.
s: Sensor distance.
u: Object distance.
r : Blur width.
ρ: pixels/mm.
σ: Blur in pixels.
Image Plane
A
us
v
fr
Defocus Blur Equation
σ = ρr = ρfs
2N(
1
f− 1
u− 1
s) (1)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 9 / 26
![Page 16: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/16.jpg)
Overview of Depth from DefocusModeling Defocus Blur
f : Focal length.
A: Aperture.
N: f-number f /A.
s: Sensor distance.
u: Object distance.
r : Blur width.
ρ: pixels/mm.
σ: Blur in pixels.
Image Plane
A
us
v
fr
Defocus Blur Equation
σ = ρr = ρfs
2N(
1
f− 1
u− 1
s) (1)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 9 / 26
![Page 17: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/17.jpg)
Overview of Depth from DefocusModeling Defocus Blur
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20
2
4
6
8
10
12
1/u m−1
blur
rad
ius
(pix
els)
focus = 1 m, f =50 mm, f/A = 22
0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
2
4
6
8
10
12
u (meters)bl
ur r
adiu
s (p
ixel
s)
focus = 1 m, f =50 mm, f/A = 22
Defocus Blur Equation
|σ| = ρfs
2N|1f− 1
u− 1
s|
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 10 / 26
![Page 18: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/18.jpg)
Overview of Depth from DefocusModeling Defocused Images
Let,
I0 : True sharp image
I : Observed image
H(|σ|) : Defocus PSF
N (0, σ2N): Gaussian noise
(a) Pillbox
(b) Gaussian
Defocused Image Formation Equation
I = I0 ∗ H(|σ|) + η, where, η ∈ N (0, σ2N) (2)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 11 / 26
![Page 19: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/19.jpg)
Overview of Depth from DefocusDepth Estimation
Let,
IS : Sharper image
IB : Blurrier image
HS & HB : PSFs for the sharper and blurrier images
ηS & ηB : Gaussian noise from N (0, σ2N)
Pair of Defocused Images
IS = I0 ∗ HS + ηS (3)
IB = I0 ∗ HB + ηB (4)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 12 / 26
![Page 20: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/20.jpg)
Overview of Depth from DefocusDepth Estimation
Let,
IS : Sharper image
IB : Blurrier image
HS & HB : PSFs for the sharper and blurrier images
ηS & ηB : Gaussian noise from N (0, σ2N)
Pair of Defocused Images
IS = I0 ∗ HS + ηS (3)
IB = I0 ∗ HB + ηB (4)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 12 / 26
![Page 21: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/21.jpg)
Overview of Depth from DefocusDepth Estimation
Let,
IS : Sharper image
IB : Blurrier image
HS & HB : PSFs for the sharper and blurrier images
ηS & ηB : Gaussian noise from N (0, σ2N)
Pair of Defocused Images
IS = I0 ∗ HS + ηS (3)
IB = I0 ∗ HB + ηB (4)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 12 / 26
![Page 22: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/22.jpg)
Overview of Depth from DefocusDepth Estimation
Relative Blur
HB = HS ∗ HR (5)
Relative Blur Radius
σS = min{|σ1|, |σ2|}σB = max{|σ1|, |σ2|}
σR =√σ2B − σ2S (6)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 13 / 26
![Page 23: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/23.jpg)
Overview of Depth from DefocusDepth Estimation
Relative Blur
HB = HS ∗ HR (5)
Relative Blur Radius
σS = min{|σ1|, |σ2|}σB = max{|σ1|, |σ2|}
σR =√σ2B − σ2S (6)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 13 / 26
![Page 24: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/24.jpg)
Overview of Depth from DefocusDepth Estimation
A Simplification
Recall Eqs. 3 and 4,
IS = I0 ∗ HS + ηS
IB = I0 ∗ HB + ηB
Assume,
IB ≈ IS ∗ HR + ηB (7)
Relative Blur Estimation Problem
arg minHR
‖IB − IS ∗ HR‖ (8)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 14 / 26
![Page 25: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/25.jpg)
Overview of Depth from DefocusDepth Estimation
A Simplification
Recall Eqs. 3 and 4,
IS = I0 ∗ HS + ηS
IB = I0 ∗ HB + ηB
Assume,
IB ≈ IS ∗ HR + ηB (7)
Relative Blur Estimation Problem
arg minHR
‖IB − IS ∗ HR‖ (8)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 14 / 26
![Page 26: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/26.jpg)
Overview of Depth from DefocusImplementation
1 Apply isotropic diffusion to the sharper image i.e. IS ∗ HR
2 For every pixel choose HR that minimizes ‖IB − IS ∗ HR‖3 Compute depth corresponding to HR
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 15 / 26
![Page 27: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/27.jpg)
Overview of Depth from DefocusImplementation
1 Apply isotropic diffusion to the sharper image i.e. IS ∗ HR
2 For every pixel choose HR that minimizes ‖IB − IS ∗ HR‖
3 Compute depth corresponding to HR
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 15 / 26
![Page 28: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/28.jpg)
Overview of Depth from DefocusImplementation
1 Apply isotropic diffusion to the sharper image i.e. IS ∗ HR
2 For every pixel choose HR that minimizes ‖IB − IS ∗ HR‖3 Compute depth corresponding to HR
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 15 / 26
![Page 29: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/29.jpg)
Initial Quadric Surface Fitting
Quadric Surface Model
φ(u, v) = (u, v , h(u, v))
where, h(u, v) =1
2au2 + buv +
1
2cv2 + eu + fv + g
Quadric Fitting for a Patch
1 For every 7× 7 patch.
2 Fit the quadric in the perspective projection space usingLeast-Squares.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 16 / 26
![Page 30: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/30.jpg)
Variational RelaxationExample
Figure: Neighborhood Support
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 17 / 26
![Page 31: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/31.jpg)
Variational RelaxationExample
Figure: Neighborhood Support
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 17 / 26
![Page 32: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/32.jpg)
Variational RelaxationExample
Figure: Neighborhood Support
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 17 / 26
![Page 33: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/33.jpg)
Variational Relaxation
Variational Relaxation Update Equation (Sander and Zucker [1])
D̂i (v) =1
|Ni |∑j∈Ni
Dj(v) (9)
Information Theoretic Based Update (Mathur et al. [3])
D̂i (v) =1
|Ni |∑j∈Ni
λjDj(v) (10)
where λj is a measure of confidence in node j .
Proposed Weight
λ′j = λj f (‖IB − IS ∗ HR‖). (11)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 18 / 26
![Page 34: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/34.jpg)
Variational Relaxation
Variational Relaxation Update Equation (Sander and Zucker [1])
D̂i (v) =1
|Ni |∑j∈Ni
Dj(v) (9)
Information Theoretic Based Update (Mathur et al. [3])
D̂i (v) =1
|Ni |∑j∈Ni
λjDj(v) (10)
where λj is a measure of confidence in node j .
Proposed Weight
λ′j = λj f (‖IB − IS ∗ HR‖). (11)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 18 / 26
![Page 35: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/35.jpg)
Variational Relaxation
Variational Relaxation Update Equation (Sander and Zucker [1])
D̂i (v) =1
|Ni |∑j∈Ni
Dj(v) (9)
Information Theoretic Based Update (Mathur et al. [3])
D̂i (v) =1
|Ni |∑j∈Ni
λjDj(v) (10)
where λj is a measure of confidence in node j .
Proposed Weight
λ′j = λj f (‖IB − IS ∗ HR‖). (11)
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 18 / 26
![Page 36: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/36.jpg)
Surface Re-estimation
Energy Minimization Model
E (s) = arg mins
∑‖IB − IS ∗ HR(s)‖+ α
∑(r ,θ)
(‖∇2
(r ,θ)s‖ − κ(r ,θ))
(12)
Where, κ(r ,θ) is the estimated curvature in a direction (r , θ).
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 19 / 26
![Page 37: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/37.jpg)
Results
Figure: 3D point cloud with normal
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 20 / 26
![Page 38: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/38.jpg)
Results
(a) Initial (b) Iteration 10
Figure: XY-plane projection of normals. Starting with noisy depthmap.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 21 / 26
![Page 39: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/39.jpg)
Results
(a) Initial (b) Iteration 10
Figure: XY-plane projection of normals. Starting with MAP-MRF depthmap.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 22 / 26
![Page 40: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/40.jpg)
Observations/Limitations
Noise needs to be low to get meaningful estimates.
Low resolution depthmap =⇒ mostly planar.
Boundary needs to be handled carefully.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 23 / 26
![Page 41: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/41.jpg)
Next Steps
Use λ′ weighting in variational relaxation.
Re-estimate surface using the refined estimates.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 24 / 26
![Page 42: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/42.jpg)
References
[1] P. T. Sander and S. W. Zucker, “Inferring surface trace and differential structurefrom 3-d images,” IEEE PAMITC, vol. 12, no. 9, pp. 833–854, 1990.
[2] F. Ferrie, J. Lagarde, and P. Whaite, “Darboux frames, snakes, and super-quadrics:geometry from the bottom up,” PAMI, IEEE Transactions on, vol. 15, no. 8, pp.771–784, Aug 1993.
[3] S. Mathur and F. Ferrie, “Edge localization in surface reconstruction using optimalestimation theory,” in CVPR, 1997., Jun 1997, pp. 833–838.
[4] P. Savadjiev, F. Ferrie, and K. Siddiqi, “Surface recovery from 3d point data using acombined parametric and geometric flow approach,” in EMMCVPR, 2003, vol.2683, pp. 325–340.
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 25 / 26
![Page 43: Inferring Differential Structure from Defocused Imagesfmannan/766/shape_inference_dfd.pdf · Variational Relaxation Update Equation (Sander and Zucker [1]) D^ i(v) = 1 jN ij X j2N](https://reader035.vdocuments.net/reader035/viewer/2022071014/5fcd42854dd28c53963aa7c9/html5/thumbnails/43.jpg)
The End
Fahim Mannan (SOCS,McGill University) COMP 766 April 8, 2014 26 / 26