multi-view stereo via volumetric graph-cuts george vogiatzis, philip h. s. torr roberto cipolla
TRANSCRIPT
Multi-view Stereo via Multi-view Stereo via Volumetric Graph-cutsVolumetric Graph-cuts
George Vogiatzis, Philip H. S. Torr Roberto Cipolla
Shape From ImagesShape From Images
Dense Stereo Dense Stereo reconstruction problem:reconstruction problem:• Input
• Set of images of a scene I={I1,…,IK}
• Camera matrices P1,…,PK
• Output• Surface model
Shape parametrisationShape parametrisation
• Disparity-map parameterisation• MRF formulation – good optimisation
techniques exist (Graph-cuts, Loopy BP)
• MRF smoothness is viewpoint dependent
• Disparity is unique per pixel – only functions represented
Shape parametrisationShape parametrisation
• Volumetric parameterisation – e.g. Level-sets, Space carving etc.• Able to cope with non-functions• Convergence properties not well
understood• Memory intensive• For Space carving, no simple way to
impose surface smoothness
Solution ?Solution ?
• Cast volumetric methods in MRF framework
• Benefits:• General surfaces can be represented• Optimisation is tractable (MRF solvers)• Occlusions can be approximately
modelled• Smoothness is viewpoint independent
Graph cutsGraph cuts
40
30
23
12
13
5
540
24
1
50
3
4
2021
13
Graph cutsGraph cuts
40
30
23
12
13
5
540
24
1
50
3
4
2021
13
Graph cutsGraph cuts
40
30
23
12
13
5
540
24
1
50
3
4
2021
5+5+1+4+3=18
13
Volumetric Graph cuts for Volumetric Graph cuts for segmentationsegmentation
• Volume is discretized • A binary MRF is defined on the voxels • Regular grid (6 or 26 neighbourhood)• Voxels are labelled as OBJECT and
BACKGROUND• Labelling cost set by OBJECT /
BACKGROUND intensity statistics• Compatibility cost set by edge intensities
Volumetric Graph cuts for Volumetric Graph cuts for stereostereo• How to define ‘Inside’ and ‘Outside’
labels
• How to deal with occlusion
Volumetric Graph cutsVolumetric Graph cuts
Source
Sink
Min cut
FaceFace
Face - Visual Hull Face - Visual Hull
Face - SliceFace - Slice
Face - Slice with graphcutFace - Slice with graphcut
Face - ReconstructionFace - Reconstruction
Protrusion problemProtrusion problem
• ‘Balooning’ force• favouring bigger volumes
L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131–1147, November 1993.
Protrusion problemProtrusion problem
• ‘Balooning’ force• favouring bigger volumes
L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131–1147, November 1993.
Protrusion problemProtrusion problem
Protrusion problemProtrusion problem
GraphGraph
ResultsResults
• Model House
ResultsResults
• Model House – Visual Hull
ResultsResults
• Model House
ResultsResults• Stone carving
ResultsResults
• Haniwa
SummarySummary
• Novel formulation for multiview stereo
• Volumetric scene representation
• Computationally tractable global optimisation using Graph-cuts.
• Visual hull for occlusions and geometric constraint
BenefitsBenefits
1. General surfaces and objects can be fully represented and computed as a single surface.
2. The representation and smoothness constraint is image and viewpoint independent.
3. Multiple views of the scene can be used with occlusions approximately modelled.
4. Optimisation is computationally tractable, using existing max-flow algorithms.