dynamic 3d reconstruction of deformable surfaces with stereo.pdf
TRANSCRIPT
Dynamic 3D reconstruction of deformable surfaces with stereo
Frédéric Devernay, Frédéric Huguet
INRIA/PERCEPTION CNRS/Géosciences Azur
Project Geolstereo● Geophysic : study of earthquakes and
landslides● Numerical simulations are often not reliable● Too many parameters● Need for experimentations with real data● 3D physical modeling
Geophysic: earthquakes
Faille de San Andreas, Californie
Observations Numerical Modeling
PhysicsSismic Waves
Landslides : La Clapière (06)
100m
50.106 m3 involved Velocity of about 1 meter/yearThe surface of failure is at 100 meters depth.
A very heterogeneous mechanical structure.
Main characteristics:
3D Experimental modellingModel geometry
Similarity criterias: the model must obey the real physical laws at a
little scale
10 mm in the model
=
500 m in reality
3D experimental modelling
50 m
Mount Pépoiri (Mercantour)
Initial Data● Stereo sequence● can be chosen as we want
t
t
Stereo● Two points of view of the same scene● M can be found with its projections● Rectification simplifies the problem
ml mr
M
C l C r
Observations● Links between the image pairs
I l0 I r
0
I lt I r
t
d0
tu , v
x , y
d t
ModellingPiecewise regular surface
Energy design
We take into account the spatial and temporal relationships between the images
Minimization : 3 non linear coupled PDE at each time t
E=E DataE Reg
State of the artStereo with variational methods
Terzopoulos 86 : keeping the discontinuities
Faugeras & Keriven 98 : level set method
Pons et al 05 : the only one 4D method known. 2 steps: reconstruction and scene flow estimation
Brox et al 04 : robust optical flow
Modelling
● Optical Flow contribution:
● Stereo contribution
● Constant illumination contribution:
E Data=∫C flotC stereoCillumC lambert
C flot= I lt xu , yv −I l
0 x , y2
C stereo= I rt xud x , y , yv−I l
t xu , yv 2
C illum=∇ I lt xu , yv −∇ I l
0 x , y 2
ModellingEReg=∫
∣∇ u∣2∣∇ v∣2∣∇ d∣2
x Aubert et al 97
Brox et al 04
/
= baseline/height
Solving the PDE● Non convex functional : local minima possible● Multiresolution algorithm● Gaussian images pyramid pairs● At the coarsest detail level, disparity and flow should
be less than 1 pixel● Semi implicit numerical scheme: more stability
Scene Flow● Simultaneous disparity and optical flow allows us to
find the surface displacement at any time t
C1C2
2D flow
3D flow
Disparity
M
Preliminary results● Work in progress : first basic example
L , t=1 R , t=1
L , t=6 R , t=6
Preliminary results● Disparity and Optical Flow
L , t=1
L , t=6
R , t=1
R , t=6
Future Work ● Proof of convergence
● Theoretical study of the equations
● Application to big real images (3000*2000): algorithm
optimization : domain decomposition ...
Conclusion● Non classical application of computer vision
● Algorithm to reconstruct scene flow in 1
step with variational method
● Help to geophysicist