![Page 1: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/1.jpg)
Optical flow
Cordelia Schmid
![Page 2: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/2.jpg)
Motion field
• The motion field is the projection of the 3D scene motion into the image
![Page 3: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/3.jpg)
Optical flow
• Definition: optical flow is the apparent motion of brightness patterns in the image
• Ideally, optical flow would be the same as the motion field
• Have to be careful: apparent motion can be caused by lighting changes without any actual motion– Think of a uniform rotating sphere under fixed lighting
vs. a stationary sphere under moving illumination
![Page 4: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/4.jpg)
Estimating optical flow
• Given two subsequent frames, estimate the apparent motion field u(x,y) and v(x,y) between them
• Key assumptions• Brightness constancy: projection of the same point looks the
same in every frame• Small motion: points do not move very far• Spatial coherence: points move like their neighbors
I(x,y,t–1) I(x,y,t)
![Page 5: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/5.jpg)
Brightness Constancy Equation:),()1,,( ),,(),( tyxyx vyuxItyxI
),(),(),,()1,,( yxvIyxuItyxItyxI yx
Linearizing the right side using Taylor expansion:
The brightness constancy constraint
I(x,y,t–1) I(x,y,t)
0 tyx IvIuIHence,
![Page 6: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/6.jpg)
The brightness constancy constraint
• How many equations and unknowns per pixel?– One equation, two unknowns
• What does this constraint mean?
• The component of the flow perpendicular to the gradient (i.e., parallel to the edge) is unknown
0 tyx IvIuI
0)','( vuI
edge
(u,v)
(u’,v’)
gradient
(u+u’,v+v’)
If (u, v) satisfies the equation, so does (u+u’, v+v’) if
0),( tIvuI
![Page 7: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/7.jpg)
The aperture problem
Perceived motion
![Page 8: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/8.jpg)
The aperture problem
Actual motion
![Page 9: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/9.jpg)
Solving the aperture problem• How to get more equations for a pixel?• Spatial coherence constraint: pretend the pixel’s
neighbors have the same (u,v)– E.g., if we use a 5x5 window, that gives us 25 equations per pixel
B. Lucas and T. Kanade. An iterative image registration technique with an application tostereo vision. In International Joint Conference on Artificial Intelligence,1981.
)(
)()(
)()(
)()()()(
2
1
22
11
nt
t
t
nynx
yx
yx
I
II
vu
II
IIII
x
xx
xx
xxxx
![Page 10: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/10.jpg)
Lucas-Kanade flow• Linear least squares problem
The summations are over all pixels in the window
Solution given by
)(
)()(
)()(
)()()()(
2
1
22
11
nt
t
t
nynx
yx
yx
I
II
vu
II
IIII
x
xx
xx
xxxx
1122
nnbdA
bAA)dA TT (
ty
tx
yyyx
yxxx
IIII
vu
IIIIIIII
![Page 11: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/11.jpg)
Lucas-Kanade flow
• Recall the Harris corner detector: M = ATA is the second moment matrix
• When is the system solvable?• By looking at the eigenvalues of the second moment matrix• The eigenvectors and eigenvalues of M relate to edge
direction and magnitude • The eigenvector associated with the larger eigenvalue points
in the direction of fastest intensity change, and the other eigenvector is orthogonal to it
ty
tx
yyyx
yxxx
IIII
vu
IIIIIIII
![Page 12: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/12.jpg)
Uniform region
– gradients have small magnitude– small 1, small 2– system is ill-conditioned
![Page 13: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/13.jpg)
Edge
– gradients have one dominant direction– large 1, small 2– system is ill-conditioned
![Page 14: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/14.jpg)
High-texture or corner region
– gradients have different directions, large magnitudes– large 1, large 2– system is well-conditioned
![Page 15: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/15.jpg)
Optical Flow Results
![Page 16: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/16.jpg)
Multi-resolution registration
![Page 17: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/17.jpg)
Coarse to fine optical flow estimation
![Page 18: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/18.jpg)
Optical Flow Results
![Page 19: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/19.jpg)
Horn & Schunck algorithm
Additional smoothness constraint :• nearby point have similar optical flow• Addition constraint
,))()(( 2222 dxdyvvuue yxyxs
B.K.P. Horn and B.G. Schunck, "Determining optical flow." Artificial Intelligence,1981
![Page 20: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/20.jpg)
Horn & Schunck algorithm
Additional smoothness constraint :
,))()(( 2222 dxdyvvuue yxyxs besides OF constraint equation term
,)( 2dxdyIvIuIe tyxc minimize es+ec λ regularization parameter
B.K.P. Horn and B.G. Schunck, "Determining optical flow." Artificial Intelligence,1981
![Page 21: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/21.jpg)
Horn & Schunck algorithm
Coupled PDEs solved using iterative methods and finite differences
![Page 22: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/22.jpg)
Horn & Schunck
• Works well for small displacements– For example Middlebury sequence
![Page 23: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/23.jpg)
Large displacement estimation in optical flow
Large displacement is still an open problem in optical flow estimation
MPI Sintel dataset
![Page 24: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/24.jpg)
Large displacement optical flow
Classical optical flow [Horn and Schunck 1981]
► energy:
► minimization using a coarse-to-fine scheme
Large displacement approaches:► LDOF [Brox and Malik 2011]
a matching term, penalizing the difference between flow and HOG matches
► MDP-Flow2 [Xu et al. 2012]expensive fusion of matches (SIFT + PatchMatch) and estimated flow at each level
► DeepFlow [Weinzaepfel et al. 2013]deep matching + flow refinement with variational approach
color/gradient constancy smoothness constraint
![Page 25: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/25.jpg)
Deep Matching: main idea
Each subpatch is allowed to move:► independently► in a limited range depending on its size
The approach is fast to compute using convolution and max-pooling
The idea is applied recursively
First image Second image
![Page 26: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/26.jpg)
Deep Matching (1)
Reference image
Target image
convolution
non-overlapping patches of 4x4 pixels
![Page 27: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/27.jpg)
Deep Matching (2)
response maps for each 4x4 patch
max-pooling sub-sampling aggregation
response maps of 8x8 patches
max-pooling
(3x3 filter)
sub-sampling
(half)
aggregation
![Page 28: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/28.jpg)
Deep Matching (2)
response maps for each 4x4 patch
response maps of 8x8 patches
max-pooling sub-sampling aggregation
max-pooling sub-sampling aggregation
max-pooling sub-sampling aggregation
…
Pipeline similar in spirit to deep convolutional nets [Lecun et al. 1998]
response maps of 16x16 patches
response maps of 32x32 patches
![Page 29: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/29.jpg)
Deep Matching (3)
Multi-scale response pyramid
Extract scale-space local maxima
Backtrack quasi-dense correspondences
Bottom-up Top-down
![Page 30: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/30.jpg)
Deep Matching (3)
First image
Second image
local maximum
![Page 31: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/31.jpg)
Deep Matching: example results
Repetitive textures
First image Second image
![Page 32: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/32.jpg)
Deep Matching: example results
Non-rigid deformation
First image Second image
![Page 33: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/33.jpg)
DeepFlow
Classical optical flow [Horn and Schunck 1981]
► energy
Integration of Deep Matching
► energy
► matches guide the flow► similar to [Brox and Malik 2011]
Minimization using:► coarse-to-fine strategy► fixed point iterations► Successive Over Relaxation (SOR)
![Page 34: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/34.jpg)
Experimental results: datasets
MPI-Sintel [Butler et al. 2012]
► sequences from a realistic animated movie► large displacements (>20px for 17.5% of pixels)► atmospheric effects and motion blur
![Page 35: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/35.jpg)
Experimental results: datasets
KITTI [Geiger et al. 2013]
► sequences captured from a driving platform► large displacements (>20px for 16% of pixels)► real-world: lightings, surfaces, materials
![Page 36: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/36.jpg)
Experimental results: sample results
Ground-truth
LDOF [Brox & Malik 2011]
MDP-Flow2 [Xu et al. 2012]
DeepFlow
![Page 37: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/37.jpg)
Experimental results: sample results
Ground-truth
LDOF [Brox & Malik 2011]
MDP-Flow2 [Xu et al. 2012]
DeepFlow
![Page 38: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/38.jpg)
Experimental results: improvements due to Deep Matching
Comparison on MPI-Sintel training set► AEE: average endpoint error► s40+: only on large displacements
HOG matching Deep Matching
![Page 39: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/39.jpg)
EpicFlow: Sparse-to-dense interpolation based on Deep Matching
accurate quasi dense matches with DeepMatching
[Revaud et al., CVPR’15]
![Page 40: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/40.jpg)
Does not respect motion boundariesInterpolation Ground-Truth
Approach: Sparse-to-dense interpolation based on Deep Matching
![Page 41: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/41.jpg)
► image edges often coincide with motion boundaries (recall 95%)► state-of-the-art SED detector [structured forest for edge detection, Dollar’13]
image
ground-truth flow
SED edges
ground-truth motion boundaries
Approach: Sparse-dense interpolation with motion boundaries
![Page 42: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/42.jpg)
Approach: Sparse-dense interpolation with motion boundaries
EpicFlow:
• Matching [Deep Matching]• Sparse-dense interpolation preserving motion boundaries
Geodesic distance based on edges [SED]• Refinement: One-level energy minimization with variational approach
![Page 43: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/43.jpg)
Distance : edge-aware geodesic distance
geodesic distance:► shortest distance ► knowing a cost map C
Cost map C:► image edges ► here computed with SED p
q
![Page 44: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/44.jpg)
Sparse-to-dense Interpolation: edge-aware geodesic distance
Image edges
Geodesic distance
Geodesic distance
Matches
100 closest matches
100 closest matches
![Page 45: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/45.jpg)
Comparing Interpolation/EpicFlow/DeepFlow
![Page 46: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/46.jpg)
Comparison to the state of the art
![Page 47: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/47.jpg)
Comparison to the state of the art (AEE)
TF + OFM : Kennedy’15. Optical flow with geometric occlusion estimation and fusion of multiple framesNLTGV-SC: Ranftl’14. Non-local total generalized variation for optical flow estimation.
Method Error on MPI-Sintel Error on Kitti Timings
EpicFlow 6.28 3.8 16.4s
TF+OFM 6.73 5.0 ~500s
DeepFlow 7.21 5.8 19s
NLTGV-SC 8.75 3.8 16s (GPU)
![Page 48: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/48.jpg)
Failure cases
Missing matches (spear and horns of dragon)
Missing contours(arm)
![Page 49: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/49.jpg)
CNN to estimate optical flow: FlowNet
[A. Dosovitskiy et al. ICCV’15]
![Page 50: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/50.jpg)
Architecture FlowNetSimple
![Page 51: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/51.jpg)
Architecture FlowNetCorrelation
![Page 52: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/52.jpg)
Synthetic dataset for training: Flying chairs
A dataset of approx. 23k image pairs
![Page 53: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/53.jpg)
Experimental results
S: simple, C: correlation, v: variational refinement, ft:fine-tuning
![Page 54: optical flow 2016 - InriaLarge displacement optical flow Classical optical flow [Horn and Schunck 1981] energy: minimization using a coarse-to-fine scheme Large displacement approaches:](https://reader034.vdocuments.net/reader034/viewer/2022042205/5ea766fef5db945374582047/html5/thumbnails/54.jpg)
Experimental results