stereo. stereopsis reading: chapter 11. the stereopsis problem: fusion and reconstruction human...
Post on 20-Jan-2016
221 views
TRANSCRIPT
![Page 1: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/1.jpg)
Stereo
![Page 2: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/2.jpg)
STEREOPSIS
Reading: Chapter 11.
• The Stereopsis Problem: Fusion and Reconstruction• Human Stereopsis and Random Dot Stereograms• Cooperative Algorithms• Correlation-Based Fusion• Multi-Scale Edge Matching• Dynamic Programming• Using Three or More Cameras
![Page 3: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/3.jpg)
An Application: Mobile Robot Navigation
The Stanford Cart,H. Moravec, 1979.
The INRIA Mobile Robot, 1990.
Courtesy O. Faugeras and H. Moravec.
![Page 4: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/4.jpg)
Reconstruction / Triangulation
![Page 5: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/5.jpg)
(Binocular) Fusion
![Page 6: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/6.jpg)
Reconstruction
• Linear Method: find P such that
• Non-Linear Method: find Q minimizing
![Page 7: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/7.jpg)
Rectification
All epipolar lines are parallel in the rectified image plane.
![Page 8: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/8.jpg)
Image pair rectification
simplify stereo matching by warping the images
Apply projective transformation so that epipolar linescorrespond to horizontal scanlines
e
e
map epipole e to (1,0,0)
try to minimize image distortion
problem when epipole in (or close to) the image
He001
![Page 9: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/9.jpg)
![Page 10: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/10.jpg)
![Page 11: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/11.jpg)
Polar re-parameterization around epipoles
Requires only (oriented) epipolar geometry
Preserve length of epipolar linesChoose so that no pixels are
compressed
original image rectified image
Polar rectification(Pollefeys et al. ICCV’99)
Works for all relative motionsGuarantees minimal image size
![Page 12: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/12.jpg)
polar rectification: example
![Page 13: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/13.jpg)
polar rectification: example
![Page 14: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/14.jpg)
Reconstruction from Rectified Images
Disparity: d=u’-u. Depth: z = -B/d.
![Page 15: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/15.jpg)
Human Stereopsis: Binocular Fusion
How are the correspondences established?
Julesz (1971): Is the mechanism for binocular fusiona monocular process or a binocular one??• There is anecdotal evidence for the latter (camouflage).
• Random dot stereograms provide an objective answerBP!
![Page 16: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/16.jpg)
A Cooperative Model (Marr and Poggio, 1976)
Excitory connections: continuity
Inhibitory connections: uniqueness
Iterate: C = C - w C + C .e i 0
Reprinted from Vision: A Computational Investigation into the Human Representation and Processing of Visual Information by David Marr. 1982 by David Marr. Reprinted by permission of Henry Holt and Company, LLC.
![Page 17: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/17.jpg)
Correlation Methods (1970--)
Normalized Correlation: minimize instead.
Slide the window along the epipolar line until w.w’ is maximized.
2Minimize |w-w’|.
![Page 18: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/18.jpg)
Correlation Methods: Foreshortening Problems
Solution: add a second pass using disparity estimates to warpthe correlation windows, e.g. Devernay and Faugeras (1994).
Reprinted from “Computing Differential Properties of 3D Shapes from Stereopsis without 3D Models,” by F. Devernay and O. Faugeras, Proc. IEEE Conf. on Computer Vision and Pattern Recognition (1994). 1994 IEEE.
![Page 19: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/19.jpg)
Multi-Scale Edge Matching (Marr, Poggio and Grimson, 1979-81)
• Edges are found by repeatedly smoothing the image and detectingthe zero crossings of the second derivative (Laplacian).• Matches at coarse scales are used to offset the search for matchesat fine scales (equivalent to eye movements).
![Page 20: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/20.jpg)
Multi-Scale Edge Matching (Marr, Poggio and Grimson, 1979-81)
One of the twoinput images
Image Laplacian
Zeros of the Laplacian
Reprinted from Vision: A Computational Investigation into the Human Representation and Processing of Visual Information by David Marr. 1982 by David Marr. Reprinted by permission of Henry Holt and Company, LLC.
![Page 21: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/21.jpg)
Multi-Scale Edge Matching (Marr, Poggio and Grimson, 1979-81)
Reprinted from Vision: A Computational Investigation into the Human Representation and Processing of Visual Information by David Marr. 1982 by David Marr. Reprinted by permission of Henry Holt and Company, LLC.
![Page 22: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/22.jpg)
The Ordering Constraint
But it is not always the case..
In general the pointsare in the same orderon both epipolar lines.
![Page 23: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/23.jpg)
Dynamic Programming (Baker and Binford, 1981)
Find the minimum-cost path going monotonicallydown and right from the top-left corner of thegraph to its bottom-right corner.
• Nodes = matched feature points (e.g., edge points).• Arcs = matched intervals along the epipolar lines.• Arc cost = discrepancy between intervals.
![Page 24: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/24.jpg)
Dynamic Programming (Ohta and Kanade, 1985)
Reprinted from “Stereo by Intra- and Intet-Scanline Search,” by Y. Ohta and T. Kanade, IEEE Trans. on Pattern Analysis and MachineIntelligence, 7(2):139-154 (1985). 1985 IEEE.
![Page 25: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/25.jpg)
Three Views
The third eye can be used for verification..
![Page 26: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/26.jpg)
More Views (Okutami and Kanade, 1993)
Pick a reference image, and slide the correspondingwindow along the corresponding epipolar lines of allother images, using inverse depth relative to the firstimage as the search parameter.
Use the sum of correlation scores to rank matches.
Reprinted from “A Multiple-Baseline Stereo System,” by M. Okutami and T. Kanade, IEEE Trans. on PatternAnalysis and Machine Intelligence, 15(4):353-363 (1993). \copyright 1993 IEEE.
![Page 27: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/27.jpg)
Stereo matching
Optimal path(dynamic programming )
Similarity measure(SSD or NCC)
Constraints• epipolar
• ordering
• uniqueness
• disparity limit
• disparity gradient limit
Trade-off
• Matching cost (data)
• Discontinuities (prior)
(Cox et al. CVGIP’96; Koch’96; Falkenhagen´97; Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)
![Page 28: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/28.jpg)
Hierarchical stereo matchingD
ow
nsam
plin
g
(Gau
ssia
n p
yra
mid
)
Dis
pari
ty p
rop
ag
ati
on
Allows faster computation
Deals with large disparity ranges
(Falkenhagen´97;Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)
![Page 29: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/29.jpg)
Disparity map
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y),y)
![Page 30: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/30.jpg)
Example: reconstruct image from neighboring images
![Page 31: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/31.jpg)
I1 I2 I10
Reprinted from “A Multiple-Baseline Stereo System,” by M. Okutami and T. Kanade, IEEE Trans. on PatternAnalysis and Machine Intelligence, 15(4):353-363 (1993). \copyright 1993 IEEE.
![Page 32: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/32.jpg)
Real-time stereo on graphics hardware
Computes Sum-of-Square-Differences Hardware mip-map generation used to aggregate results over
support region Trade-off between small and large support window
Ruigang Yang and Marc Pollefeys
140M disparity hypothesis/sec on Radeon 9700pro140M disparity hypothesis/sec on Radeon 9700proe.g. 512x512x20disparities at 30Hze.g. 512x512x20disparities at 30Hz
Shape of a kernel for summing up 6 levels
![Page 33: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/33.jpg)
(1x1)
(1x1+2x2)
(1x1+2x2 +4x4+8x8)
(1x1+2x2 +4x4+8x8 +16x16)
Combine multiple aggregation windows using hardware mipmap and multiple texture units in single pass
![Page 34: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/34.jpg)
Live stereo demoPC with ATI Radeon 9700pro
Bumblebee (Point Grey)
OpenGL 1.4 (Fragment programs)
Code available at:
http://vis.uky.edu/~ryang/research/ViewSyn/
![Page 35: Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms](https://reader036.vdocuments.net/reader036/viewer/2022062423/56649d3a5503460f94a14566/html5/thumbnails/35.jpg)
More on stereo …The Middleburry Stereo Vision Research Pagehttp://cat.middlebury.edu/stereo/
Recommended reading
D. Scharstein and R. Szeliski. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms.IJCV 47(1/2/3):7-42, April-June 2002. PDF file (1.15 MB) - includes current evaluation.Microsoft Research Technical Report MSR-TR-2001-81, November 2001. PDF file (1.27 MB).