introduction and course...
TRANSCRIPT
![Page 1: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/1.jpg)
Two-view geometry
15-463, 15-663, 15-862Computational Photography
Fall 2018, Lecture 15http://graphics.cs.cmu.edu/courses/15-463
![Page 2: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/2.jpg)
Course announcements
• Homework 4 is out.- Due October 26th.- Start early: part 3 (lightfield capture) takes a lot of time to get right.- Any questions?
• Due October 21st: Project ideas posted on Piazza.
• Extra office hours this afternoon, 5-7 pm.
• ECE Seminar tomorrow: Rajiv Laroia, “Is Computational Imaging the future of Photography?”
- New time and date: noon – 1:30 pm, Scaife Hall 125
![Page 3: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/3.jpg)
Light camera L16
• Use multiple views (i.e., lightfield) to refocus.• Use deconvolution to keep thin (i.e., skip compound lens).
![Page 4: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/4.jpg)
Overview of today’s lecture
• Leftover from lecture 13
• Reminder about pinhole and lens cameras
• Camera matrix.
• Other camera models.
• Camera calibration.
![Page 5: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/5.jpg)
Slide credits
Many of these slides were adapted from:
• Kris Kitani (16-385, Spring 2017).• Srinivasa Narasimhan (16-720, Fall 2017).
![Page 6: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/6.jpg)
Overview of today’s lecture
• Leftover from lecture 13: camera calibration.
• Triangulation.
• Epipolar geometry.
• Essential matrix.
• Fundamental matrix.
• 8-point algorithm.
![Page 7: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/7.jpg)
Triangulation
![Page 8: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/8.jpg)
Triangulation
image 2image 1
Given
camera 1 with matrix camera 2 with matrix
![Page 9: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/9.jpg)
Triangulation
image 2image 1
camera 1 with matrix camera 2 with matrix
Which 3D points map
to x?
![Page 10: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/10.jpg)
Triangulation
image 2image 1
camera 1 with matrix camera 2 with matrix
How can you compute
this ray?
![Page 11: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/11.jpg)
Triangulation
image 2image 1
camera 1 with matrix camera 2 with matrix
+
Why does this
point map to x?
Create two points on the ray:
1) find the camera center; and
2) apply the pseudo-inverse of P on x.
Then connect the two points.
![Page 12: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/12.jpg)
Triangulation
image 2image 1
camera 1 with matrix camera 2 with matrix
+How do we find the
exact point on the ray?
![Page 13: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/13.jpg)
Triangulation
image 2image 1
Find 3D object pointWill the lines intersect?
camera 1 with matrix camera 2 with matrix
![Page 14: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/14.jpg)
Triangulation
image 2image 1
Find 3D object point(no single solution due to noise)
camera 1 with matrix camera 2 with matrix
![Page 15: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/15.jpg)
Given a set of (noisy) matched points
and camera matrices
Estimate the 3D point
Triangulation
![Page 16: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/16.jpg)
known known
Can we compute X from a single
correspondence x?
![Page 17: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/17.jpg)
known known
Can we compute X from two
correspondences x and x’?
![Page 18: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/18.jpg)
known known
Can we compute X from two
correspondences x and x’?
yes if perfect measurements
![Page 19: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/19.jpg)
There will not be a point that satisfies both constraints
because the measurements are usually noisy
known known
Need to find the best fit
Can we compute X from two
correspondences x and x’?
yes if perfect measurements
![Page 20: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/20.jpg)
Also, this is a similarity relation because it involves homogeneous coordinates
Same ray direction but differs by a scale factor
How do we solve for unknowns in a similarity relation?
(homorogeneous
coordinate)
(homogeneous
coordinate)
![Page 21: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/21.jpg)
How do we solve for unknowns in a similarity relation?
Remove scale factor, convert to linear system and solve with SVD.
Also, this is a similarity relation because it involves homogeneous coordinates
Same ray direction but differs by a scale factor
(inhomogeneous
coordinate)
(homogeneous
coordinate)
![Page 22: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/22.jpg)
How do we solve for unknowns in a similarity relation?
Remove scale factor, convert to linear system and solve with SVD!
Also, this is a similarity relation because it involves homogeneous coordinates
Same ray direction but differs by a scale factor
(inhomogeneous
coordinate)
(homogeneous
coordinate)
![Page 23: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/23.jpg)
Recall: Cross ProductVector (cross) product
takes two vectors and returns a vector perpendicular to both
cross product of two vectors in
the same direction is zero
remember this!!!
![Page 24: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/24.jpg)
Same direction but differs by a scale factor
Cross product of two vectors of same direction is zero
(this equality removes the scale factor)
![Page 25: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/25.jpg)
![Page 26: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/26.jpg)
![Page 27: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/27.jpg)
Using the fact that the cross product should be zero
Third line is a linear combination of the first and second lines.
(x times the first line plus y times the second line)
One 2D to 3D point correspondence give you 2 equations
![Page 28: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/28.jpg)
Third line is a linear combination of the first and second lines.
(x times the first line plus y times the second line)
One 2D to 3D point correspondence give you 2 equations
Using the fact that the cross product should be zero
![Page 29: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/29.jpg)
Now we can make a system of linear equations
(two lines for each 2D point correspondence)
![Page 30: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/30.jpg)
How do we solve homogeneous linear system?
Concatenate the 2D points from both images
sanity check! dimensions?
![Page 31: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/31.jpg)
How do we solve homogeneous linear system?
Concatenate the 2D points from both images
S V D !
![Page 32: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/32.jpg)
Recall: Total least squares
(matrix form)
(Warning: change of notation. x is a vector of parameters!)
constraint
minimize
subject tominimize
Solution is the eigenvector
corresponding to smallest eigenvalue of
(Rayleigh quotient)
![Page 33: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/33.jpg)
Epipolar geometry
![Page 34: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/34.jpg)
Epipolar geometry
Image plane
![Page 35: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/35.jpg)
Epipolar geometry
Image plane
Baseline
![Page 36: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/36.jpg)
Epipolar geometry
Image plane
Baseline
Epipole(projection of o’ on the image plane)
![Page 37: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/37.jpg)
Epipolar geometry
Epipole(projection of o’ on the image plane)
Baseline
Epipolar plane
Image plane
Baseline
![Page 38: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/38.jpg)
Epipolar geometry
Epipole(projection of o’ on the image plane)
Baseline
Epipolar plane
Epipolar line(intersection of Epipolar
plane and image plane)
Image plane
![Page 39: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/39.jpg)
Quiz
What is this?
![Page 40: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/40.jpg)
Quiz
Epipolar plane
![Page 41: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/41.jpg)
Quiz
Epipolar plane
What is this?
![Page 42: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/42.jpg)
Quiz
Epipolar plane
Epipolar line(intersection of Epipolar
plane and image plane)
![Page 43: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/43.jpg)
Quiz
Epipolar plane
Epipolar line(intersection of Epipolar
plane and image plane)
What is this?
![Page 44: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/44.jpg)
Quiz
Epipolar plane
Epipolar line(intersection of Epipolar
plane and image plane)
Epipole(projection of o’ on the image plane)
![Page 45: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/45.jpg)
Quiz
Epipolar plane
Epipolar line(intersection of Epipolar
plane and image plane)
Epipole(projection of o’ on the image plane)
What is this?
![Page 46: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/46.jpg)
Quiz
Epipolar plane
Epipolar line(intersection of Epipolar
plane and image plane)
Epipole(projection of o’ on the image plane)
Baseline
![Page 47: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/47.jpg)
Epipolar constraint
Potential matches for lie on the epipolar line
![Page 48: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/48.jpg)
Epipolar constraint
Potential matches for lie on the epipolar line
![Page 49: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/49.jpg)
The point x (left image) maps to a ___________ in the right image
The baseline connects the ___________ and ____________
An epipolar line (left image) maps to a __________ in the right image
An epipole e is a projection of the ______________ on the image plane
All epipolar lines in an image intersect at the ______________
![Page 50: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/50.jpg)
Converging cameras
Where is the epipole in this image?
![Page 51: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/51.jpg)
Converging cameras
It’s not always in the image Where is the epipole in this image?
here!
![Page 52: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/52.jpg)
Parallel cameras
Where is the epipole?
![Page 53: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/53.jpg)
Parallel cameras
epipole at infinity
![Page 54: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/54.jpg)
The epipolar constraint is an important concept for stereo vision
Left image Right image
Task: Match point in left image to point in right image
How would you do it?
![Page 55: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/55.jpg)
Recall:Epipolar constraint
Potential matches for lie on the epipolar line
![Page 56: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/56.jpg)
The epipolar constraint is an important concept for stereo vision
Left image Right image
Task: Match point in left image to point in right image
Want to avoid search over entire image
Epipolar constraint reduces search to a single line
![Page 57: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/57.jpg)
The epipolar constraint is an important concept for stereo vision
Left image Right image
Task: Match point in left image to point in right image
How do you compute the epipolar line?
Want to avoid search over entire image
Epipolar constraint reduces search to a single line
![Page 58: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/58.jpg)
The essential matrix
![Page 59: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/59.jpg)
Recall:Epipolar constraint
Potential matches for lie on the epipolar line
![Page 60: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/60.jpg)
Given a point in one image,
multiplying by the essential matrix will tell us
the epipolar line in the second view.
![Page 61: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/61.jpg)
Motivation
The Essential Matrix is a 3 x 3 matrix that
encodes epipolar geometry
Given a point in one image,
multiplying by the essential matrix will tell us
the epipolar line in the second view.
![Page 62: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/62.jpg)
Epipolar Line
in vector form
If the point is on the epipolar line then
Representing the …
![Page 63: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/63.jpg)
Epipolar Line
in vector form
If the point is on the epipolar line then
![Page 64: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/64.jpg)
Recall: Dot Product
dot product of two orthogonal vectors is zero
![Page 65: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/65.jpg)
vector representing the line is
normal (orthogonal) to the plane
vector representing the point x is
inside the plane
Therefore:
![Page 66: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/66.jpg)
So if and then
![Page 67: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/67.jpg)
So if and then
![Page 68: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/68.jpg)
Essential Matrix vs HomographyWhat’s the difference between the essential matrix and a homography?
![Page 69: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/69.jpg)
Essential Matrix vs Homography
They are both 3 x 3 matrices but …
Essential matrix maps a
point to a line
Homography maps a
point to a point
What’s the difference between the essential matrix and a homography?
![Page 70: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/70.jpg)
Where does the Essential matrix come from?
![Page 71: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/71.jpg)
![Page 72: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/72.jpg)
Does this look familiar?
![Page 73: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/73.jpg)
Camera-camera transform just like world-camera transform
![Page 74: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/74.jpg)
These three vectors are coplanar
![Page 75: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/75.jpg)
If these three vectors are coplanar then
![Page 76: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/76.jpg)
If these three vectors are coplanar then
![Page 77: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/77.jpg)
Recall: Cross ProductVector (cross) product
takes two vectors and returns a vector perpendicular to both
![Page 78: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/78.jpg)
If these three vectors are coplanar then
![Page 79: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/79.jpg)
If these three vectors are coplanar then
![Page 80: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/80.jpg)
putting it together
coplanarityrigid motion
![Page 81: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/81.jpg)
Can also be written as a matrix multiplication
Cross product
Skew symmetric
![Page 82: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/82.jpg)
putting it together
coplanarityrigid motion
![Page 83: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/83.jpg)
putting it together
coplanarityrigid motion
![Page 84: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/84.jpg)
putting it together
coplanarityrigid motion
![Page 85: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/85.jpg)
putting it together
coplanarityrigid motion
Essential Matrix
[Longuet-Higgins 1981]
![Page 86: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/86.jpg)
properties of the E matrix
Longuet-Higgins equation
(points in normalized coordinates)
![Page 87: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/87.jpg)
properties of the E matrix
Epipolar lines
Longuet-Higgins equation
(points in normalized coordinates)
![Page 88: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/88.jpg)
properties of the E matrix
Epipolar lines
Longuet-Higgins equation
Epipoles
(points in normalized camera coordinates)
![Page 89: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/89.jpg)
Recall:Epipolar constraint
Potential matches for lie on the epipolar line
![Page 90: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/90.jpg)
Given a point in one image,
multiplying by the essential matrix will tell us
the epipolar line in the second view.
Assumption:
points aligned to camera coordinate axis (calibrated camera)
![Page 91: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/91.jpg)
How do you generalize
to uncalibrated
cameras?
![Page 92: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/92.jpg)
The fundamental matrix
![Page 93: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/93.jpg)
The
Fundamental matrix
is a
generalization
of the
Essential matrix,
where the assumption of
calibrated cameras
is removed
![Page 94: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/94.jpg)
The Essential matrix operates on image points expressed in
normalized coordinates
(points have been aligned (normalized) to camera coordinates)
image
point
camera
point
![Page 95: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/95.jpg)
The Essential matrix operates on image points expressed in
normalized coordinates
(points have been aligned (normalized) to camera coordinates)
Writing out the epipolar constraint in terms of image coordinates
image
point
camera
point
![Page 96: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/96.jpg)
Same equation works in image coordinates!
it maps pixels to epipolar lines
![Page 97: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/97.jpg)
properties of the E matrix
Epipolar lines
Longuet-Higgins equation
Epipoles
(points in image coordinates)
F
F
F F
F F
![Page 98: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/98.jpg)
Depends on both intrinsic and extrinsic parameters
Breaking down the fundamental matrix
![Page 99: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/99.jpg)
Depends on both intrinsic and extrinsic parameters
Breaking down the fundamental matrix
How would you solve for F?
![Page 100: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/100.jpg)
The 8-point algorithm
![Page 101: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/101.jpg)
Assume you have M matched image points
Each correspondence should satisfy
How would you solve for the 3 x 3 F matrix?
![Page 102: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/102.jpg)
Each correspondence should satisfy
How would you solve for the 3 x 3 F matrix?
S V D
Assume you have M matched image points
![Page 103: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/103.jpg)
Each correspondence should satisfy
How would you solve for the 3 x 3 F matrix?
Set up a homogeneous linear system with 9 unknowns
Assume you have M matched image points
![Page 104: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/104.jpg)
How many equation do you get from one correspondence?
![Page 105: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/105.jpg)
ONE correspondence gives you ONE equation
![Page 106: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/106.jpg)
Set up a homogeneous linear system with 9 unknowns
How many equations do you need?
![Page 107: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/107.jpg)
Each point pair (according to epipolar constraint)
contributes only one scalar equation
Note: This is different from the Homography estimation where
each point pair contributes 2 equations.
We need at least 8 points
Hence, the 8 point algorithm!
![Page 108: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/108.jpg)
How do you solve a homogeneous linear system?
![Page 109: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/109.jpg)
How do you solve a homogeneous linear system?
minimize
subject to
Total Least Squares
![Page 110: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/110.jpg)
How do you solve a homogeneous linear system?
minimize
subject to
Total Least Squares
S V D !
![Page 111: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/111.jpg)
Eight-Point Algorithm
0. (Normalize points)
1. Construct the M x 9 matrix A
2. Find the SVD of A
3. Entries of F are the elements of column of V
corresponding to the least singular value
4. (Enforce rank 2 constraint on F)
5. (Un-normalize F)
![Page 112: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/112.jpg)
Eight-Point Algorithm
0. (Normalize points)
1. Construct the M x 9 matrix A
2. Find the SVD of A
3. Entries of F are the elements of column of V
corresponding to the least singular value
4. (Enforce rank 2 constraint on F)
5. (Un-normalize F) See Hartley-Zisserman
for why we do this
![Page 113: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/113.jpg)
Eight-Point Algorithm
0. (Normalize points)
1. Construct the M x 9 matrix A
2. Find the SVD of A
3. Entries of F are the elements of column of V
corresponding to the least singular value
4. (Enforce rank 2 constraint on F)
5. (Un-normalize F) How do we do this?
![Page 114: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/114.jpg)
Eight-Point Algorithm
0. (Normalize points)
1. Construct the M x 9 matrix A
2. Find the SVD of A
3. Entries of F are the elements of column of V
corresponding to the least singular value
4. (Enforce rank 2 constraint on F)
5. (Un-normalize F) How do we do this?
S V D !
![Page 115: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/115.jpg)
Problem: Given a matrix F, find the matrix F’ of rank k that is closest to F,
Enforcing rank constraints
min𝐹′
rank 𝐹′ =𝑘
𝐹 − 𝐹′ 2
Solution: Compute the singular value decomposition of F,
𝐹 = 𝑈Σ𝑉𝑇
Form a matrix Σ’ by replacing all but the k largest singular values in Σ with 0.
Then the problem solution is the matrix F’ formed as,
𝐹′ = 𝑈Σ′𝑉𝑇
![Page 116: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/116.jpg)
Eight-Point Algorithm
0. (Normalize points)
1. Construct the M x 9 matrix A
2. Find the SVD of A
3. Entries of F are the elements of column of V
corresponding to the least singular value
4. (Enforce rank 2 constraint on F)
5. (Un-normalize F)
![Page 117: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/117.jpg)
Example
![Page 118: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/118.jpg)
epipolar lines
![Page 119: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/119.jpg)
![Page 120: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/120.jpg)
![Page 121: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/121.jpg)
Where is the epipole?
How would you compute it?
![Page 122: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/122.jpg)
The epipole is in the right null space of F
How would you solve for the epipole?
![Page 123: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/123.jpg)
The epipole is in the right null space of F
How would you solve for the epipole?
S V D !
![Page 124: Introduction and course overviewgraphics.cs.cmu.edu/courses/15-463/2018_fall/lectures/lecture15.pdf · • Leftover from lecture 13: camera calibration. •Triangulation. •Epipolar](https://reader033.vdocuments.net/reader033/viewer/2022043010/5fa0f4c9ca4ae54054588631/html5/thumbnails/124.jpg)
Basic reading:• Szeliski textbook, Section 8.1 (not 8.1.1-8.1.3), Chapter 11, Section 12.2.• Hartley and Zisserman, Section 11.12.
References