calibration and single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · a...
TRANSCRIPT
![Page 1: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/1.jpg)
Calibration and Single view meterology
Lecture slides by Steve Seitz (mostly)Lecture presented by Varsha
CS 543/ ECE 54926th January 2010
![Page 2: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/2.jpg)
Todays class
• Recovering the camera intrinsic and extrinsic parameters from an image
• Recovering size in the world• Projecting from one plane to another (if time allows)
![Page 3: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/3.jpg)
Recovering the camera intrinsic and extrinsic parameters from an image
![Page 4: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/4.jpg)
Recap 3D to 2D: “perspective” projection
Matrix Projection:
C’-(u0,v0) optical center
- focal length in x and y directions - skew
!,"
![Page 5: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/5.jpg)
CSE 576, Spring 2008 Projective Geometry 45
Calibration using a reference objectPlace a known object in the scene
• identify correspondence between image and scene• compute mapping from scene to image
Issues• must know geometry very accurately• must know 3D->2D correspondence
![Page 6: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/6.jpg)
Estimating the projection matrixPlace a known object in the scene
• identify correspondence between image and scene• compute mapping from scene to image
![Page 7: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/7.jpg)
Direct linear calibration
![Page 8: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/8.jpg)
Direct linear calibration
![Page 9: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/9.jpg)
Projective Geometry
Direct linear calibration
Ax=0 form
![Page 10: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/10.jpg)
Direct linear calibration
Can solve for mij by linear least squares
Ax=0 form
![Page 11: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/11.jpg)
Direct linear calibrationAdvantage:
• Very simple to formulate and solve
Disadvantages:• Doesn’t tell you the camera parameters• Doesn’t model radial distortion• Hard to impose constraints (e.g., known focal length)• Doesn’t minimize the right error function
For these reasons, nonlinear methods are preferred• Define error function E between projected 3D points and image positions
– E is nonlinear function of intrinsics, extrinsics, radial distortion
• Minimize E using nonlinear optimization techniques– e.g., variants of Newton’s method
![Page 12: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/12.jpg)
Summary • Known correspondences (ui,vi) ~ (Xi,Yi,Zi)• Compute mij solving system of linear equations• May use this to initialize non linear error minimization
problem to recover more accurate mij• Note that some normalization of points is necessary
for result to be invariant to different choices of coordinate systems.
![Page 13: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/13.jpg)
Camera calibration from an image• What if world coordinates are not known?• Can we use scene features(vanishing points)?
![Page 14: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/14.jpg)
• Scene contains lines along directions that are orthogonal
14
![Page 15: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/15.jpg)
Recap: Vanishing points
image plane
line on ground plane
vanishing point v
Vanishing point• projection of a point at infinity
cameracenter
C
![Page 16: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/16.jpg)
vanishing points
V
P0
D
![Page 17: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/17.jpg)
CSE 576, Spring 2008 Projective Geometry 22
vanishing points
Properties• P∞ is a point at infinity, v is its projection• They depend only on line direction
V
P0
D
v = K!R t
"P! = KRD
t! >"
![Page 18: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/18.jpg)
Camera calibration
•From vanishing points corresponding to 3 orthogonal directions of world
[0 0 1]T
![Page 19: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/19.jpg)
• zero skew, unit aspect ratio
• 3 finite vanishing points: get f, u0, v0 • 2 finite and one infinite : u0,v0 as point on vf1 vf2
closest to image center, get f • 2 infinite vanishing points : f cant be recovered u0, v0
is at the third vanishing point
K =
!
"f 0 u0
0 f v0
0 0 1
#
$K!1 =
!
"1/f 0 !u0/f0 1/f !v0/f0 0 1
#
$
vTj K!T K!1vk = 0vT
i K!T K!1vj = 0
vTi K!T K!1vk = 0
![Page 20: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/20.jpg)
Rotation from vanishing points• 1st column vector of Rotation matrix
• from
!vi = KRei
R =!R1c R2c R3c
"
R1c = !K!1vi
||R1c||2 = 1!
R1c
![Page 21: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/21.jpg)
Summary • Orthogonality constraints for solving K• Rotation directly from vanishing points if camera is
known
![Page 22: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/22.jpg)
• Calibrating camera using vanishing points • Advantages
• No need to specify point correspondences• Could be completely automatic
• Disadvantages • Problems due to infinite vanishing points• inaccuracies in computation of vanishing points
![Page 23: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/23.jpg)
Recovering size in the world
![Page 24: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/24.jpg)
Fun with vanishing points
![Page 25: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/25.jpg)
Perspective cues
![Page 26: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/26.jpg)
Perspective cues
![Page 27: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/27.jpg)
Comparing heights
VanishingPoint
![Page 28: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/28.jpg)
Measuring height
1
2
3
4
55.4
2.83.3
Camera height
What is the height the man?
![Page 29: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/29.jpg)
C
Measuring height without a ruler
ground plane
Compute Z from image measurements• Need more than vanishing points to do this
Z
![Page 30: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/30.jpg)
The cross ratioA Projective Invariant
• Something that does not change under projective transformations (including perspective projection)
P1
P2
P3P4
The cross-ratio of 4 collinear points
Can permute the point ordering• 4! = 24 different orders (but only 6 distinct values)
This is the fundamental invariant of projective geometry
![Page 31: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/31.jpg)
vZ
image cross ratio
Measuring height
B (bottom of object)
ground plane
R
scene cross ratio
∞
scene points represented as
image points as
H
tr
b
T (top of object)
R (reference point)
C
![Page 32: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/32.jpg)
Measuring height
RH
vz
r
b
t
H
b0
t0vvx vy
vanishing line (horizon)
image cross ratio
![Page 33: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/33.jpg)
Summary• Cross ratio is invariant to projective transformation• Apply this to get relative sizes of objects • Perspective is important for infering object sizes
![Page 34: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/34.jpg)
Projecting from one plane to another
![Page 35: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/35.jpg)
Homography
Homography is a mapping between points of one plane to another under projective transformation through a single point
x’=Hx H is 3X3 matrix, has 8 degrees of freedom
![Page 36: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/36.jpg)
Examples: Image rectification
To unwarp (rectify) an image• solve for homography H given x and x’• solve equations of the form: wx’ = Hx
– linear in unknowns: w and coefficients of H– H is defined up to an arbitrary scale factor– how many points are necessary to solve for H?
pp’
![Page 37: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/37.jpg)
Projective Geometry
Solving for homographies
![Page 38: Calibration and Single view meterologydhoiem.cs.illinois.edu/...projectivegeomtery.pdf · A Projective Invariant • Something that does not change under projective transformations](https://reader030.vdocuments.net/reader030/viewer/2022040807/5e48787f11d542719a0276c6/html5/thumbnails/38.jpg)
Solving for homographies
A h 0
Defines a least squares problem:2n × 9 9 2n
• Since h is only defined up to scale, solve for unit vector ĥ• Solution: ĥ = eigenvector of ATA with smallest eigenvalue• Works with 4 or more points