panorama signups panorama project issues? nalwa handout announcements
Post on 21-Dec-2015
250 views
TRANSCRIPT
![Page 1: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/1.jpg)
• Panorama signups• Panorama project issues?• Nalwa handout
Announcements
![Page 2: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/2.jpg)
Projective geometry
Readings• Mundy, J.L. and Zisserman, A., Geometric Invariance in Computer Vision, Chapter 23:
Appendix: Projective Geometry for Machine Vision, MIT Press, Cambridge, MA, 1992, pp. 463-534 (for this week, read 23.1 - 23.5, 23.10)
– available online: http://www.cs.cmu.edu/~ph/869/papers/zisser-mundy.pdf
Ames Room
![Page 3: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/3.jpg)
Projective geometry—what’s it good for?
Uses of projective geometry• Drawing• Measurements• Mathematics for projection• Undistorting images• Focus of expansion• Camera pose estimation, match move• Object recognition
![Page 4: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/4.jpg)
Applications of projective geometry
Vermeer’s Music Lesson
![Page 5: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/5.jpg)
1 2 3 4
1
2
3
4
Measurements on planes
Approach: unwarp then measure
What kind of warp is this?
![Page 6: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/6.jpg)
Image rectification
To unwarp (rectify) an image• solve for homography H given p and p’• solve equations of the form: wp’ = Hp
– linear in unknowns: w and coefficients of H
– H is defined up to an arbitrary scale factor
– how many points are necessary to solve for H?
pp’
work out on board
![Page 7: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/7.jpg)
Solving for homographies
![Page 8: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/8.jpg)
Solving for homographies
A h 0
Linear least squares• Since h is only defined up to scale, solve for unit vector ĥ• Minimize
2n × 9 9 2n
• Solution: ĥ = eigenvector of ATA with smallest eigenvalue• Works with 4 or more points
![Page 9: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/9.jpg)
(0,0,0)
The projective planeWhy do we need homogeneous coordinates?
• represent points at infinity, homographies, perspective projection, multi-view relationships
What is the geometric intuition?• a point in the image is a ray in projective space
• Each point (x,y) on the plane is represented by a ray (sx,sy,s)– all points on the ray are equivalent: (x, y, 1) (sx, sy, s)
y
xz
image plane (z = 1)
(sx,sy,s)(x,y,1)
![Page 10: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/10.jpg)
Projective linesWhat is a line in projective space?
• A line is a plane of rays through origin– all rays (x,y,z) satisfying: ax + by + cz = 0
z
y
x
cba0 :notationvectorin
• A line is also represented as a homogeneous 3-vector ll p
(0,0,0)
![Page 11: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/11.jpg)
l
Point and line duality• A line l is a homogeneous 3-vector• It is to every point (ray) p on the line: l p=0
p1p2
What is the intersection of two lines l1 and l2 ?
• p is to l1 and l2 p = l1 l2
Points and lines are dual in projective space• every property of points also applies to lines
l1
l2
p
What is the line l spanned by rays p1 and p2 ?
• l is to p1 and p2 l = p1 p2
• l is the plane normal
![Page 12: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/12.jpg)
Ideal points and lines
Ideal point (“point at infinity”)• p (x, y, 0) – parallel to image plane• It has infinite image coordinates
(sx,sy,0)y
xz image plane
Ideal line• l (a, b, 0) – parallel to image plane
(a,b,0)y
xz image plane
• Corresponds to a line in the image (finite coordinates)
![Page 13: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/13.jpg)
Homographies of points and linesComputed by 3x3 matrix multiplication
• To transform a point: p’ = Hp• To transform a line: lp=0 l’p’=0
– 0 = lp = lH-1Hp = lH-1p’ l’ = lH-1
– lines are transformed by postmultiplication of H-1
![Page 14: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/14.jpg)
3D projective geometryThese concepts generalize naturally to 3D
• Homogeneous coordinates– Projective 3D points have four coords: P = (X,Y,Z,W)
• Duality– A plane N is also represented by a 4-vector
– Points and planes are dual in 3D: N P=0
• Projective transformations– Represented by 4x4 matrices T: P’ = TP, N’ = N T-1
![Page 15: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/15.jpg)
3D to 2D: “perspective” projection
Matrix Projection: ΠPp
1****
****
****
Z
Y
X
s
sy
sx
What is not preserved under perspective projection?
What IS preserved?
![Page 16: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/16.jpg)
Vanishing points
Vanishing point• projection of a point at infinity
image plane
cameracenter
ground plane
vanishing point
![Page 17: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/17.jpg)
Vanishing points (2D)
image plane
cameracenter
line on ground plane
vanishing point
![Page 18: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/18.jpg)
Vanishing points
Properties• Any two parallel lines have the same vanishing point• The ray from C through v point is parallel to the lines• An image may have more than one vanishing point
image plane
cameracenter
C
line on ground plane
vanishing point V
line on ground plane
![Page 19: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/19.jpg)
Vanishing lines
Multiple Vanishing Points• Any set of parallel lines on the plane define a vanishing point• The union of all of these vanishing points is the horizon line
– also called vanishing line• Note that different planes define different vanishing lines
v1 v2
![Page 20: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/20.jpg)
Vanishing lines
Multiple Vanishing Points• Any set of parallel lines on the plane define a vanishing point• The union of all of these vanishing points is the horizon line
– also called vanishing line• Note that different planes define different vanishing lines
![Page 21: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/21.jpg)
Computing vanishing points
Properties• P is a point at infinity, v is its projection
• They depend only on line direction
• Parallel lines P0 + tD, P1 + tD intersect at P
V
DPP t 0
0/1
/
/
/
1Z
Y
X
ZZ
YY
XX
ZZ
YY
XX
t D
D
D
t
t
DtP
DtP
DtP
tDP
tDP
tDP
PP
ΠPv
P0
D
![Page 22: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/22.jpg)
Computing vanishing lines
Properties• l is intersection of horizontal plane through C with image plane
• Compute l from two sets of parallel lines on ground plane
• All points at same height as C project to l
• Provides way of comparing height of objects in the scene
ground plane
lC
![Page 23: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/23.jpg)
Fun with vanishing points
![Page 24: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/24.jpg)
Perspective cues
![Page 25: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/25.jpg)
Perspective cues
![Page 26: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/26.jpg)
Perspective cues
![Page 27: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/27.jpg)
Comparing heights
VanishingVanishingPointPoint
![Page 28: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/28.jpg)
Measuring height
1
2
3
4
55.4
2.8
3.3
Camera height
![Page 29: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/29.jpg)
q1
Computing vanishing points (from lines)
Intersect p1q1 with p2q2
v
p1
p2
q2
Least squares version• Better to use more than two lines and compute the “closest” point of
intersection• See notes by Bob Collins for one good way of doing this:
– http://www-2.cs.cmu.edu/~ph/869/www/notes/vanishing.txt
![Page 30: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/30.jpg)
C
Measuring height without a ruler
ground plane
Compute Y from image measurements• Need more than vanishing points to do this
Y
![Page 31: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/31.jpg)
The cross ratio
A Projective Invariant• Something that does not change under projective transformations
(including perspective projection)
P1
P2
P3
P4
1423
2413
PPPP
PPPP
The cross-ratio of 4 collinear points
Can permute the point ordering• 4! = 24 different orders (but only 6 distinct values)
This is the fundamental invariant of projective geometry
1i
i
i
i Z
Y
X
P
3421
2431
PPPP
PPPP
![Page 32: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/32.jpg)
vZ
r
t
b
tvbr
rvbt
Z
Z
image cross ratio
Measuring height
B (bottom of object)
T (top of object)
R (reference point)
ground plane
HC
TBR
RBT
scene cross ratio
1
Z
Y
X
P
1
y
x
pscene points represented as image points as
R
H
R
H
R
![Page 33: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/33.jpg)
Measuring height
RH
vz
r
b
t
R
H
Z
Z
tvbr
rvbt
image cross ratio
H
b0
t0
vvx vy
vanishing line (horizon)
![Page 34: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/34.jpg)
Measuring height vz
r
b
t0
vx vy
vanishing line (horizon)
v
t0
m0
What if the point on the ground plane b0 is not known?
• Here the guy is standing on the box, height of box is known
• Use one side of the box to help find b0 as shown above
b0
t1
b1
![Page 35: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/35.jpg)
C
Measurements within reference plane
Solve for homography H relating reference plane to image plane• H maps reference plane (X,Y) coords to image plane (x,y) coords
• Fully determined from 4 known points on ground plane
– Option A: physically measure 4 points on ground
– Option B: find a square, guess the dimensions
– Option C: Note H = columns 1,2,4 projection matrix» derive on board
• Given (x, y), can find (X,Y) by H-1
reference plane
TYX 10 Tyx 1
image plane
![Page 36: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/36.jpg)
Criminisi et al., ICCV 99Complete approach
• Load in an image• Click on lines parallel to X axis
– repeat for Y, Z axes
• Compute vanishing points• Specify 3D and 2D positions of 4 points on reference plane• Compute homography H• Specify a reference height • Compute 3D positions of several points• Create a 3D model from these points• Extract texture maps• Output a VRML model
![Page 37: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/37.jpg)
Vanishing points and projection matrix
************
Π 4321 ππππ
1π 2π 3π 4π
T00011 Ππ = vx (X vanishing point)
Z3Y2 , similarly, vπvπ
origin worldof projection10004 TΠπ
ovvvΠ ZYXNot So Fast! We only know v’s up to a scale factor
ovvvΠ ZYX cba• Can fully specify by providing 3 reference points
![Page 38: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/38.jpg)
3D Modeling from a photograph
![Page 39: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/39.jpg)
3D Modeling from a photograph
![Page 40: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/40.jpg)
Camera calibrationGoal: estimate the camera parameters
• Version 1: solve for projection matrix
ΠXx
1****
****
****
Z
Y
X
s
sy
sx
• Version 2: solve for camera parameters separately– intrinsics (focal length, principle point, pixel size)
– extrinsics (rotation angles, translation)
– radial distortion
![Page 41: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/41.jpg)
Calibration: Basic Idea
Place a known object in the scene• identify correspondence between image and scene• compute mapping from scene to image
Issues• must know geometry very accurately• must know 3D->2D correspondence
![Page 42: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/42.jpg)
Chromaglyphs
Courtesy of Bruce Culbertson, HP Labshttp://www.hpl.hp.com/personal/Bruce_Culbertson/ibr98/chromagl.htm
![Page 43: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/43.jpg)
Estimating the Projection Matrix
Place a known object in the scene• identify correspondence between image and scene• compute mapping from scene to image
![Page 44: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/44.jpg)
Direct Linear Calibration
![Page 45: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/45.jpg)
Direct Linear Calibration
Can solve for mij by linear least squares• use eigenvector trick that we used for homographies
![Page 46: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/46.jpg)
Direct linear calibration
Advantages:• Very simple to formulate and solve
• Once you know the projection matrix, can compute intrinsics
and extrinsics using matrix factorizations
Disadvantages?• Doesn’t model radial distortion
• Hard to impose constraints (e.g., known focal length)
• Doesn’t minimize the right error function
For these reasons, nonlinear methods are preferred• Define error function E between projected 3D points and image positions
– E is nonlinear function of intrinsics, extrinsics, radial distortion
• Minimize E using nonlinear optimization techniques
– e.g., variants of Newton’s method (e.g., Levenberg Marquart)
![Page 47: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/47.jpg)
Alternative: Multi-plane calibration
Images courtesy Jean-Yves Bouguet, Intel Corp.
Advantage• Only requires a plane• Don’t have to know positions/orientations• Good code available online!
– Zhengyou Zhang’s web site: http://research.microsoft.com/~zhang/Calib/
– Intel’s OpenCV library: http://www.intel.com/research/mrl/research/opencv/
– Matlab version by Jean-Yves Bouget: http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
![Page 48: Panorama signups Panorama project issues? Nalwa handout Announcements](https://reader035.vdocuments.net/reader035/viewer/2022062221/56649d5d5503460f94a3cfdc/html5/thumbnails/48.jpg)
SummaryThings to take home from this lecture
• Homogeneous coordinates and their geometric intuition• Homographies• Points and lines in projective space
– projective operations: line intersection, line containing two points
– ideal points and lines (at infinity)
• Vanishing points and lines and how to compute them• Single view measurement
– within a reference plane
– height
• Cross ratio• Camera calibration
– using vanishing points
– direct linear method