uncalibrated geometry & stratification
DESCRIPTION
Uncalibrated Geometry & Stratification. Stefano Soatto UCLA Yi Ma UIUC. Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial scene knowledge. Overview. calibrated coordinates. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/1.jpg)
Uncalibrated Geometry & Stratification
Stefano SoattoUCLA
Yi MaUIUC
![Page 2: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/2.jpg)
September 15, 2003 ICRA2003, Taipei
Overview
• Calibration with a rig
• Uncalibrated epipolar geometry
• Ambiguities in image formation
• Stratified reconstruction
• Autocalibration with partial scene knowledge
![Page 3: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/3.jpg)
September 15, 2003 ICRA2003, Taipei
Uncalibrated Camera
pixelcoordinates
calibratedcoordinates
Linear transformation
![Page 4: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/4.jpg)
September 15, 2003 ICRA2003, Taipei
Uncalibrated Camera
xyc
• Pixel coordinates
• Projection matrix
Uncalibrated camera
• Image plane coordinates • Camera extrinsic parameters
• Perspective projection
Calibrated camera
![Page 5: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/5.jpg)
September 15, 2003 ICRA2003, Taipei
Taxonomy on Uncalibrated Reconstruction
• is known, back to calibrated case
• is unknown Calibration with complete scene knowledge (a rig) –
estimate Uncalibrated reconstruction despite the lack of
knowledge of Autocalibration (recover from uncalibrated images)
• Use partial knowledge Parallel lines, vanishing points, planar motion, constant
intrinsic
• Ambiguities, stratification (multiple views)
![Page 6: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/6.jpg)
September 15, 2003 ICRA2003, Taipei
Calibration with a RigUse the fact that both 3-D and 2-D coordinates of feature points on a pre-fabricated object (e.g., a cube) are known.
![Page 7: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/7.jpg)
September 15, 2003 ICRA2003, Taipei
Calibration with a Rig
• Eliminate unknown scales
• Factor the into and using QR decomposition• Solve for translation
• Recover projection matrix
• Given 3-D coordinates on known object
![Page 8: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/8.jpg)
September 15, 2003 ICRA2003, Taipei
Uncalibrated Camera vs. Distorted Space• Inner product in Euclidean space: compute distances and angles
• Calibration transforming spatial coordinates
• Transformation induced a new inner product
• (the metric of the space) and are equivalent
![Page 9: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/9.jpg)
September 15, 2003 ICRA2003, Taipei
Calibrated vs. Uncalibrated Space
![Page 10: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/10.jpg)
September 15, 2003 ICRA2003, Taipei
Calibrated vs. Uncalibrated Space
Distances and angles are modified by
![Page 11: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/11.jpg)
September 15, 2003 ICRA2003, Taipei
Motion in the Distorted Space
Uncalibrated space Calibrated space
• Uncalibrated coordinates are related by
• Conjugate of the Euclidean group
![Page 12: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/12.jpg)
September 15, 2003 ICRA2003, Taipei
Uncalibrated Camera or Distorted Space
Uncalibrated camera with a calibration matrix K viewing points in Euclidean space and moving with (R,T) is equivalent to a calibrated camera viewing points in distorted space governed by S and moving with a motion conjugate to (R,T)
![Page 13: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/13.jpg)
September 15, 2003 ICRA2003, Taipei
Uncalibrated Epipolar Geometry
• Epipolar constraint
• Fundamental matrix
• Equivalent forms of
![Page 14: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/14.jpg)
September 15, 2003 ICRA2003, Taipei
Properties of the Fundamental Matrix
Imagecorrespondences
• Epipolar lines
• Epipoles
![Page 15: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/15.jpg)
September 15, 2003 ICRA2003, Taipei
Properties of the Fundamental Matrix
A nonzero matrix is a fundamental matrix if has a singular value decomposition (SVD) with
for some .
There is little structure in the matrix except that
![Page 16: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/16.jpg)
September 15, 2003 ICRA2003, Taipei
What Does F Tell Us?
• can be inferred from point matches (eight-point algorithm)
• Cannot extract motion, structure and calibration from one fundamental matrix (two views)
• allows reconstruction up to a projective transformation (as we will see soon)
• encodes all the geometric information among two views when no additional information is available
![Page 17: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/17.jpg)
September 15, 2003 ICRA2003, Taipei
Decomposing the Fundamental Matrix
• Decomposition of is not unique
• Unknown parameters - ambiguity
• Corresponding projection matrix
• Decomposition of the fundamental matrix into a skew symmetric matrix and a nonsingular matrix
![Page 18: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/18.jpg)
September 15, 2003 ICRA2003, Taipei
• Potential ambiguities
• Ambiguity in (can be recovered uniquely – QR)
• Structure of the motion parameters
• Just an arbitrary choice of reference coordinate frame
Ambiguities in Image Formation
![Page 19: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/19.jpg)
September 15, 2003 ICRA2003, Taipei
Ambiguities in Image Formation
Structure of motion parameters
![Page 20: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/20.jpg)
September 15, 2003 ICRA2003, Taipei
Ambiguities in Image Formation
•For any invertible 4 x 4 matrix
•In the uncalibrated case we cannot distinguish between camera imaging word from camera imaging distorted world
•In general, is of the following form
• In order to preserve the choice of the first reference frame we can restrict some DOF of
Structure of the projection matrix
![Page 21: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/21.jpg)
September 15, 2003 ICRA2003, Taipei
Structure of the Projective Ambiguity• For i-th frame
• 1st frame as reference
• Choose the projective reference frame then ambiguity is
• can be further decomposed as
![Page 22: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/22.jpg)
September 15, 2003 ICRA2003, Taipei
Geometric Stratification (cont)
![Page 23: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/23.jpg)
September 15, 2003 ICRA2003, Taipei
Projective Reconstruction
•From points, extract , from which extract and
•Canonical decomposition
•Projection matrices
![Page 24: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/24.jpg)
September 15, 2003 ICRA2003, Taipei
Projective Reconstruction• Given projection matrices recover projective
structure
• Given 2 images and no prior information, the scene can be recovered up a 4-parameter family of solutions. This is the best one can do without knowing calibration!
• This is a linear problem and can be solve using least-squares techniques.
![Page 25: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/25.jpg)
September 15, 2003 ICRA2003, Taipei
Affine Upgrade
•Upgrade projective structure to an affine structure
•Exploit partial scene knowledge Vanishing points, no skew, known principal point
•Special motions Pure rotation, pure translation, planar motion,
rectilinear motion•Constant camera parameters (multi-view)
![Page 26: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/26.jpg)
September 15, 2003 ICRA2003, Taipei
Affine Upgrade Using Vanishing Points
maps points on the plane
to points with affine coordinates
![Page 27: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/27.jpg)
September 15, 2003 ICRA2003, Taipei
Vanishing Point Estimation from Parallelism
![Page 28: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/28.jpg)
September 15, 2003 ICRA2003, Taipei
Euclidean Upgrade• Exploit special motions (e.g. pure rotation)
• If Euclidean is the goal, perform Euclidean reconstruction directly (no stratification)
• Direct autocalibration (Kruppa’s equations)
• Multiple-view case (absolute quadric)
![Page 29: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/29.jpg)
September 15, 2003 ICRA2003, Taipei
Direct Autocalibration Methods
satisfies the Kruppa’s equations
The fundamental matrix
If the fundamental matrix is known up to scale
Under special motions, Kruppa’s equations become linear.
Solution to Kruppa’s equations is sensitive to noises.
![Page 30: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/30.jpg)
September 15, 2003 ICRA2003, Taipei
Direct Stratification from Multiple Views
we obtain the absolute quadric contraints
From the recovered projective projection matrix
Partial knowledge in (e.g. zero skew, square pixel) renders the above constraints linear and easier to solve.
The projection matrices can be recovered from the multiple-view rank method to be introduced later.
![Page 31: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/31.jpg)
September 15, 2003 ICRA2003, Taipei
Direct Methods - Summary
![Page 32: Uncalibrated Geometry & Stratification](https://reader035.vdocuments.net/reader035/viewer/2022062310/568160ee550346895dd024cf/html5/thumbnails/32.jpg)
September 15, 2003 ICRA2003, Taipei
Summary of (Auto)calibration Methods