uncalibrated reconstruction calibration with a rig uncalibrated epipolar geometry ambiguities in...
DESCRIPTION
ICRA Taxonomy of uncalibrated reconstruction K is known, back to calibrated case K is unknown Calibration (with a rig) – estimate K Uncalibrated reconstruction despite the lack of knowledge of K Autocalibration (recover K from images) Use complete scene knowledge (a rig) Use partial knowledge Parallel lines, vanishing points, planar motion, constant intrinsic Ambiguities, stratification (multiple views)TRANSCRIPT
Uncalibrated reconstruction
Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial scene knowledge
S. Soatto, J. Kosecka
ICRA 2003 2
Uncalibrated Camera
• Image Plane Coordinates
• Perspective projection
Calibrated camera
• Pixel coordinates
• Camera intrinsic parameters
• Projection Matrix
Uncalibrated camera
xyc
ICRA 2003 3
Taxonomy of uncalibrated reconstruction
K is known, back to calibrated case K is unknown
Calibration (with a rig) – estimate K Uncalibrated reconstruction despite the lack
of knowledge of K Autocalibration (recover K from images)
Use complete scene knowledge (a rig) Use partial knowledge
Parallel lines, vanishing points, planar motion, constant intrinsic
Ambiguities, stratification (multiple views)
ICRA 2003 4
Calibration with a rig
• Given 3-D coordinates on known object
• Eliminate unknown scales
• Recover Projection Matrix•
• Factor the into and using QR decomposition• Solve for translation
ICRA 2003 5
Uncalibrated camera vs. distorted space
Inner product in Euclidean space : compute distances and angles
Calibration K transforming spatial coordinates
Transformation induced on the inner product
S (the metric of the space) and K are equivalent
ICRA 2003 6
Calibrated vs. Uncalibrated Space
ICRA 2003 7
Calibrated vs. Uncalibrated Space
Distances and angles are modified by S
ICRA 2003 8
Motion in the distorted space
• Uncalibrated coordinates are related by
• Conjugate of the Euclidean group
Calibrated space Uncalibrated space
ICRA 2003 9
Uncalibrated Camera or Distorted Space
Uncalibrated camera with Calibration matrix K viewing points in Euclidean space and moving with (R,T) is equivalent to calibrated camera viewing points in distorted space governed by S and moving with motion conjugate to (R,T)
ICRA 2003
Uncalibrated Epipolar Geometry
• Fundamental matrix
• equivalent forms of F
• Epipolar Constraint
ICRA 2003 11
Imagecorrespondences
• Epipolar lines• Epipoles
Properties of the fundamental matrix
ICRA 2003 12
Characterization of the fundamental matrix
A nonzero matrix is a fundamental matrix if has A singular value decomposition (SVD) with
For some .
There is little structure in the matrix except that
ICRA 2003 13
Decomposing the fundamental matrix• Decomposition of the Fundamental matrix into skewsymmetric matrix and nonsingular matrix
• Decomposition of F is not unique
• Unknown parameters - ambiguity
• Corresponding projection matrix
ICRA 2003 14
What does F tell us?
F can be inferred from point matches (8-point algorithm)
Cannot extract motion, structure and calibration from one fundamental matrix (2 views)
F allows reconstruction up to a projective transformation (as we will see soon)
F encodes all the geometric information among two views when no additional information is available
ICRA 2003 15
Ambiguities in the image formation Potential Ambiguities
Ambiguity in K (K can be recovered uniquely – Cholesky or QR)
Structure of the motion parameters
Just an arbitrary choice of reference frame
ICRA 2003 16
Ambiguities in the image formationStructure of the Motion Parameters
ICRA 2003 17
Ambiguities in the image formation
For any invertible 4 x 4 matrix H In the uncalibrated case we cannot
distinguish between camera imaging word
from camera imaging distorted world In general, H is of the following form
In order to preserve the choice of the first reference frame we can restrict some DOF of H
Structure of the projection matrix
ICRA 2003 18
Structure of the projective ambiguity
• H can be further decomposed as
• For i-th frame
• 1st frame as reference
• Choose the projective reference frame
then ambiguity is
ICRA 2003 19
Geometric stratification (contd.)
ICRA 2003 20
Projective reconstruction
From points, extract F; from F extract X
Canonical decomposition
Projection matrices
ICRA 2003 21
Projective reconstruction• Given projection matrices recover projective structure
• Given 2 images and no prior information, the scene canbe recovered up a a 4-parameter family of solutions. This is the best one can do!
ICRA 2003 22
Affine upgrade
Upgrade projective structure to an affine structure
Exploit partial scene knowledge Vanishing points, no skew, known principal point
Special motions Pure rotation, pure translation, planar motion,
rectilinear motion Constant camera parameters (multi-view)
ICRA 2003 23
Affine upgrade using vanishing points
Maps the points
To points with affine coordinates
ICRA 2003 24
Vanishing point estimation
ICRA 2003 25
Euclidean upgrade
Exploit special motions (e.g. pure rotation)
If Euclidean is the goal, perform Euclidean reconstruction directly (no stratification)
Multi-view reconstruction (tomorrow)
Autocalibration (Kruppa)
ICRA 2003 26
Overview of the methods
ICRA 2003 27
Direct stratification for multiple views
Absolute Quadric Constraint
ICRA 2003 28
Direct methods - Summary
ICRA 2003 29
Summary