1

CS252A, Fall 2012 Computer Vision I

Stereo

Computer Vision I CSE252A Lecture 14

CS252A, Fall 2012 Computer Vision I

Announcement •  HW3 assigned •  No class on Thursday 12/6 •  Extra class on Tuesday 12/4 at 6:30PM

CS252A, Fall 2012 Computer Vision I

Canny Edge Detector •  Smooth with Gaussian •  Compute Gradient: df/dx, df/dy •  Compute magnitude and direction •  Non-maximal suppression – local maximum

of the magnitude of the gradient, in the direction of the gradient.

•  Linking with hysteresis thresholding

CS252A, Fall 2012 Computer Vision I

Shi-Tomasi Corner Detector •  Filter image with a Gaussian. •  Compute the gradient everywhere. •  Move window over image and construct C

over the window.

•  Use linear algebra to find λ1 and λ2.

•  If they are both large, we have a corner. 1.  Let e(x,y) = min(λ1(x,y), λ2(x,y)) 2.  (x,y) is a corner if it’s local maximum of e(x,y)

and e(x,y) > τ

Parameters: Gaussian std. dev, window size, threshold

€

C(x,y) =Ix2∑ IxIy∑

IxIy∑ Iy2∑

⎡

⎣ ⎢ ⎢

⎤

⎦ ⎥ ⎥

CS252A, Fall 2012 Computer Vision I

Binocular Stereopsis: Mars Given two images of a scene where relative locations of cameras are known, estimate depth of all common scene points.

Two images of Mars

CS252A, Fall 2012 Computer Vision I

An Application: Mobile Robot Navigation

The Stanford Cart, H. Moravec, 1979.

The INRIA Mobile Robot, 1990.

Courtesy O. Faugeras and H. Moravec.

2

CS252A, Fall 2012 Computer Vision I

Mars Exploratory Rovers: Spirit and Opportunity

CS252A, Fall 2012 Computer Vision I

Commercial Stereo Heads

CS252A, Fall 2012 Computer Vision I

Need for correspondence

Truco Fig. 7.5

CS252A, Fall 2012 Computer Vision I

Triangulation

Nalwa Fig. 7.2

CS252A, Fall 2012 Computer Vision I

Stereo Vision Outline •  Offline: Calibrate cameras & determine

“epipolar geometry” •  Online

1.  Acquire stereo images 2.  Rectify images to convenient epipolar geometry 3.  Establish correspondence 4.  Estimate depth A

B

C

D

CS252A, Fall 2012 Computer Vision I

BINOCULAR STEREO SYSTEM Estimating Depth\

2D world with 1-D image plane Two measurements: XL, XR Two unknowns: X,Z

Constants: Baseline: d Focal length: f

Disparity: (XL - XR)

Z = d f (XL - XR)

X = d XL (XL - XR)

(Adapted from Hager)

Z

X (0,0) (d,0)

Z=f

(X,Z)

XL XR

XL=f(X/Z) XR=f((X-d)/Z)

3

CS252A, Fall 2012 Computer Vision I

Reconstruction: General 3-D case

•  Linear Method: find P such that Where M is camera matrix

•  Non-Linear Method: find Q minimizing where q=MQ and q’=M’Q

Given two image measurements p and p’, estimate P.

M M’

CS252A, Fall 2012 Computer Vision I

Two Approaches 1.  Feature-Based

–  From each image, process “monocular” image to obtain cues (e.g., corners, lines).

–  Establish correspondence between

2.  Area-Based –  Directly compare image regions between the

two images.

CS252A, Fall 2012 Computer Vision I

Human Stereopsis: Binocular Fusion

How are the correspondences established?

Julesz (1971): Is the mechanism for binocular fusion a monocular process or a binocular one?? •  There is anecdotal evidence for the latter (camouflage).

•  Random dot stereograms provide an objective answer CS252A, Fall 2012 Computer Vision I

Random Dot Stereograms

CS252A, Fall 2012 Computer Vision I

Random Dot Stereograms

CS252A, Fall 2012 Computer Vision I

Was Rembrandt Stereo Blind? •  Detail of a 1639 etching.

4

CS252A, Fall 2012 Computer Vision I

•  In Rembrandt's painted self-portraits (left panel) in which the eyes are clearly visible, his left eye frequently looks straight out and the right off to the side. It is the opposite in his etchings (right panel).

CS252A, Fall 2012 Computer Vision I

Need for correspondence

Truco Fig. 7.5

CS252A, Fall 2012 Computer Vision I

Epipolar Constraint

•  Potential matches for p have to lie on the corresponding epipolar line l’.

•  Potential matches for p’ have to lie on the corresponding epipolar line l.

CS252A, Fall 2012 Computer Vision I

Epipolar Geometry

•  Epipolar Plane

•  Epipoles •  Epipolar Lines

•  Baseline

CS252A, Fall 2012 Computer Vision I

Family of epipolar Planes

Family of planes π and lines l and l’ Intersection in e and e’

O O’

CS252A, Fall 2012 Computer Vision I

Skew Symmetric Matrix & Cross Product

•  The cross product a x b of two vectors a and b can be expressed a matrix vector product [ax]b where[ax] is the skew symmetric matrix:

•  A matrix S is skew symmetric iff S = -ST €

a×[ ] =

0 −a3 a2a3 0 −a1−a2 a1 0

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

5

CS252A, Fall 2012 Computer Vision I

Epipolar Constraint: Calibrated Case

Essential Matrix (Longuet-Higgins, 1981)

CS252A, Fall 2012 Computer Vision I

Calibration

Determine intrinsic parameters and extrinsic relation of two cameras