Passive Object Tracking from Stereo Vision

Michael H. Rosenthal

May 1, 2000

Outline

• Purpose

• Camera Calibration

• Data Collection

• Object Tracking

• Depth from Stereo

• Results

• Conclusions

Purpose

• 3D visualization - blood vessels, organs, teapots

• Stereo displays require head tracking

• Cumbersome trackers are undesirable

Purpose

• Passive tracking offers tetherless option

• Speed and accuracy is a concern

How To Do It

• Mount reflectors on tracking target

• Calibrate pair of cameras

• Identify reflectors in each image

• Calculate positions using stereo disparity

• Update tracking model using new positions

Camera Calibration

• Select six points on known calibration target

• Solve for six extrinsic and four intrinsic parameters (ignore distortion)

• Used derivative of Tsai’s

calibration code (www.cs.cmu.edu/~rgw/TsaiDesc.html)

• Thanks to Herman Towles and Ruigang Yang of Office of the Future for adapted code

Data Collection

• Mounted target on optical rail

• Translated target at 1mm per frame over 20cm

• Rotated target through 120 degrees

Object Tracking

• We want to follow an object through a scene

• Challenge - what targets are best for tracking?

• Spheres yield spatially ambiguous results

• Complex shapes limit ambiguity but are hard to track

Model of glasses with tracking targets

Object Tracking

• I chose a square and a rectangle - unambiguous, but surprisingly hard to track

• Square is easy - calculate centroid as weighted average of position and intensity

• Rectangle is the problem - need center and orientation, makes problem non-trivial Model of glasses with tracking targets

Object Tracking

• Calculate centroid of rectangle

• Search for shortest axis at some angle through centroid

• Find edges along short and long axes using derivatives

• Use endpoints of long axes as tracking targets

• Use prior results for future frames

Model of glasses with tracking targets

Depth from Stereo

• Use point pairings to get depth

• Find shortest segment between the two pixel rays

• Use midpoint as position estimate

• Trucco section 7.4 describes equations

Results

Translation• Moved stage by

210mm• Average measured

translation: 178mm• 16% error

Rotation• Rotated target by 60

degrees• Averaged measured

rotation: 70 degrees• 16% error

Neither showed high noise, so systematic error is likely (calibration?)

Conclusions

• Camera calibration is somewhat challenging

• Good camera calibration is very challenging

• Robust object tracking will require significant development

• Simplified targets will reduce complexity