john a. bender applications in real-time 3d tracking collaborators: pietro perona luis goncalves ken...
Post on 20-Dec-2015
213 views
TRANSCRIPT
John A. Bender
Applications in Real-time 3D Tracking
Collaborators:Pietro Perona
Luis GoncalvesKen Goldberg
Karl ChenIlan Lobel
Steve Nowlin
Camera Design
Orange Micro iBOT IEEE 1394(FireWire) webcam - $100 custom modifications
• 3-port FireWire PCI card has total bandwidth of 400 Mb/s - $100
• 320x240 pixels, 3 color channels, 8 bits/pixel, 30 fps, 3 cameras= 166 Mb/s
• isochronous sampling??
• processing time??
Camera Modeling
focal lengthprincipal
point
parameters of interest…
camera matrix:
Fx 0 Px
0 Fy Py
0 0 1
point on image plane: [ (x-Px)/Fx (y-Py)/Fy 1 ] (homogeneous coordinate)
3D Modeling
transformations between camerasare modeled with rotation/translationcomponents
translation vector: [ Tx Ty Tz ]rotation vector: [ Rx Ry Rz ]
rotation matrix:X0X1 X0Y1 X0Z1
Y0X1 Y0Y1 Y0Z1
Z0X1 Z0Y1 Z0Z1
3D Modeling
transform rays into a common coordinate system and intersect
transformation for a homogeneous 3D point (z=1): X0 = R10*X1 + T10
Image Processing
• Microsoft DirectX (DirectShow) interfaces with hardware
• image callback executed on every frame with timestamp,each camera runs in a separate thread
• background modeling / foreground masking withIntel Performance Primitives optimized image processing libraries
• ellipse fitting using Intel Open Computer Vision libraries
• triangulation including ellipse size
• fish determination
• TCP network streaming
• executes at ~20 Hz on a Dell 2.8 GHz Pentium 4 with1.5 GB of 533 MHz DDR SDRAM
Path Determination
each point is a 3D point whichcould contain a fish
each column is a different time point
extend each path to all possible positions
cull paths with too-low traversal probability
Path Determination
add dummy node with fixed transitionprobability in case of disappearances
place dummy node at last known position
Path Determination
Rendering
• virtual fish tank
• each virtual fish follows the position of a real fish
• Newtonian motion model for virtual fish ensures smooth movement
• camera position in virtual tank set to head of gold fish