ai final presentation- an introduction to feature tracking
TRANSCRIPT
An Introduction to Feature Tracking
N. D.
Outline
➔ Introduction: What is feature tracking?
➔ Features
➔ Feature Tracking Algorithm
➔ Alternatives
➔ Applications
➔ Q & A
Introduction: Motivation
How does object tracking work?
How do you determine the
similarity between images?
Introduction: Preview
You have two images. How does the first map onto the second?
Introduction: Feature Tracking
From Wikipedia: Motion estimation is...
“Process of determining motion vectors that describe the transformation
from one 2D image to another, usually from adjacent frames in a video
sequence. ”
Features: Definition
Official Definition:
“a n-tuple or vector whose components are functions of the initial
measurement pattern variables or some subset of the initial
measurement pattern variables.”
Human-Speak Definition:
A location on an image where you track change.
Features
(A location on an image where you track change
==
Any location) ?
Features
(A location on an image where you track change
==
Any location) ?
true
Features
What is a good feature?
Features: An Example
(1)
(3)
(5)
(2)
(4)
(6)
Features: Computed Answer
Features: Computed Answer
Features: The Good, the Bad, the Ugly
A good feature should be locally unique and invariant to geometric
transformations.
Good Bad Ugly
Features: Local Uniqueness
To determine local uniqueness, look at sub-parts of the feature.
Good
Bad
Ugly
. . .
. . .
. . .
Tracking: Algorithm Overview
Inputs: (a: Image, b: Image)
Step 1: vectora = Extract Features ( a ) vectorb = Extract Features ( b )
Step 2: matches = Compute Difference ( vectora , vectorb ) Step 3: result = Eliminate False Positives ( matches )
Tracking: Shoes Example
Inputs:
Tracking: Shoes Example
Output: Inputs:
Tracking: Shoes Example --- Step 1
Step 1: Feature Extraction using the Harris Operator
Tracking: Shoes Example --- Corner Detection
The Harris Operator is used in corner detection.
Tracking: Shoes Example --- Harris Points
Harris Operator output: bright points = locally unique candidates
Tracking: Shoes Example --- Choosing Points
Choose the brightest points that are at least delta distance apart.
Tracking: Shoes Example --- Choosing Points
Same result, on the colored images:
Tracking: Shoes Example --- Harris Equation
where ➔ W is a weights kernel (Gaussian kernel; smoothing) ➔ Ix is the horizontal Sobel operator (horizontal edge detection) ➔ Iy is the vertical Sobel operator (vertical edge detection)
Tracking: Shoes Example --- Sobel Operator
Sobelx and Sobely applied to Image a
[ Horizontal edge detection ] [ Vertical edge detection ]
Tracking: Shoes Example --- Step 2
Step 2: Feature Matching using the Sum of Squared Differences (alternative: Normalized Cross Correlation) to measure error and find
similarity between images.
Tracking: Shoes Example --- SSD
Low error measurements between features means a potential match.
Tracking: Shoes Example --- SSD Equation
where
➔ E(u, v) is the error measurement at (u, v) compared to (x, y) ➔ W(x, y) is a weights kernel (Gaussian kernel; smoothing) ➔ I(x, y) is the Sobel operator (edge detection)
Tracking: Shoes Example --- Step 3
Step 3: Eliminate False Positives using the Random Sample Consensus (RANSAC) method
Tracking: Shoes Example --- RANSAC
“An iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers. ”
Tracking: Flowers Example
Example #2:
Tracking: Museum Example
Example #3:
(A Few) Alternatives
➔ Pixel-by-pixel computation (horribly inefficient)
➔ Relational matching
◆ Bag-of-Words
◆ Both generative or discriminative models
➔ Particle Filter
➔ Multi-Scale Oriented Patches
➔ Eigenfaces / Space of Faces
Applications: Feature Extraction & Tracking
➔ Film ◆ Video tracking ◆ Replacement sets or characters (ex: Gollum) ◆ Motion tracking markers
➔ Biometrics
◆ Facial recognition (Eigenfaces) ◆ Xbox Kinect (?)
Questions ? Ask : Clap
(You may clap.)
Sources Gollum Motion Estimation - Wiki Ransac - Wiki Car with Tracking Markers Motion Tracking Markers Joffrey Clapping GIF Glossary of Computer Vision Terms Skeleton code for reading in/writing out images from ColorFront Flowers Pictures from ColorFront Computer Vision: Algorithms and Applications by Richard Szeliski